00002000:00010000:3.0:1789760743.132425:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 54747136, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x1a1c8 00010000:00010000:3.0:1789760743.132435:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c17b0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.132450:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760743.132455:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.132463:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c17b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54747136->54751231] (req 54747136->54751231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.132477:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c17b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54747136->54751231] (req 54747136->54751231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760743.137830:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 54747136 -> 54751232 00002000:00010000:3.0:1789760743.137835:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 106952 -> 106960 00010000:00010000:3.0:1789760743.137847:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c17b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54747136->54751231] (req 54747136->54751231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.137859:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c17b0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54747136->54751231] (req 54747136->54751231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.137870:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c17b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54747136->54751231] (req 54747136->54751231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.137878:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760743.137883:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c17b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54747136->54751231] (req 54747136->54751231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.137891:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c17b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54747136->54751231] (req 54747136->54751231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.137910:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c17b0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54747136->54751231] (req 54747136->54751231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.137919:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c17b0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54747136->54751231] (req 54747136->54751231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.138154:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554472448:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5812us (6493us total) trans 4294980675 rc 0/0 00000100:00100000:3.0:1789760743.138165:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13377 00000100:00100000:3.0:1789760743.138175:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686366898225 time 686377940091 state 0x2 ref 0 00000100:00100000:3.0:1789760743.142658:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554472576 00000100:00100000:3.0:1789760743.142668:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554472576 00000100:00100000:3.0:1789760743.142721:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686366898225 time 686377940091 state 0x0 ref 1 00000100:00100000:3.0:1789760743.142737:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686378200507 time 686377940091 state 0x1 ref 1 00000100:00100000:3.0:1789760743.142769:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686378200507 time 686388387384 state 0x0 ref 1 00000100:00100000:3.0:1789760743.142781:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554472576:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760743.142831:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 54751232, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x1a1d0 00010000:00010000:3.0:1789760743.142842:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.142859:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760743.142864:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.142871:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54751232->54755327] (req 54751232->54755327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.142885:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54751232->54755327] (req 54751232->54755327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760743.151473:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 54751232 -> 54755328 00002000:00010000:3.0:1789760743.151478:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 106960 -> 106968 00010000:00010000:3.0:1789760743.151491:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54751232->54755327] (req 54751232->54755327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.151504:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54751232->54755327] (req 54751232->54755327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.151513:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54751232->54755327] (req 54751232->54755327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.151520:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760743.151525:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54751232->54755327] (req 54751232->54755327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.151534:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17b7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54751232->54755327] (req 54751232->54755327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.151552:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17b7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54751232->54755327] (req 54751232->54755327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.151564:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17b7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54751232->54755327] (req 54751232->54755327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.151657:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554472576:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8872us (9088us total) trans 4294980676 rc 0/0 00000100:00100000:3.0:1789760743.151667:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13378 00000100:00100000:3.0:1789760743.151676:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686378200507 time 686388387384 state 0x2 ref 0 00000100:00100000:3.0:1789760743.166850:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554472704 00000100:00100000:3.0:1789760743.166862:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554472704 00000100:00100000:3.0:1789760743.166917:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686378200507 time 686388387384 state 0x0 ref 1 00000100:00100000:3.0:1789760743.166933:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686388647800 time 686388387384 state 0x1 ref 1 00000100:00100000:3.0:1789760743.166964:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686388647800 time 686412582206 state 0x0 ref 1 00000100:00100000:3.0:1789760743.166986:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554472704:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760743.167030:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 54755328, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x1a1d8 00010000:00010000:3.0:1789760743.167039:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17be lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.167054:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760743.167057:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.167063:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54755328->54759423] (req 54755328->54759423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.167081:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54755328->54759423] (req 54755328->54759423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760743.175598:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 54755328 -> 54759424 00002000:00010000:3.0:1789760743.175603:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 106968 -> 106976 00010000:00010000:3.0:1789760743.175614:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54755328->54759423] (req 54755328->54759423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.175626:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17be lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54755328->54759423] (req 54755328->54759423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.175635:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54755328->54759423] (req 54755328->54759423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.175642:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760743.175647:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54755328->54759423] (req 54755328->54759423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.175656:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17be lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54755328->54759423] (req 54755328->54759423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.175674:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17be lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54755328->54759423] (req 54755328->54759423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.175681:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17be lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54755328->54759423] (req 54755328->54759423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.175895:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554472704:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8914us (9183us total) trans 4294980677 rc 0/0 00000100:00100000:3.0:1789760743.175905:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13379 00000100:00100000:3.0:1789760743.175915:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686388647800 time 686412582206 state 0x2 ref 0 00000100:00100000:3.0:1789760743.186874:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554472832 00000100:00100000:3.0:1789760743.186885:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554472832 00000100:00100000:3.0:1789760743.186941:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686388647800 time 686412582206 state 0x0 ref 1 00000100:00100000:3.0:1789760743.186956:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686412842622 time 686412582206 state 0x1 ref 1 00000100:00100000:3.0:1789760743.186995:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686412842622 time 686432613459 state 0x0 ref 1 00000100:00100000:3.0:1789760743.187008:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554472832:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760743.187054:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 54759424, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x1a1e0 00010000:00010000:3.0:1789760743.187068:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c17c5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.187085:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760743.187088:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.187094:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c17c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54759424->54763519] (req 54759424->54763519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.187107:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c17c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54759424->54763519] (req 54759424->54763519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760743.196211:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 54759424 -> 54763520 00002000:00010000:3.0:1789760743.196216:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 106976 -> 106984 00010000:00010000:3.0:1789760743.196229:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c17c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54759424->54763519] (req 54759424->54763519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.196241:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c17c5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54759424->54763519] (req 54759424->54763519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.196249:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c17c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54759424->54763519] (req 54759424->54763519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.196255:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760743.196260:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c17c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54759424->54763519] (req 54759424->54763519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.196267:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c17c5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54759424->54763519] (req 54759424->54763519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.196287:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c17c5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54759424->54763519] (req 54759424->54763519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.196294:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c17c5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54759424->54763519] (req 54759424->54763519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.196642:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554472832:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9631us (9988us total) trans 4294980678 rc 0/0 00000100:00100000:3.0:1789760743.196653:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13380 00000100:00100000:3.0:1789760743.196662:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686412842622 time 686432613459 state 0x2 ref 0 00000001:00080000:3.0:1789760743.207890:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980679 is committed 00000100:00100000:3.0:1789760743.217013:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554473088 00000100:00100000:3.0:1789760743.217022:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554473088 00000100:00100000:3.0:1789760743.217075:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686432873875 time 686447084603 state 0x0 ref 1 00000100:00100000:3.0:1789760743.217091:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686447345019 time 686447084603 state 0x1 ref 1 00000100:00100000:3.0:1789760743.217122:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686447345019 time 686462740315 state 0x0 ref 1 00000100:00100000:3.0:1789760743.217133:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554473088:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760743.217181:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 54767616, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x1a1f0 00010000:00010000:3.0:1789760743.217190:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c17d3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.217206:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760743.217210:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.217216:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c17d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54767616->54771711] (req 54767616->54771711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.217230:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c17d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54767616->54771711] (req 54767616->54771711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.239980:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760743.275003:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980683 is committed 00000100:00100000:3.0:1789760743.281919:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760743.293928:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980684 is committed 00000100:00100000:3.0:1789760743.306118:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554473728 00000100:00100000:3.0:1789760743.306127:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554473728 00000100:00100000:3.0:1789760743.306181:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686515220249 time 686530664350 state 0x0 ref 1 00000100:00100000:3.0:1789760743.306197:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686530924766 time 686530664350 state 0x1 ref 1 00000100:00100000:3.0:1789760743.306285:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686530924766 time 686551902381 state 0x0 ref 1 00000100:00100000:3.0:1789760743.306299:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554473728:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760743.306359:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 54788096, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x1a218 00010000:00010000:3.0:1789760743.306369:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17f6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.306613:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760743.306619:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.306625:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54788096->54792191] (req 54788096->54792191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.306640:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54788096->54792191] (req 54788096->54792191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760743.315201:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 54788096 -> 54792192 00002000:00010000:3.0:1789760743.315207:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 107032 -> 107040 00010000:00010000:3.0:1789760743.315220:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54788096->54792191] (req 54788096->54792191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.315233:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17f6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54788096->54792191] (req 54788096->54792191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.315242:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54788096->54792191] (req 54788096->54792191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.315248:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760743.315252:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54788096->54792191] (req 54788096->54792191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.315261:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17f6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54788096->54792191] (req 54788096->54792191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.315283:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17f6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54788096->54792191] (req 54788096->54792191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.315291:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c17f6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54788096->54792191] (req 54788096->54792191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.316201:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554473728:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9896us (10197us total) trans 4294980685 rc 0/0 00000100:00100000:3.0:1789760743.316214:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13387 00000100:00100000:3.0:1789760743.316223:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686530924766 time 686551902381 state 0x2 ref 0 00000100:00100000:3.0:1789760743.320300:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554473856 00000100:00100000:3.0:1789760743.320312:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554473856 00000100:00100000:3.0:1789760743.320524:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686530924766 time 686551902381 state 0x0 ref 1 00000100:00100000:3.0:1789760743.320541:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686552162797 time 686551902381 state 0x1 ref 1 00000100:00100000:3.0:1789760743.335039:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554473984 00000100:00100000:3.0:1789760743.335049:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554473984 00000100:00100000:3.0:1789760743.335104:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686552162797 time 686566273510 state 0x0 ref 1 00000100:00100000:3.0:1789760743.335120:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686566533926 time 686566273510 state 0x1 ref 1 00000100:00100000:3.0:1789760743.335205:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686566533926 time 686580822671 state 0x0 ref 1 00000100:00100000:3.0:1789760743.335219:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554473984:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760743.335264:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 54796288, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x1a228 00010000:00010000:3.0:1789760743.335273:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c1804 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.335289:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760743.335294:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.336279:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c1804 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54796288->54800383] (req 54796288->54800383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.336307:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c1804 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54796288->54800383] (req 54796288->54800383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.370219:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554474240 00000100:00100000:3.0:1789760743.370229:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554474240 00000100:00100000:3.0:1789760743.370282:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686581083087 time 686603652044 state 0x0 ref 1 00000100:00100000:3.0:1789760743.370406:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686603912460 time 686603652044 state 0x1 ref 1 00000100:00100000:3.0:1789760743.370448:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686603912460 time 686616066053 state 0x0 ref 1 00000100:00100000:3.0:1789760743.370465:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554474240:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760743.370513:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 54804480, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x1a238 00010000:00010000:3.0:1789760743.370522:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c1812 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.370537:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760743.370542:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.370549:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c1812 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54804480->54808575] (req 54804480->54808575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.370562:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c1812 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54804480->54808575] (req 54804480->54808575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760743.379651:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980689 is committed 00000100:00100000:3.0:1789760743.403637:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554474496 00000100:00100000:3.0:1789760743.403647:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554474496 00000100:00100000:3.0:1789760743.403699:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686616326469 time 686638162878 state 0x0 ref 1 00000100:00100000:3.0:1789760743.403713:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686638423294 time 686638162878 state 0x1 ref 1 00000100:00100000:3.0:1789760743.403752:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686638423294 time 686649370284 state 0x0 ref 1 00000100:00100000:3.0:1789760743.403768:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554474496:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760743.403819:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 54812672, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x1a248 00010000:00010000:3.0:1789760743.403829:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c1820 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.403844:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760743.403848:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.403855:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c1820 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54812672->54816767] (req 54812672->54816767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.403867:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c1820 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54812672->54816767] (req 54812672->54816767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760743.409876:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 54812672 -> 54816768 00002000:00010000:3.0:1789760743.409881:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 107080 -> 107088 00010000:00010000:3.0:1789760743.409894:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c1820 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54812672->54816767] (req 54812672->54816767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.409907:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c1820 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54812672->54816767] (req 54812672->54816767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.409915:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c1820 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54812672->54816767] (req 54812672->54816767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.409922:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760743.409928:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c1820 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54812672->54816767] (req 54812672->54816767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.409937:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c1820 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54812672->54816767] (req 54812672->54816767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.409957:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c1820 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54812672->54816767] (req 54812672->54816767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.409965:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c1820 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54812672->54816767] (req 54812672->54816767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.410206:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554474496:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6294us (6696us total) trans 4294980691 rc 0/0 00000100:00100000:3.0:1789760743.410217:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13393 00000100:00100000:3.0:1789760743.410227:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686638423294 time 686649370284 state 0x2 ref 0 00002000:00010000:3.0:1789760743.444294:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 54820864 -> 54824960 00002000:00010000:3.0:1789760743.444300:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 107096 -> 107104 00010000:00010000:3.0:1789760743.444313:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c182e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54820864->54824959] (req 54820864->54824959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.444324:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c182e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54820864->54824959] (req 54820864->54824959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.444333:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c182e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54820864->54824959] (req 54820864->54824959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.444518:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760743.444525:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c182e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54820864->54824959] (req 54820864->54824959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.444535:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c182e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54820864->54824959] (req 54820864->54824959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.444555:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c182e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54820864->54824959] (req 54820864->54824959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.444562:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c182e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54820864->54824959] (req 54820864->54824959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.444656:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09312a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554474752:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7098us (7547us total) trans 4294980693 rc 0/0 00000100:00100000:3.0:1789760743.444668:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13395 00000100:00100000:3.0:1789760743.444677:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686663780141 time 686683160097 state 0x2 ref 0 00000100:00100000:3.0:1789760743.463874:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554475008 00000100:00100000:3.0:1789760743.463883:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554475008 00000100:00100000:3.0:1789760743.463951:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686683420513 time 686695080273 state 0x0 ref 1 00000100:00100000:3.0:1789760743.463967:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686695340689 time 686695080273 state 0x1 ref 1 00000100:00100000:3.0:1789760743.464012:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686695340689 time 686709630560 state 0x0 ref 1 00000100:00100000:3.0:1789760743.464028:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09312680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554475008:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760743.464081:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 54829056, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x1a268 00010000:00010000:3.0:1789760743.464090:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c183c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.464115:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760743.464119:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.464126:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c183c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54829056->54833151] (req 54829056->54833151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.464140:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c183c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54829056->54833151] (req 54829056->54833151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760743.469802:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 54829056 -> 54833152 00002000:00010000:3.0:1789760743.469807:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 107112 -> 107120 00010000:00010000:3.0:1789760743.469821:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c183c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54829056->54833151] (req 54829056->54833151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.469833:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c183c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54829056->54833151] (req 54829056->54833151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.469842:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c183c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54829056->54833151] (req 54829056->54833151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.469849:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760743.469854:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c183c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54829056->54833151] (req 54829056->54833151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.469863:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c183c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54829056->54833151] (req 54829056->54833151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.469881:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c183c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54829056->54833151] (req 54829056->54833151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.469889:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c183c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54829056->54833151] (req 54829056->54833151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.470098:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09312680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554475008:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6068us (6342us total) trans 4294980695 rc 0/0 00000100:00100000:3.0:1789760743.470110:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13397 00000100:00100000:3.0:1789760743.470119:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686695340689 time 686709630560 state 0x2 ref 0 00000100:00100000:3.0:1789760743.510920:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554475520 00000100:00100000:3.0:1789760743.510929:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554475520 00000100:00100000:3.0:1789760743.510991:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686730030712 time 686744456557 state 0x0 ref 1 00000100:00100000:3.0:1789760743.511005:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686744716973 time 686744456557 state 0x1 ref 1 00000100:00100000:3.0:1789760743.511039:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686744716973 time 686756657082 state 0x0 ref 1 00000100:00100000:3.0:1789760743.511050:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09310700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554475520:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760743.511092:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 54845440, mtime: 0x6aad94e6, atime: 0x0, ctime: 0x6aad94e6, blocks: 0x1a288 00010000:00010000:3.0:1789760743.511102:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c1858 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.511118:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760743.511122:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.511128:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c1858 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54845440->54849535] (req 54845440->54849535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.511139:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c1858 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54845440->54849535] (req 54845440->54849535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.550030:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760743.590104:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760743.634520:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:3.0:1789760743.654304:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 54886400 -> 54890496 00002000:00010000:3.0:1789760743.654310:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 107224 -> 107232 00010000:00010000:3.0:1789760743.654323:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c189e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54886400->54890495] (req 54886400->54890495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.654336:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c189e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54886400->54890495] (req 54886400->54890495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.654353:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c189e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54886400->54890495] (req 54886400->54890495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.654361:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760743.654366:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c189e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54886400->54890495] (req 54886400->54890495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.654593:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c189e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54886400->54890495] (req 54886400->54890495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.654624:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c189e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54886400->54890495] (req 54886400->54890495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.654633:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c189e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54886400->54890495] (req 54886400->54890495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.654729:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554476800:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8447us (8678us total) trans 4294980709 rc 0/0 00000100:00100000:3.0:1789760743.654739:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13411 00000100:00100000:3.0:1789760743.654748:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686880941485 time 686891885419 state 0x2 ref 0 00000100:00100000:3.0:1789760743.707627:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554477312 00000100:00100000:3.0:1789760743.707636:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554477312 00000100:00100000:3.0:1789760743.707691:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686922110227 time 686937742667 state 0x0 ref 1 00000100:00100000:3.0:1789760743.707706:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686938003083 time 686937742667 state 0x1 ref 1 00000100:00100000:3.0:1789760743.707746:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686938003083 time 686953365099 state 0x0 ref 1 00000100:00100000:3.0:1789760743.707759:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09312680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554477312:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760743.707807:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 54902784, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0x1a2f8 00010000:00010000:3.0:1789760743.707817:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c18ba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.707844:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=686s v=(0 0) 00010000:00010000:3.0:1789760743.707848:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.707854:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c18ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54902784->54906879] (req 54902784->54906879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.707868:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c18ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54902784->54906879] (req 54902784->54906879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760743.714989:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 54902784 -> 54906880 00002000:00010000:3.0:1789760743.714994:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 107256 -> 107264 00010000:00010000:3.0:1789760743.715007:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c18ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54902784->54906879] (req 54902784->54906879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.715021:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c18ba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54902784->54906879] (req 54902784->54906879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.715031:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c18ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54902784->54906879] (req 54902784->54906879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.715039:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760743.715044:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c18ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54902784->54906879] (req 54902784->54906879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.715053:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c18ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54902784->54906879] (req 54902784->54906879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.715078:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c18ba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54902784->54906879] (req 54902784->54906879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.715086:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c18ba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54902784->54906879] (req 54902784->54906879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.715184:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09312680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554477312:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7423us (7743us total) trans 4294980713 rc 0/0 00000100:00100000:3.0:1789760743.715196:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13415 00000100:00100000:3.0:1789760743.715205:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 686938003083 time 686953365099 state 0x2 ref 0 00000100:00100000:3.0:1789760743.767754:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760743.803101:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760743.817288:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554478336 00000100:00100000:3.0:1789760743.817298:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554478336 00000100:00100000:3.0:1789760743.817352:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 687037435085 time 687048960518 state 0x0 ref 1 00000100:00100000:3.0:1789760743.817367:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 687049220934 time 687048960518 state 0x1 ref 1 00000100:00100000:3.0:1789760743.817656:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 687049220934 time 687063272030 state 0x0 ref 1 00000100:00100000:3.0:1789760743.817670:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554478336:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760743.817725:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 54935552, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0x1a338 00010000:00010000:3.0:1789760743.817736:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c18f2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.817759:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760743.817764:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.817771:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c18f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54935552->54939647] (req 54935552->54939647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.817787:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c18f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54935552->54939647] (req 54935552->54939647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760743.826214:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 54935552 -> 54939648 00002000:00010000:3.0:1789760743.826221:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 107320 -> 107328 00010000:00010000:3.0:1789760743.826240:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c18f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54935552->54939647] (req 54935552->54939647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.826265:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c18f2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54935552->54939647] (req 54935552->54939647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.826275:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c18f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54935552->54939647] (req 54935552->54939647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.826283:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760743.826288:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c18f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54935552->54939647] (req 54935552->54939647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.826298:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c18f2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54935552->54939647] (req 54935552->54939647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.826333:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c18f2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54935552->54939647] (req 54935552->54939647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.826342:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c18f2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54935552->54939647] (req 54935552->54939647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.826462:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554478336:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8791us (9348us total) trans 4294980721 rc 0/0 00000100:00100000:3.0:1789760743.826471:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13423 00000100:00100000:3.0:1789760743.826481:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 687049220934 time 687063272030 state 0x2 ref 0 00000100:00100000:3.0:1789760743.831818:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554478464 00000100:00100000:3.0:1789760743.831828:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554478464 00000100:00100000:3.0:1789760743.831880:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 687049220934 time 687063272030 state 0x0 ref 1 00000100:00100000:3.0:1789760743.831895:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 687063532446 time 687063272030 state 0x1 ref 1 00000100:00100000:3.0:1789760743.831926:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 687063532446 time 687077544785 state 0x0 ref 1 00000100:00100000:3.0:1789760743.831937:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554478464:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760743.831993:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 54939648, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0x1a340 00010000:00010000:3.0:1789760743.832002:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c18f9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.832017:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760743.832021:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.832028:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c18f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54939648->54943743] (req 54939648->54943743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.832042:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c18f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54939648->54943743] (req 54939648->54943743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.832231:0:3646:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760743.832247:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb09313800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573900544:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760743.832299:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760743.833139:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb09313800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573900544:0@lo:13:osp-pre-0-0.0 00002000:00010000:3.0:1789760743.841499:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 54939648 -> 54943744 00002000:00010000:3.0:1789760743.841505:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 107328 -> 107336 00010000:00010000:3.0:1789760743.841518:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c18f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54939648->54943743] (req 54939648->54943743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.841531:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c18f9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54939648->54943743] (req 54939648->54943743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.841540:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c18f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54939648->54943743] (req 54939648->54943743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.841548:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760743.841553:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c18f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54939648->54943743] (req 54939648->54943743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.841561:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c18f9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54939648->54943743] (req 54939648->54943743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.841580:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c18f9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54939648->54943743] (req 54939648->54943743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.841588:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c18f9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54939648->54943743] (req 54939648->54943743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.841830:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554478464:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9889us (10491us total) trans 4294980722 rc 0/0 00000100:00100000:3.0:1789760743.841840:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13424 00000100:00100000:3.0:1789760743.841850:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 687063532446 time 687077544785 state 0x2 ref 0 00000100:00100000:3.0:1789760743.874516:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554478848 00000100:00100000:3.0:1789760743.874530:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554478848 00000100:00100000:3.0:1789760743.874587:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 687092237935 time 687105967182 state 0x0 ref 1 00000100:00100000:3.0:1789760743.874603:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 687106227598 time 687105967182 state 0x1 ref 1 00000100:00100000:3.0:1789760743.874643:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 687106227598 time 687120252743 state 0x0 ref 1 00000100:00100000:3.0:1789760743.874657:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09313480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554478848:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760743.874715:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 54951936, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0x1a358 00010000:00010000:3.0:1789760743.874725:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c190e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.874742:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760743.874747:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.874753:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c190e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54951936->54956031] (req 54951936->54956031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.874768:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c190e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54951936->54956031] (req 54951936->54956031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760743.884857:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 54951936 -> 54956032 00002000:00010000:3.0:1789760743.884864:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 107352 -> 107360 00010000:00010000:3.0:1789760743.884877:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c190e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54951936->54956031] (req 54951936->54956031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.884890:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c190e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54951936->54956031] (req 54951936->54956031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.884899:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c190e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54951936->54956031] (req 54951936->54956031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.884906:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760743.884912:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c190e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54951936->54956031] (req 54951936->54956031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.884922:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c190e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54951936->54956031] (req 54951936->54956031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.884941:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c190e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54951936->54956031] (req 54951936->54956031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.884955:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c190e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54951936->54956031] (req 54951936->54956031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.885078:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09313480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554478848:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10420us (12172us total) trans 4294980725 rc 0/0 00000100:00100000:3.0:1789760743.885089:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13427 00000100:00100000:3.0:1789760743.885100:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 687106227598 time 687120252743 state 0x2 ref 0 00000100:00100000:3.0:1789760743.890064:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554478976 00000100:00100000:3.0:1789760743.890074:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554478976 00000100:00100000:3.0:1789760743.890127:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 687106227598 time 687120252743 state 0x0 ref 1 00000100:00100000:3.0:1789760743.890143:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 687120513159 time 687120252743 state 0x1 ref 1 00000100:00100000:3.0:1789760743.890175:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 687120513159 time 687135794139 state 0x0 ref 1 00000100:00100000:3.0:1789760743.890187:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554478976:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760743.890232:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 54956032, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0x1a360 00010000:00010000:3.0:1789760743.890241:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c1915 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.890258:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760743.890263:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.890269:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c1915 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54956032->54960127] (req 54956032->54960127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.890287:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c1915 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54956032->54960127] (req 54956032->54960127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.943928:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554479360 00000100:00100000:3.0:1789760743.943938:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554479360 00000100:00100000:3.0:1789760743.944003:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 687153308208 time 687175548693 state 0x0 ref 1 00000100:00100000:3.0:1789760743.944019:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 687175809109 time 687175548693 state 0x1 ref 1 00000100:00100000:3.0:1789760743.944243:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 687175809109 time 687189860762 state 0x0 ref 1 00000100:00100000:3.0:1789760743.944258:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554479360:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760743.944310:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 54968320, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0x1a378 00010000:00010000:3.0:1789760743.944320:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c192a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.944337:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760743.944355:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.944363:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c192a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54968320->54972415] (req 54968320->54972415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.944408:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c192a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54968320->54972415] (req 54968320->54972415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760743.950572:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 54968320 -> 54972416 00002000:00010000:3.0:1789760743.950578:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 107384 -> 107392 00010000:00010000:3.0:1789760743.950590:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c192a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54968320->54972415] (req 54968320->54972415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.950603:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c192a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54968320->54972415] (req 54968320->54972415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.950625:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c192a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54968320->54972415] (req 54968320->54972415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.950633:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760743.950639:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c192a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54968320->54972415] (req 54968320->54972415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.950648:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c192a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54968320->54972415] (req 54968320->54972415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.950668:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c192a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54968320->54972415] (req 54968320->54972415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.950676:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c192a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54968320->54972415] (req 54968320->54972415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760743.950773:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554479360:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6514us (7014us total) trans 4294980729 rc 0/0 00000100:00100000:3.0:1789760743.950783:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13431 00000100:00100000:3.0:1789760743.950794:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 687175809109 time 687189860762 state 0x2 ref 0 00000100:00100000:3.0:1789760743.998688:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554479872 00000100:00100000:3.0:1789760743.998976:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554479872 00000100:00100000:3.0:1789760743.999033:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 687219823459 time 687232273580 state 0x0 ref 1 00000100:00100000:3.0:1789760743.999048:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 687232533996 time 687232273580 state 0x1 ref 1 00000100:00100000:3.0:1789760743.999119:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 687232533996 time 687244736777 state 0x0 ref 1 00000100:00100000:3.0:1789760743.999131:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554479872:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760743.999176:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 54984704, mtime: 0x6aad94e7, atime: 0x0, ctime: 0x6aad94e7, blocks: 0x1a398 00010000:00010000:3.0:1789760743.999185:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c1946 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.999201:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=687s v=(0 0) 00010000:00010000:3.0:1789760743.999205:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760743.999211:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c1946 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54984704->54988799] (req 54984704->54988799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760743.999224:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c1946 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [54984704->54988799] (req 54984704->54988799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760744.018185:0:8453:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760744.027601:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760744.057625:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980737 is committed 00000100:00100000:3.0:1789760744.062645:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760744.068577:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980738 is committed 00000100:00100000:3.0:1789760744.103306:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760744.113471:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980741 is committed 00000001:00080000:3.0:1789760744.127186:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980742 is committed 00000001:00080000:3.0:1789760744.140761:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980743 is committed 00000001:00080000:3.0:1789760744.191714:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980748 is committed 00000001:00080000:3.0:1789760744.203064:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980749 is committed 00000001:00080000:3.0:1789760744.213287:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980750 is committed 00000001:00080000:3.0:1789760744.223502:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980751 is committed 00000100:00100000:3.0:1789760744.238514:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760744.286494:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760744.327456:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760744.367230:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760744.424341:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760744.454579:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760744.473720:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760744.513267:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760744.534551:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980772 is committed 00000001:00080000:3.0:1789760744.552195:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980773 is committed 00000100:00100000:3.0:1789760744.562585:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760744.568828:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980774 is committed 00000100:00100000:3.0:1789760744.573140:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760744.580866:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980775 is committed 00000100:00100000:3.0:1789760744.630806:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760744.666203:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980780 is committed 00000100:00100000:3.0:1789760744.670640:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760744.686996:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980781 is committed 00000001:00080000:3.0:1789760744.697657:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980782 is committed 00000100:00100000:3.0:1789760744.716867:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760744.762247:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760744.785700:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980788 is committed 00000100:00100000:3.0:1789760744.805559:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760744.864105:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980793 is committed 00000100:00100000:3.0:1789760744.956511:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760745.015257:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760745.096255:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:3.0:1789760745.148295:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 55287808 -> 55291904 00002000:00010000:3.0:1789760745.148301:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 108008 -> 108016 00010000:00010000:3.0:1789760745.148315:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c1b4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55287808->55291903] (req 55287808->55291903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.148328:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c1b4c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55287808->55291903] (req 55287808->55291903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.148345:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c1b4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55287808->55291903] (req 55287808->55291903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.148353:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760745.148358:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c1b4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55287808->55291903] (req 55287808->55291903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.148367:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c1b4c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55287808->55291903] (req 55287808->55291903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.148527:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c1b4c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55287808->55291903] (req 55287808->55291903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.148537:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c1b4c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55287808->55291903] (req 55287808->55291903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.148668:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554489344:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13116us (13350us total) trans 4294980807 rc 0/0 00000100:00100000:3.0:1789760745.148679:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13509 00000100:00100000:3.0:1789760745.148688:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 688362720601 time 688381154728 state 0x2 ref 0 00000100:00100000:3.0:1789760745.153226:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760745.199180:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760745.215165:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760745.239476:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980813 is committed 00000100:00100000:3.0:1789760745.247922:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760745.261073:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760745.297527:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760745.306559:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980817 is committed 00000100:00100000:3.0:1789760745.315512:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760745.347082:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760745.390313:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980822 is committed 00000100:00100000:3.0:1789760745.401799:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760745.413321:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980823 is committed 00000100:00100000:3.0:1789760745.422634:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554491520 00000100:00100000:3.0:1789760745.422648:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554491520 00000100:00100000:3.0:1789760745.469681:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760745.487270:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980826 is committed 00000100:00100000:3.0:1789760745.494718:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554491904 00000100:00100000:3.0:1789760745.494732:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554491904 00000100:00100000:3.0:1789760745.494804:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 688696505848 time 688715711818 state 0x0 ref 1 00000100:00100000:3.0:1789760745.494823:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 688715972234 time 688715711818 state 0x1 ref 1 00000100:00100000:3.0:1789760745.494861:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 688715972234 time 688740479415 state 0x0 ref 1 00000100:00100000:3.0:1789760745.495001:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09314a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554491904:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760745.495068:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 55369728, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0x1a688 00010000:00010000:3.0:1789760745.495080:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c1bd8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.495098:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760745.495104:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.495112:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c1bd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55369728->55373823] (req 55369728->55373823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.495130:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c1bd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55369728->55373823] (req 55369728->55373823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760745.508906:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 55369728 -> 55373824 00002000:00010000:3.0:1789760745.508912:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 108168 -> 108176 00010000:00010000:3.0:1789760745.508930:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c1bd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55369728->55373823] (req 55369728->55373823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.508944:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c1bd8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55369728->55373823] (req 55369728->55373823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.508963:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c1bd8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55369728->55373823] (req 55369728->55373823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.508976:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760745.508982:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c1bd8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55369728->55373823] (req 55369728->55373823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.508999:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c1bd8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55369728->55373823] (req 55369728->55373823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.509025:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c1bd8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55369728->55373823] (req 55369728->55373823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.509035:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c1bd8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55369728->55373823] (req 55369728->55373823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.509161:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09314a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554491904:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14282us (15057us total) trans 4294980827 rc 0/0 00000100:00100000:3.0:1789760745.509173:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13529 00000100:00100000:3.0:1789760745.509184:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 688715972234 time 688740479415 state 0x2 ref 0 00000001:00080000:3.0:1789760745.558183:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980829 is committed 00000100:00100000:3.0:1789760745.566302:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760745.582862:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554492416 00000100:00100000:3.0:1789760745.582879:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554492416 00000100:00100000:3.0:1789760745.582964:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 688791717385 time 688812183058 state 0x0 ref 1 00000100:00100000:3.0:1789760745.582985:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 688812443474 time 688812183058 state 0x1 ref 1 00000100:00100000:3.0:1789760745.583014:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 688812443474 time 688828633344 state 0x0 ref 1 00000100:00100000:3.0:1789760745.583029:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09316d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554492416:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760745.583095:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 55386112, mtime: 0x6aad94e8, atime: 0x0, ctime: 0x6aad94e8, blocks: 0x1a6a8 00010000:00010000:3.0:1789760745.583108:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1bf4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.583127:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=688s v=(0 0) 00010000:00010000:3.0:1789760745.583132:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.583139:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1bf4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55386112->55390207] (req 55386112->55390207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.583156:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1bf4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55386112->55390207] (req 55386112->55390207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760745.593000:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 55386112 -> 55390208 00002000:00010000:3.0:1789760745.593006:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 108200 -> 108208 00010000:00010000:3.0:1789760745.593024:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1bf4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55386112->55390207] (req 55386112->55390207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.593039:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1bf4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55386112->55390207] (req 55386112->55390207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.593048:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1bf4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55386112->55390207] (req 55386112->55390207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.593062:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760745.593068:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1bf4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55386112->55390207] (req 55386112->55390207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.593079:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1bf4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55386112->55390207] (req 55386112->55390207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.593104:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1bf4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55386112->55390207] (req 55386112->55390207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.593112:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1bf4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55386112->55390207] (req 55386112->55390207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.593280:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09316d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554492416:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10246us (11178us total) trans 4294980831 rc 0/0 00000100:00100000:3.0:1789760745.593293:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13533 00000100:00100000:3.0:1789760745.593306:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 688812443474 time 688828633344 state 0x2 ref 0 00000100:00100000:3.0:1789760745.600258:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760745.633643:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980833 is committed 00000100:00100000:3.0:1789760745.657601:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760745.734250:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980837 is committed 00000001:00080000:3.0:1789760745.759274:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980838 is committed 00000100:00100000:3.0:1789760745.768323:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554493440 00000100:00100000:3.0:1789760745.768348:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554493440 00000100:00100000:3.0:1789760745.768452:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 688971242635 time 688995209274 state 0x0 ref 1 00000100:00100000:3.0:1789760745.768471:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 688995469690 time 688995209274 state 0x1 ref 1 00000100:00100000:3.0:1789760745.768501:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 688995469690 time 689014119467 state 0x0 ref 1 00000100:00100000:3.0:1789760745.768514:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554493440:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760745.768576:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 55418880, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0x1a6e8 00010000:00010000:3.0:1789760745.768588:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1c2c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.768608:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=689s v=(0 0) 00010000:00010000:3.0:1789760745.768613:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760745.768621:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1c2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55418880->55422975] (req 55418880->55422975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.768637:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1c2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55418880->55422975] (req 55418880->55422975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760745.778307:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 55418880 -> 55422976 00002000:00010000:3.0:1789760745.778313:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 108264 -> 108272 00010000:00010000:3.0:1789760745.778333:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1c2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55418880->55422975] (req 55418880->55422975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.778346:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1c2c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55418880->55422975] (req 55418880->55422975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.778355:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1c2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55418880->55422975] (req 55418880->55422975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.778364:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760745.778369:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1c2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55418880->55422975] (req 55418880->55422975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.778416:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1c2c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55418880->55422975] (req 55418880->55422975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.778449:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1c2c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55418880->55422975] (req 55418880->55422975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760745.778458:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1c2c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55418880->55422975] (req 55418880->55422975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760745.778549:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554493440:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10032us (10336us total) trans 4294980839 rc 0/0 00000100:00100000:3.0:1789760745.778560:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13541 00000100:00100000:3.0:1789760745.778569:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 688995469690 time 689014119467 state 0x2 ref 0 00000001:00080000:3.0:1789760745.807848:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980840 is committed 00000001:00080000:3.0:1789760745.837515:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980841 is committed 00000100:00100000:3.0:1789760745.884345:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554493952 00000100:00100000:3.0:1789760745.884357:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554493952 00000100:00100000:3.0:1789760745.884484:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 689072254960 time 689109090229 state 0x0 ref 1 00000100:00100000:3.0:1789760745.884504:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 689109350645 time 689109090229 state 0x1 ref 1 00000100:00100000:3.0:1789760745.884542:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 689109350645 time 689130160775 state 0x0 ref 1 00000100:00100000:3.0:1789760745.884743:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554493952:12345-192.168.203.22@tcp:4:dd.0 00000001:00080000:3.0:1789760745.940831:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980845 is committed 00000001:00080000:3.0:1789760745.973028:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980846 is committed 00000001:00080000:3.0:1789760746.143979:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980852 is committed 00000100:00100000:3.0:1789760746.167984:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760746.231594:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980856 is committed 00000100:00100000:3.0:1789760746.239192:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760746.252669:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980857 is committed 00002000:00010000:3.0:1789760746.265848:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 55492608 -> 55496704 00002000:00010000:3.0:1789760746.265862:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 108408 -> 108416 00010000:00010000:3.0:1789760746.266478:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c1caa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55492608->55496703] (req 55492608->55496703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.266495:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c1caa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55492608->55496703] (req 55492608->55496703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.266504:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c1caa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55492608->55496703] (req 55492608->55496703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.266511:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760746.266517:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c1caa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55492608->55496703] (req 55492608->55496703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.266526:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c1caa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55492608->55496703] (req 55492608->55496703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.266553:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c1caa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55492608->55496703] (req 55492608->55496703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.266561:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c1caa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55492608->55496703] (req 55492608->55496703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.266804:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03cc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554495744:12345-192.168.203.22@tcp:4:dd.0 Request processed in 27034us (27621us total) trans 4294980857 rc 0/0 00000100:00100000:3.0:1789760746.266818:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13559 00000100:00100000:3.0:1789760746.266830:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 689458966469 time 689485367698 state 0x2 ref 0 00000001:00080000:3.0:1789760746.293545:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980858 is committed 00000001:00080000:3.0:1789760746.313920:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980859 is committed 00000100:00100000:3.0:1789760746.317797:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760746.329441:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980860 is committed 00000100:00100000:3.0:1789760746.346792:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554496256 00000100:00100000:3.0:1789760746.346805:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554496256 00000100:00100000:3.0:1789760746.346877:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 689548528118 time 689563751811 state 0x0 ref 1 00000100:00100000:3.0:1789760746.346896:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 689564012227 time 689563751811 state 0x1 ref 1 00000100:00100000:3.0:1789760746.346935:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 689564012227 time 689592553533 state 0x0 ref 1 00000100:00100000:3.0:1789760746.346959:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb064ed500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554496256:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760746.347020:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 55508992, mtime: 0x6aad94e9, atime: 0x0, ctime: 0x6aad94e9, blocks: 0x1a798 00010000:00010000:3.0:1789760746.347033:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c1cc6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.347226:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=689s v=(0 0) 00010000:00010000:3.0:1789760746.347232:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.347240:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c1cc6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55508992->55513087] (req 55508992->55513087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.347258:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c1cc6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55508992->55513087] (req 55508992->55513087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.386266:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb04563b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573900928:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760746.386296:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1789760746.386394:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789760746.386880:0:8439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 334 00000100:00100000:3.0:1789760746.386895:0:8439:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1876699573900800:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1789760746.386933:0:8439:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:3.0:1789760746.386999:0:8439:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1876699573900800:12345-0@lo:13:osp-pre-1-1.0 Request processed in 108us (858us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760746.387006:0:8439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 334 00000100:00100000:3.0:1789760746.387026:0:8439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 335 00000100:00100000:3.0:1789760746.387033:0:8439:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04562d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573900928:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760746.387043:0:8439:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:3.0:1789760746.387067:0:8439:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04562d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573900928:12345-0@lo:13:osp-pre-1-0.0 Request processed in 36us (771us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760746.387073:0:8439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 335 00000100:00100000:3.0:1789760746.390418:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573902080 00000100:00100000:3.0:1789760746.390424:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573902080 00000100:00100000:3.0:1789760746.390470:0:8435:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 275 00000100:00100000:3.0:1789760746.390477:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3647:x1876699573902080:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760746.390541:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3647:x1876699573902080:12345-0@lo:400:kworker.0 Request processed in 64us (2664us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760746.390549:0:8435:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 275 00000100:00080000:3.0:1789760746.390571:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760746.395587:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554496384:12345-192.168.203.22@tcp:4:dd.0 Request processed in 16334us (16663us total) trans 4294980862 rc 0/0 00000100:00100000:3.0:1789760746.395601:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13564 00000100:00100000:3.0:1789760746.395612:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 689592813949 time 689624850686 state 0x2 ref 0 00000100:00100000:3.0:1789760746.403776:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760746.463549:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760746.510558:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980868 is committed 00000100:00100000:3.0:1789760746.515096:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760746.525735:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980869 is committed 00000100:00100000:3.0:1789760746.566432:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760746.576459:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980872 is committed 00000100:00100000:3.0:1789760746.613328:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554497920 00000100:00100000:3.0:1789760746.613351:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554497920 00000100:00100000:3.0:1789760746.625333:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760746.736543:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760746.748819:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760746.773408:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554499456 00000100:00100000:3.0:1789760746.773422:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554499456 00000100:00100000:3.0:1789760746.773495:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 689995008147 time 690005989365 state 0x0 ref 1 00000100:00100000:3.0:1789760746.773518:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690006249781 time 690005989365 state 0x1 ref 1 00000100:00100000:3.0:1789760746.773550:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690006249781 time 690019168685 state 0x0 ref 1 00000100:00100000:3.0:1789760746.773563:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a270e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554499456:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760746.773631:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 55611392, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0x1a860 00010000:00010000:3.0:1789760746.773642:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c1d75 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.773663:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760746.773669:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760746.773677:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c1d75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55611392->55615487] (req 55611392->55615487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.773695:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c1d75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55611392->55615487] (req 55611392->55615487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760746.778923:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 55611392 -> 55615488 00002000:00010000:3.0:1789760746.778929:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 108640 -> 108648 00010000:00010000:3.0:1789760746.778956:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c1d75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55611392->55615487] (req 55611392->55615487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.778971:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c1d75 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55611392->55615487] (req 55611392->55615487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.778983:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c1d75 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55611392->55615487] (req 55611392->55615487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.778992:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760746.778998:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c1d75 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55611392->55615487] (req 55611392->55615487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.779009:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c1d75 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55611392->55615487] (req 55611392->55615487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.779035:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c1d75 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55611392->55615487] (req 55611392->55615487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760746.779044:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c1d75 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55611392->55615487] (req 55611392->55615487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760746.779157:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a270e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554499456:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5589us (5873us total) trans 4294980886 rc 0/0 00000100:00100000:3.0:1789760746.779170:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13588 00000100:00100000:3.0:1789760746.779183:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690006249781 time 690019168685 state 0x2 ref 0 00000100:00100000:3.0:1789760746.821278:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554499968 00000100:00100000:3.0:1789760746.821292:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554499968 00000100:00100000:3.0:1789760746.821374:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690040696376 time 690054528636 state 0x0 ref 1 00000100:00100000:3.0:1789760746.821695:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690054789052 time 690054528636 state 0x1 ref 1 00000001:00080000:3.0:1789760746.854082:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980892 is committed 00000001:00080000:3.0:1789760746.867375:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980893 is committed 00000001:00080000:3.0:1789760746.883750:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980894 is committed 00000001:00080000:3.0:1789760746.897374:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980895 is committed 00000001:00080000:3.0:1789760746.933629:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980897 is committed 00000001:00080000:3.0:1789760746.964280:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980899 is committed 00000100:00100000:3.0:1789760746.995847:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760747.008728:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980901 is committed 00000100:00100000:3.0:1789760747.015234:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00080000:3.0:1789760747.015562:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760747.070843:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00010000:00010000:3.0:1789760747.090188:0:8453:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760747.123922:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760747.166255:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554502528 00000100:00100000:3.0:1789760747.166269:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554502528 00000100:00100000:3.0:1789760747.166493:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690370691298 time 690388526719 state 0x0 ref 1 00000100:00100000:3.0:1789760747.166514:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690388787135 time 690388526719 state 0x1 ref 1 00000100:00100000:3.0:1789760747.166606:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690388787135 time 690412223315 state 0x0 ref 1 00000100:00100000:3.0:1789760747.166623:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb064ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554502528:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760747.166707:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 55709696, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0x1a920 00010000:00010000:3.0:1789760747.166719:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e1d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.166740:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760747.166747:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.166755:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55709696->55713791] (req 55709696->55713791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.166777:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55709696->55713791] (req 55709696->55713791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760747.177110:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 55709696 -> 55713792 00002000:00010000:3.0:1789760747.177117:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 108832 -> 108840 00010000:00010000:3.0:1789760747.177134:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55709696->55713791] (req 55709696->55713791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.177150:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e1d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55709696->55713791] (req 55709696->55713791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.177159:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55709696->55713791] (req 55709696->55713791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.177167:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760747.177172:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55709696->55713791] (req 55709696->55713791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.177182:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e1d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55709696->55713791] (req 55709696->55713791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.177204:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e1d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55709696->55713791] (req 55709696->55713791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.177211:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e1d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55709696->55713791] (req 55709696->55713791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.177749:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb064ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554502528:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11121us (11682us total) trans 4294980910 rc 0/0 00000100:00100000:3.0:1789760747.177763:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13612 00000100:00100000:3.0:1789760747.177775:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690388787135 time 690412223315 state 0x2 ref 0 00000100:00100000:3.0:1789760747.181305:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760747.234489:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980913 is committed 00000100:00100000:3.0:1789760747.245430:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760747.278877:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980916 is committed 00000100:00100000:3.0:1789760747.298096:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554503552 00000100:00100000:3.0:1789760747.298107:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554503552 00000100:00100000:3.0:1789760747.298161:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690518668832 time 690532164181 state 0x0 ref 1 00000100:00100000:3.0:1789760747.298177:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690532424597 time 690532164181 state 0x1 ref 1 00000100:00100000:3.0:1789760747.298205:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690532424597 time 690543823627 state 0x0 ref 1 00000100:00100000:3.0:1789760747.298217:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09317100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554503552:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760747.298263:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 55742464, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0x1a960 00010000:00010000:3.0:1789760747.298271:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e55 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.298286:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760747.298290:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.298296:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55742464->55746559] (req 55742464->55746559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.298310:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55742464->55746559] (req 55742464->55746559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760747.306789:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 55742464 -> 55746560 00002000:00010000:3.0:1789760747.306794:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 108896 -> 108904 00010000:00010000:3.0:1789760747.306807:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55742464->55746559] (req 55742464->55746559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.306818:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e55 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55742464->55746559] (req 55742464->55746559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.307228:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55742464->55746559] (req 55742464->55746559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.307235:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760747.307239:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55742464->55746559] (req 55742464->55746559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.307248:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e55 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55742464->55746559] (req 55742464->55746559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.307606:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e55 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55742464->55746559] (req 55742464->55746559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.307617:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e55 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55742464->55746559] (req 55742464->55746559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.308424:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09317100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554503552:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10202us (10728us total) trans 4294980918 rc 0/0 00000100:00100000:3.0:1789760747.308436:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13620 00000100:00100000:3.0:1789760747.308445:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690532424597 time 690543823627 state 0x2 ref 0 00000001:00080000:3.0:1789760747.337339:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980920 is committed 00000001:00080000:3.0:1789760747.356096:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980921 is committed 00000100:00100000:3.0:1789760747.362807:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554504064 00000100:00100000:3.0:1789760747.362924:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554504064 00000100:00100000:3.0:1789760747.363010:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690577858399 time 690594458407 state 0x0 ref 1 00000100:00100000:3.0:1789760747.363029:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690594718823 time 690594458407 state 0x1 ref 1 00000100:00100000:3.0:1789760747.363065:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690594718823 time 690608683643 state 0x0 ref 1 00000100:00100000:3.0:1789760747.363079:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554504064:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760747.363148:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 55758848, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0x1a980 00010000:00010000:3.0:1789760747.363160:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e71 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.363193:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760747.363198:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.363207:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55758848->55762943] (req 55758848->55762943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.363229:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55758848->55762943] (req 55758848->55762943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760747.372977:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 55758848 -> 55762944 00002000:00010000:3.0:1789760747.372984:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 108928 -> 108936 00010000:00010000:3.0:1789760747.373002:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55758848->55762943] (req 55758848->55762943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.373016:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e71 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55758848->55762943] (req 55758848->55762943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.373025:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55758848->55762943] (req 55758848->55762943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.373033:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760747.373039:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55758848->55762943] (req 55758848->55762943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.373049:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e71 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55758848->55762943] (req 55758848->55762943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.373076:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e71 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55758848->55762943] (req 55758848->55762943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.373083:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e71 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55758848->55762943] (req 55758848->55762943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.373677:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554504064:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10591us (11015us total) trans 4294980922 rc 0/0 00000100:00100000:3.0:1789760747.373693:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13624 00000100:00100000:3.0:1789760747.373714:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690594718823 time 690608683643 state 0x2 ref 0 00000100:00100000:3.0:1789760747.443776:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554504576 00000100:00100000:3.0:1789760747.443787:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554504576 00000100:00100000:3.0:1789760747.443850:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690652002147 time 690672493595 state 0x0 ref 1 00000100:00100000:3.0:1789760747.443865:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690672754011 time 690672493595 state 0x1 ref 1 00000100:00100000:3.0:1789760747.443904:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690672754011 time 690689523130 state 0x0 ref 1 00000100:00100000:3.0:1789760747.443918:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554504576:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760747.443972:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 55775232, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0x1a9a0 00010000:00010000:3.0:1789760747.443982:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e8d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.443997:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760747.444006:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.444012:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55775232->55779327] (req 55775232->55779327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.444025:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55775232->55779327] (req 55775232->55779327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760747.451022:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 55775232 -> 55779328 00002000:00010000:3.0:1789760747.451027:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 108960 -> 108968 00010000:00010000:3.0:1789760747.451040:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55775232->55779327] (req 55775232->55779327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.451053:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e8d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55775232->55779327] (req 55775232->55779327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.451062:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55775232->55779327] (req 55775232->55779327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.451068:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760747.451073:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55775232->55779327] (req 55775232->55779327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.451082:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e8d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55775232->55779327] (req 55775232->55779327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.451101:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e8d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55775232->55779327] (req 55775232->55779327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.451108:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1e8d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55775232->55779327] (req 55775232->55779327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.451235:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554504576:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7313us (7550us total) trans 4294980926 rc 0/0 00000100:00100000:3.0:1789760747.451247:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13628 00000100:00100000:3.0:1789760747.451255:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690672754011 time 690689523130 state 0x2 ref 0 00000001:00080000:3.0:1789760747.465453:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980927 is committed 00000001:00080000:3.0:1789760747.478891:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980928 is committed 00002000:00010000:3.0:1789760747.493593:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 55787520 -> 55791616 00002000:00010000:3.0:1789760747.493598:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 108984 -> 108992 00010000:00010000:3.0:1789760747.493612:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c1ea2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55787520->55791615] (req 55787520->55791615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.493625:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c1ea2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55787520->55791615] (req 55787520->55791615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.493634:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c1ea2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55787520->55791615] (req 55787520->55791615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.493642:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760747.493647:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c1ea2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55787520->55791615] (req 55787520->55791615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.493656:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c1ea2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55787520->55791615] (req 55787520->55791615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.493674:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c1ea2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55787520->55791615] (req 55787520->55791615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.493687:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c1ea2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55787520->55791615] (req 55787520->55791615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.493846:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554504960:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9053us (9281us total) trans 4294980929 rc 0/0 00000100:00100000:3.0:1789760747.493855:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13631 00000100:00100000:3.0:1789760747.493864:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690719054468 time 690730396686 state 0x2 ref 0 00000100:00100000:3.0:1789760747.497664:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554505088 00000100:00100000:3.0:1789760747.497673:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554505088 00000100:00100000:3.0:1789760747.497787:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690719054468 time 690730396686 state 0x0 ref 1 00000100:00100000:3.0:1789760747.497803:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690730657102 time 690730396686 state 0x1 ref 1 00000100:00100000:3.0:1789760747.497877:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690730657102 time 690743494738 state 0x0 ref 1 00000100:00100000:3.0:1789760747.497891:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc0f17e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554505088:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760747.497937:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 55791616, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0x1a9c0 00010000:00010000:3.0:1789760747.497958:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1ea9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.497974:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760747.497979:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.497985:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1ea9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55791616->55795711] (req 55791616->55795711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.498003:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1ea9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55791616->55795711] (req 55791616->55795711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760747.504004:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980930 is committed 00000001:00080000:3.0:1789760747.513356:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980931 is committed 00000100:00100000:3.0:1789760747.537749:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554505472 00000100:00100000:3.0:1789760747.537759:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554505472 00000100:00100000:3.0:1789760747.537815:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690754079357 time 690768539483 state 0x0 ref 1 00000100:00100000:3.0:1789760747.537830:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690768799899 time 690768539483 state 0x1 ref 1 00000100:00100000:3.0:1789760747.537868:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690768799899 time 690783486164 state 0x0 ref 1 00000100:00100000:3.0:1789760747.537882:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554505472:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760747.537928:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 55803904, mtime: 0x6aad94ea, atime: 0x0, ctime: 0x6aad94ea, blocks: 0x1a9d8 00010000:00010000:3.0:1789760747.537951:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c1ebe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.537967:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760747.537972:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.537979:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c1ebe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55803904->55807999] (req 55803904->55807999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.537992:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c1ebe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55803904->55807999] (req 55803904->55807999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760747.564704:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 55808000 -> 55812096 00002000:00010000:3.0:1789760747.564710:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 109024 -> 109032 00010000:00010000:3.0:1789760747.564723:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c1ec5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55808000->55812095] (req 55808000->55812095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.564736:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c1ec5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55808000->55812095] (req 55808000->55812095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.564746:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c1ec5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55808000->55812095] (req 55808000->55812095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.564753:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760747.564758:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c1ec5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55808000->55812095] (req 55808000->55812095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.564767:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c1ec5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55808000->55812095] (req 55808000->55812095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.564788:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c1ec5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55808000->55812095] (req 55808000->55812095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.564795:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c1ec5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55808000->55812095] (req 55808000->55812095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.564891:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554505600:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11793us (12512us total) trans 4294980934 rc 0/0 00000100:00100000:3.0:1789760747.564902:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13636 00000100:00100000:3.0:1789760747.564911:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690783746580 time 690798702251 state 0x2 ref 0 00000100:00100000:3.0:1789760747.573910:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760747.629542:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760747.680294:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980939 is committed 00000100:00100000:3.0:1789760747.684373:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760747.698594:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760747.718493:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554506624 00000100:00100000:3.0:1789760747.718503:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554506624 00000100:00100000:3.0:1789760747.718554:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690932239986 time 690950236101 state 0x0 ref 1 00000100:00100000:3.0:1789760747.718568:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690950496517 time 690950236101 state 0x1 ref 1 00000100:00100000:3.0:1789760747.718599:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 690950496517 time 690964217286 state 0x0 ref 1 00000100:00100000:3.0:1789760747.718611:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09315500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554506624:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760747.718657:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 55840768, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0x1aa20 00010000:00010000:3.0:1789760747.718667:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c1efd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.718683:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789760747.718688:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.718694:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c1efd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55840768->55844863] (req 55840768->55844863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.718714:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c1efd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55840768->55844863] (req 55840768->55844863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760747.725445:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980942 is committed 00000100:00100000:3.0:1789760747.729045:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760747.738330:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980943 is committed 00000100:00100000:3.0:1789760747.749530:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760747.755206:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980944 is committed 00000100:00100000:3.0:1789760747.778397:0:6515:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554507136 00000100:00100000:3.0:1789760747.778402:0:6515:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554507136 00000100:00100000:3.0:1789760747.778435:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 458 00000100:00100000:3.0:1789760747.778444:0:6515:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc0c825450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9765eb0b-55b9-40b6-bfdf-ffee156c3226+13:2411:x1876699554507136:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:3.0:1789760747.778513:0:6515:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc0c825450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9765eb0b-55b9-40b6-bfdf-ffee156c3226+13:2411:x1876699554507136:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 69us (507us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760747.778521:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 458 00000100:00100000:3.0:1789760747.779482:0:6520:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 815 00000100:00100000:3.0:1789760747.779494:0:6520:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09314000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2411:x1876699554507520:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:3.0:1789760747.779547:0:6520:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09314000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2411:x1876699554507520:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 56us (1579us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760747.779554:0:6520:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 815 00000001:00080000:3.0:1789760747.783129:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980945 is committed 00000001:00080000:3.0:1789760747.796425:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980946 is committed 00000001:00080000:3.0:1789760747.809861:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980947 is committed 00000001:00080000:3.0:1789760747.825084:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980948 is committed 00000100:00100000:3.0:1789760747.882859:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760747.890473:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980951 is committed 00000001:00080000:3.0:1789760747.906657:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980952 is committed 00002000:00010000:3.0:1789760747.923091:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 55885824 -> 55889920 00002000:00010000:3.0:1789760747.923097:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 109176 -> 109184 00010000:00010000:3.0:1789760747.923110:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c1f4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55885824->55889919] (req 55885824->55889919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.923124:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c1f4a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55885824->55889919] (req 55885824->55889919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.923134:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c1f4a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55885824->55889919] (req 55885824->55889919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.923141:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760747.923145:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c1f4a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55885824->55889919] (req 55885824->55889919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.923154:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c1f4a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55885824->55889919] (req 55885824->55889919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.923173:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c1f4a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55885824->55889919] (req 55885824->55889919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.923180:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c1f4a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55885824->55889919] (req 55885824->55889919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760747.923273:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09317480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554508672:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11033us (11276us total) trans 4294980953 rc 0/0 00000100:00100000:3.0:1789760747.923284:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13655 00000100:00100000:3.0:1789760747.923294:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 691141059913 time 691157843484 state 0x2 ref 0 00000100:00100000:3.0:1789760747.930393:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554508800 00000100:00100000:3.0:1789760747.930403:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554508800 00000100:00100000:3.0:1789760747.999198:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554509184 00000100:00100000:3.0:1789760747.999208:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554509184 00000100:00100000:3.0:1789760747.999263:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 691197068633 time 691222935287 state 0x0 ref 1 00000100:00100000:3.0:1789760747.999278:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 691223195703 time 691222935287 state 0x1 ref 1 00000100:00100000:3.0:1789760747.999596:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 691223195703 time 691245212333 state 0x0 ref 1 00000100:00100000:3.0:1789760747.999611:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554509184:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760747.999659:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 55902208, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0x1aa98 00010000:00010000:3.0:1789760747.999667:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c1f66 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.999683:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760747.999687:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760747.999692:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c1f66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55902208->55906303] (req 55902208->55906303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760747.999705:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c1f66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55902208->55906303] (req 55902208->55906303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.046121:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760748.221059:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554510720 00000100:00100000:3.0:1789760748.221072:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554510720 00000100:00100000:3.0:1789760748.221147:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 691421401661 time 691440101748 state 0x0 ref 1 00000100:00100000:3.0:1789760748.221228:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 691440362164 time 691440101748 state 0x1 ref 1 00000100:00100000:3.0:1789760748.221320:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 691440362164 time 691466937071 state 0x0 ref 1 00000100:00100000:3.0:1789760748.221337:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a273480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554510720:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760748.221519:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 55951360, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0x1aaf8 00010000:00010000:3.0:1789760748.221532:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1fba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.221553:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760748.221559:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.221567:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1fba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55951360->55955455] (req 55951360->55955455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.221583:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1fba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55951360->55955455] (req 55951360->55955455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760748.232066:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 55951360 -> 55955456 00002000:00010000:3.0:1789760748.232073:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 109304 -> 109312 00010000:00010000:3.0:1789760748.232091:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1fba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55951360->55955455] (req 55951360->55955455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.232108:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1fba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55951360->55955455] (req 55951360->55955455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.232118:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1fba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55951360->55955455] (req 55951360->55955455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.232127:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760748.232132:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1fba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55951360->55955455] (req 55951360->55955455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.232142:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1fba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55951360->55955455] (req 55951360->55955455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.232170:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1fba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55951360->55955455] (req 55951360->55955455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.232179:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1fba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55951360->55955455] (req 55951360->55955455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.232298:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a273480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554510720:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10958us (11395us total) trans 4294980969 rc 0/0 00000100:00100000:3.0:1789760748.232310:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13671 00000100:00100000:3.0:1789760748.232320:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 691440362164 time 691466937071 state 0x2 ref 0 00000100:00100000:3.0:1789760748.299051:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554511232 00000100:00100000:3.0:1789760748.299064:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554511232 00000100:00100000:3.0:1789760748.299136:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 691517515178 time 691532701993 state 0x0 ref 1 00000100:00100000:3.0:1789760748.299155:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 691532962409 time 691532701993 state 0x1 ref 1 00000100:00100000:3.0:1789760748.299194:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 691532962409 time 691544811958 state 0x0 ref 1 00000100:00100000:3.0:1789760748.299207:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a270700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554511232:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760748.299267:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 55967744, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0x1ab18 00010000:00010000:3.0:1789760748.299280:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1fd6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.299301:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760748.299306:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.299489:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1fd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55967744->55971839] (req 55967744->55971839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.299507:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1fd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55967744->55971839] (req 55967744->55971839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760748.308607:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 55967744 -> 55971840 00002000:00010000:3.0:1789760748.308614:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 109336 -> 109344 00010000:00010000:3.0:1789760748.308635:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1fd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55967744->55971839] (req 55967744->55971839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.308653:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1fd6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55967744->55971839] (req 55967744->55971839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.308672:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1fd6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55967744->55971839] (req 55967744->55971839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.308680:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760748.308685:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1fd6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55967744->55971839] (req 55967744->55971839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.308820:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1fd6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55967744->55971839] (req 55967744->55971839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.308850:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1fd6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55967744->55971839] (req 55967744->55971839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.308858:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1fd6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55967744->55971839] (req 55967744->55971839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.309128:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a270700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554511232:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9915us (10195us total) trans 4294980973 rc 0/0 00000100:00100000:3.0:1789760748.309142:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13675 00000100:00100000:3.0:1789760748.309153:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 691532962409 time 691544811958 state 0x2 ref 0 00000001:00080000:3.0:1789760748.342440:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980975 is committed 00000100:00100000:3.0:1789760748.363302:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554511744 00000100:00100000:3.0:1789760748.363319:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554511744 00000100:00100000:3.0:1789760748.363414:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 691581131358 time 691593805707 state 0x0 ref 1 00000100:00100000:3.0:1789760748.363437:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 691594066123 time 691593805707 state 0x1 ref 1 00000100:00100000:3.0:1789760748.363467:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 691594066123 time 691609085765 state 0x0 ref 1 00000100:00100000:3.0:1789760748.363481:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a271f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554511744:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760748.363541:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 55984128, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0x1ab38 00010000:00010000:3.0:1789760748.363553:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1ff2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.363573:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760748.363578:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.363585:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1ff2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55984128->55988223] (req 55984128->55988223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.363601:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1ff2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55984128->55988223] (req 55984128->55988223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760748.372076:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 55984128 -> 55988224 00002000:00010000:3.0:1789760748.372082:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 109368 -> 109376 00010000:00010000:3.0:1789760748.372099:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1ff2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55984128->55988223] (req 55984128->55988223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.372116:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1ff2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55984128->55988223] (req 55984128->55988223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.372138:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1ff2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55984128->55988223] (req 55984128->55988223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.372148:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760748.372154:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1ff2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55984128->55988223] (req 55984128->55988223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.372165:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1ff2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55984128->55988223] (req 55984128->55988223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.372193:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1ff2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55984128->55988223] (req 55984128->55988223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.372202:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c1ff2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [55984128->55988223] (req 55984128->55988223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.372366:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a271f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554511744:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8880us (9163us total) trans 4294980977 rc 0/0 00000100:00100000:3.0:1789760748.372435:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13679 00000100:00100000:3.0:1789760748.372448:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 691594066123 time 691609085765 state 0x2 ref 0 00000001:00080000:3.0:1789760748.382323:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980978 is committed 00000100:00100000:3.0:1789760748.419348:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554512256 00000100:00100000:3.0:1789760748.419358:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554512256 00000100:00100000:3.0:1789760748.419441:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 691637416981 time 691650789591 state 0x0 ref 1 00000100:00100000:3.0:1789760748.419456:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 691651050007 time 691650789591 state 0x1 ref 1 00000100:00100000:3.0:1789760748.419556:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 691651050007 time 691665174326 state 0x0 ref 1 00000100:00100000:3.0:1789760748.419570:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554512256:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760748.419614:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56000512, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0x1ab58 00010000:00010000:3.0:1789760748.419623:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c200e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.419639:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760748.419643:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.419649:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c200e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56000512->56004607] (req 56000512->56004607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.419663:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c200e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56000512->56004607] (req 56000512->56004607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760748.426538:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980981 is committed 00000100:00100000:3.0:1789760748.433876:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554512384 00000100:00100000:3.0:1789760748.433888:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554512384 00000100:00100000:3.0:1789760748.433956:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 691651050007 time 691665174326 state 0x0 ref 1 00000100:00100000:3.0:1789760748.433971:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 691665434742 time 691665174326 state 0x1 ref 1 00000100:00100000:3.0:1789760748.434043:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 691665434742 time 691679660692 state 0x0 ref 1 00000100:00100000:3.0:1789760748.434055:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb064eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554512384:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760748.434101:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56004608, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0x1ab60 00010000:00010000:3.0:1789760748.434109:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c2015 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.434127:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760748.434132:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.434139:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c2015 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56004608->56008703] (req 56004608->56008703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.434155:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c2015 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56004608->56008703] (req 56004608->56008703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760748.480629:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980985 is committed 00000001:00080000:3.0:1789760748.498905:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980986 is committed 00000001:00080000:3.0:1789760748.512585:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980987 is committed 00000100:00100000:3.0:1789760748.518558:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760748.524498:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294980988 is committed 00000100:00100000:3.0:1789760748.531344:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554513280 00000100:00100000:3.0:1789760748.531355:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554513280 00000100:00100000:3.0:1789760748.531437:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 691750981868 time 691764605120 state 0x0 ref 1 00000100:00100000:3.0:1789760748.531454:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 691764865536 time 691764605120 state 0x1 ref 1 00000100:00100000:3.0:1789760748.531477:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 691764865536 time 691777096007 state 0x0 ref 1 00000100:00100000:3.0:1789760748.531491:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb064ec000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554513280:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760748.531537:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56033280, mtime: 0x6aad94eb, atime: 0x0, ctime: 0x6aad94eb, blocks: 0x1ab98 00010000:00010000:3.0:1789760748.531547:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c2046 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.531569:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=691s v=(0 0) 00010000:00010000:3.0:1789760748.531573:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.531579:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c2046 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56033280->56037375] (req 56033280->56037375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.531593:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c2046 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56033280->56037375] (req 56033280->56037375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.589233:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760748.625780:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760748.675962:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760748.726479:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760748.777184:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760748.830555:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760748.874059:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760748.875000:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554516224:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760748.875048:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56127488, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0x1ac50 00010000:00010000:3.0:1789760748.875057:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c20e7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.875073:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760748.875078:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.875083:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c20e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56127488->56131583] (req 56127488->56131583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.875103:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c20e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56127488->56131583] (req 56127488->56131583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760748.883767:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56127488 -> 56131584 00002000:00010000:3.0:1789760748.883773:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 109648 -> 109656 00010000:00010000:3.0:1789760748.883786:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c20e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56127488->56131583] (req 56127488->56131583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.883798:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c20e7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56127488->56131583] (req 56127488->56131583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.884077:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c20e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56127488->56131583] (req 56127488->56131583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.884086:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760748.884090:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c20e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56127488->56131583] (req 56127488->56131583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.884207:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c20e7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56127488->56131583] (req 56127488->56131583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.884230:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c20e7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56127488->56131583] (req 56127488->56131583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.884239:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c20e7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56127488->56131583] (req 56127488->56131583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.884439:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554516224:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9837us (10384us total) trans 4294981012 rc 0/0 00000100:00100000:3.0:1789760748.884451:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13714 00000100:00100000:3.0:1789760748.884460:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692106848794 time 692120204156 state 0x2 ref 0 00000100:00100000:3.0:1789760748.922718:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760748.939317:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554516736 00000100:00100000:3.0:1789760748.939330:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554516736 00000100:00100000:3.0:1789760748.939405:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692157219522 time 692168502886 state 0x0 ref 1 00000100:00100000:3.0:1789760748.939420:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692168763302 time 692168502886 state 0x1 ref 1 00000100:00100000:3.0:1789760748.939456:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692168763302 time 692185073591 state 0x0 ref 1 00000100:00100000:3.0:1789760748.939469:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb064edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554516736:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760748.939514:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56143872, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0x1ac70 00010000:00010000:3.0:1789760748.939523:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2103 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.939538:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760748.939542:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760748.939548:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2103 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56143872->56147967] (req 56143872->56147967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.939562:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2103 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56143872->56147967] (req 56143872->56147967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760748.946163:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789760748.949751:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56143872 -> 56147968 00002000:00010000:3.0:1789760748.949759:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 109680 -> 109688 00010000:00010000:3.0:1789760748.949772:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2103 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56143872->56147967] (req 56143872->56147967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.949784:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2103 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56143872->56147967] (req 56143872->56147967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.949793:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2103 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56143872->56147967] (req 56143872->56147967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.949800:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760748.949805:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2103 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56143872->56147967] (req 56143872->56147967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.949814:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2103 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56143872->56147967] (req 56143872->56147967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.949833:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2103 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56143872->56147967] (req 56143872->56147967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760748.949840:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2103 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56143872->56147967] (req 56143872->56147967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760748.949955:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb064edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554516736:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10484us (12169us total) trans 4294981016 rc 0/0 00000100:00100000:3.0:1789760748.949967:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13718 00000100:00100000:3.0:1789760748.949977:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692168763302 time 692185073591 state 0x2 ref 0 00000100:00100000:3.0:1789760749.007581:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554517248 00000100:00100000:3.0:1789760749.007592:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554517248 00000100:00100000:3.0:1789760749.007663:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692215964141 time 692235629983 state 0x0 ref 1 00000100:00100000:3.0:1789760749.007682:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692235890399 time 692235629983 state 0x1 ref 1 00000100:00100000:3.0:1789760749.007721:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692235890399 time 692253340081 state 0x0 ref 1 00000100:00100000:3.0:1789760749.007734:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554517248:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760749.007778:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56160256, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0x1ac90 00010000:00010000:3.0:1789760749.007786:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c211f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.007802:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760749.007806:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.007811:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c211f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56160256->56164351] (req 56160256->56164351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.007825:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c211f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56160256->56164351] (req 56160256->56164351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760749.014231:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56160256 -> 56164352 00002000:00010000:3.0:1789760749.014236:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 109712 -> 109720 00010000:00010000:3.0:1789760749.014250:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c211f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56160256->56164351] (req 56160256->56164351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.014263:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c211f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56160256->56164351] (req 56160256->56164351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.014273:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c211f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56160256->56164351] (req 56160256->56164351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.014280:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760749.014285:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c211f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56160256->56164351] (req 56160256->56164351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.014294:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c211f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56160256->56164351] (req 56160256->56164351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.014360:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c211f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56160256->56164351] (req 56160256->56164351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.014369:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c211f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56160256->56164351] (req 56160256->56164351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.014509:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554517248:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6773us (7045us total) trans 4294981020 rc 0/0 00000100:00100000:3.0:1789760749.014520:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13722 00000100:00100000:3.0:1789760749.014530:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692235890399 time 692253340081 state 0x2 ref 0 00000100:00100000:3.0:1789760749.065256:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554517760 00000100:00100000:3.0:1789760749.065265:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554517760 00000100:00100000:3.0:1789760749.065314:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692285325497 time 692300217291 state 0x0 ref 1 00000100:00100000:3.0:1789760749.065329:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692300477707 time 692300217291 state 0x1 ref 1 00000100:00100000:3.0:1789760749.065358:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692300477707 time 692310976720 state 0x0 ref 1 00000100:00100000:3.0:1789760749.065369:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb064ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554517760:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760749.065443:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56176640, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0x1acb0 00010000:00010000:3.0:1789760749.065452:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c213b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.065467:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760749.065471:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.065477:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c213b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56176640->56180735] (req 56176640->56180735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.065491:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c213b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56176640->56180735] (req 56176640->56180735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760749.070608:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56176640 -> 56180736 00002000:00010000:3.0:1789760749.070613:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 109744 -> 109752 00010000:00010000:3.0:1789760749.070637:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c213b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56176640->56180735] (req 56176640->56180735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.070650:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c213b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56176640->56180735] (req 56176640->56180735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.070658:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c213b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56176640->56180735] (req 56176640->56180735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.070664:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760749.070668:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c213b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56176640->56180735] (req 56176640->56180735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.070677:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c213b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56176640->56180735] (req 56176640->56180735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.070697:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c213b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56176640->56180735] (req 56176640->56180735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.070705:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c213b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56176640->56180735] (req 56176640->56180735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.070844:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb064ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554517760:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5471us (5667us total) trans 4294981024 rc 0/0 00000100:00100000:3.0:1789760749.070855:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13726 00000100:00100000:3.0:1789760749.070864:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692300477707 time 692310976720 state 0x2 ref 0 00000100:00100000:3.0:1789760749.089852:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554518016 00000100:00100000:3.0:1789760749.089863:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554518016 00000100:00100000:3.0:1789760749.089915:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692311237136 time 692321416937 state 0x0 ref 1 00000100:00100000:3.0:1789760749.089943:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692321677353 time 692321416937 state 0x1 ref 1 00000100:00100000:3.0:1789760749.089975:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692321677353 time 692335593373 state 0x0 ref 1 00000100:00100000:3.0:1789760749.089986:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554518016:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760749.090029:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56184832, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0x1acc0 00010000:00010000:3.0:1789760749.090039:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2149 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.090055:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760749.090059:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.090065:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2149 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56184832->56188927] (req 56184832->56188927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.090077:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2149 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56184832->56188927] (req 56184832->56188927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760749.097040:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56184832 -> 56188928 00002000:00010000:3.0:1789760749.097045:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 109760 -> 109768 00010000:00010000:3.0:1789760749.097058:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2149 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56184832->56188927] (req 56184832->56188927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.097078:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2149 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56184832->56188927] (req 56184832->56188927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.097088:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2149 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56184832->56188927] (req 56184832->56188927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.097095:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760749.097100:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2149 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56184832->56188927] (req 56184832->56188927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.097109:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2149 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56184832->56188927] (req 56184832->56188927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.097129:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2149 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56184832->56188927] (req 56184832->56188927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.097137:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2149 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56184832->56188927] (req 56184832->56188927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.097855:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554518016:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7864us (8204us total) trans 4294981026 rc 0/0 00000100:00100000:3.0:1789760749.097867:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13728 00000100:00100000:3.0:1789760749.097877:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692321677353 time 692335593373 state 0x2 ref 0 00000001:00080000:3.0:1789760749.113901:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981027 is committed 00000100:00100000:3.0:1789760749.118981:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554518272 00000100:00100000:3.0:1789760749.118990:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554518272 00000100:00100000:3.0:1789760749.119040:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692335853789 time 692354217589 state 0x0 ref 1 00000100:00100000:3.0:1789760749.119055:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692354478005 time 692354217589 state 0x1 ref 1 00000100:00100000:3.0:1789760749.119084:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692354478005 time 692364702943 state 0x0 ref 1 00000100:00100000:3.0:1789760749.119096:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb064ef100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554518272:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760749.119142:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56193024, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0x1acd0 00010000:00010000:3.0:1789760749.119152:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2157 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.119167:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760749.119171:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.119178:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2157 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56193024->56197119] (req 56193024->56197119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.119191:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2157 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56193024->56197119] (req 56193024->56197119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760749.125637:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981028 is committed 00000001:00080000:3.0:1789760749.144155:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981029 is committed 00000100:00100000:3.0:1789760749.180657:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554518784 00000100:00100000:3.0:1789760749.180667:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554518784 00000100:00100000:3.0:1789760749.180794:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692397239389 time 692413818580 state 0x0 ref 1 00000100:00100000:3.0:1789760749.180812:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692414078996 time 692413818580 state 0x1 ref 1 00000100:00100000:3.0:1789760749.180846:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692414078996 time 692426464578 state 0x0 ref 1 00000100:00100000:3.0:1789760749.180860:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554518784:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760749.181031:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56209408, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0x1acf0 00010000:00010000:3.0:1789760749.181042:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2173 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.181058:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760749.181063:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.181070:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2173 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56209408->56213503] (req 56209408->56213503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.181082:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2173 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56209408->56213503] (req 56209408->56213503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760749.191133:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56209408 -> 56213504 00002000:00010000:3.0:1789760749.191139:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 109808 -> 109816 00010000:00010000:3.0:1789760749.191152:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2173 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56209408->56213503] (req 56209408->56213503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.191166:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2173 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56209408->56213503] (req 56209408->56213503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.191174:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2173 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56209408->56213503] (req 56209408->56213503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.191186:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760749.191190:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2173 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56209408->56213503] (req 56209408->56213503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.191198:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2173 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56209408->56213503] (req 56209408->56213503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.191217:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2173 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56209408->56213503] (req 56209408->56213503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.191224:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2173 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56209408->56213503] (req 56209408->56213503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.191467:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554518784:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10605us (10900us total) trans 4294981032 rc 0/0 00000100:00100000:3.0:1789760749.191478:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13734 00000100:00100000:3.0:1789760749.191487:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692414078996 time 692426464578 state 0x2 ref 0 00000001:00080000:3.0:1789760749.224224:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981034 is committed 00000001:00080000:3.0:1789760749.247231:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981035 is committed 00000100:00100000:3.0:1789760749.252331:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554519296 00000100:00100000:3.0:1789760749.252340:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554519296 00000100:00100000:3.0:1789760749.252587:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692459803165 time 692480463325 state 0x0 ref 1 00000100:00100000:3.0:1789760749.252602:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692480723741 time 692480463325 state 0x1 ref 1 00000100:00100000:3.0:1789760749.252631:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692480723741 time 692498249634 state 0x0 ref 1 00000100:00100000:3.0:1789760749.252642:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554519296:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760749.252694:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56225792, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0x1ad10 00010000:00010000:3.0:1789760749.252703:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c218f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.252719:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760749.252723:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.252729:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c218f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56225792->56229887] (req 56225792->56229887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.252743:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c218f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56225792->56229887] (req 56225792->56229887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760749.260574:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56225792 -> 56229888 00002000:00010000:3.0:1789760749.260580:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 109840 -> 109848 00010000:00010000:3.0:1789760749.260594:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c218f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56225792->56229887] (req 56225792->56229887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.260608:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c218f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56225792->56229887] (req 56225792->56229887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.260617:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c218f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56225792->56229887] (req 56225792->56229887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.260629:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760749.260634:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c218f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56225792->56229887] (req 56225792->56229887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.260643:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c218f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56225792->56229887] (req 56225792->56229887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.260661:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c218f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56225792->56229887] (req 56225792->56229887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.260669:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c218f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56225792->56229887] (req 56225792->56229887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.260856:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554519296:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8211us (8611us total) trans 4294981036 rc 0/0 00000100:00100000:3.0:1789760749.260866:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13738 00000100:00100000:3.0:1789760749.260875:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692480723741 time 692498249634 state 0x2 ref 0 00000100:00100000:3.0:1789760749.324411:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554519808 00000100:00100000:3.0:1789760749.324422:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554519808 00000100:00100000:3.0:1789760749.324473:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692535454005 time 692550920285 state 0x0 ref 1 00000100:00100000:3.0:1789760749.324498:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692551180701 time 692550920285 state 0x1 ref 1 00000100:00100000:3.0:1789760749.324518:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692551180701 time 692570137108 state 0x0 ref 1 00000100:00100000:3.0:1789760749.324533:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554519808:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760749.324575:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56242176, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0x1ad30 00010000:00010000:3.0:1789760749.324584:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21ab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.324600:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760749.324605:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.324612:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56242176->56246271] (req 56242176->56246271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.324626:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56242176->56246271] (req 56242176->56246271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760749.334359:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56242176 -> 56246272 00002000:00010000:3.0:1789760749.334365:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 109872 -> 109880 00010000:00010000:3.0:1789760749.334398:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56242176->56246271] (req 56242176->56246271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.334411:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21ab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56242176->56246271] (req 56242176->56246271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.334420:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56242176->56246271] (req 56242176->56246271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.334428:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760749.334433:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56242176->56246271] (req 56242176->56246271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.334441:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21ab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56242176->56246271] (req 56242176->56246271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.334467:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21ab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56242176->56246271] (req 56242176->56246271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.334475:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21ab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56242176->56246271] (req 56242176->56246271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.334563:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554519808:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10027us (10248us total) trans 4294981040 rc 0/0 00000100:00100000:3.0:1789760749.334574:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13742 00000100:00100000:3.0:1789760749.334583:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692551180701 time 692570137108 state 0x2 ref 0 00000100:00100000:3.0:1789760749.380274:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554520320 00000100:00100000:3.0:1789760749.380284:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554520320 00000100:00100000:3.0:1789760749.380346:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692600135092 time 692611667977 state 0x0 ref 1 00000100:00100000:3.0:1789760749.380361:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692611928393 time 692611667977 state 0x1 ref 1 00000100:00100000:3.0:1789760749.380485:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692611928393 time 692626101357 state 0x0 ref 1 00000100:00100000:3.0:1789760749.380499:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe16050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554520320:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760749.380546:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56258560, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0x1ad50 00010000:00010000:3.0:1789760749.380557:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21c7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.380574:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760749.380578:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.380584:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56258560->56262655] (req 56258560->56262655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.380598:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56258560->56262655] (req 56258560->56262655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760749.387275:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56258560 -> 56262656 00002000:00010000:3.0:1789760749.387282:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 109904 -> 109912 00010000:00010000:3.0:1789760749.387295:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56258560->56262655] (req 56258560->56262655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.387367:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56258560->56262655] (req 56258560->56262655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.387474:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56258560->56262655] (req 56258560->56262655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.387485:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760749.387489:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56258560->56262655] (req 56258560->56262655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.387499:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56258560->56262655] (req 56258560->56262655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.387520:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21c7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56258560->56262655] (req 56258560->56262655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.387529:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21c7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56258560->56262655] (req 56258560->56262655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.387623:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe16050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554520320:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7123us (7790us total) trans 4294981044 rc 0/0 00000100:00100000:3.0:1789760749.387635:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13746 00000100:00100000:3.0:1789760749.387644:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692611928393 time 692626101357 state 0x2 ref 0 00000001:00080000:3.0:1789760749.415117:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981046 is committed 00000100:00100000:3.0:1789760749.443271:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554520832 00000100:00100000:3.0:1789760749.443282:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554520832 00000100:00100000:3.0:1789760749.443335:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692653610434 time 692669810753 state 0x0 ref 1 00000100:00100000:3.0:1789760749.443350:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692670071169 time 692669810753 state 0x1 ref 1 00000100:00100000:3.0:1789760749.443410:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692670071169 time 692689028162 state 0x0 ref 1 00000100:00100000:3.0:1789760749.443422:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554520832:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760749.443466:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56274944, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0x1ad70 00010000:00010000:3.0:1789760749.443477:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21e3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.443499:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760749.443504:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.443510:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56274944->56279039] (req 56274944->56279039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.443524:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56274944->56279039] (req 56274944->56279039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760749.450523:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56274944 -> 56279040 00002000:00010000:3.0:1789760749.450529:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 109936 -> 109944 00010000:00010000:3.0:1789760749.450540:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56274944->56279039] (req 56274944->56279039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.450552:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21e3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56274944->56279039] (req 56274944->56279039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.450562:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56274944->56279039] (req 56274944->56279039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.450569:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760749.450574:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56274944->56279039] (req 56274944->56279039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.450584:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21e3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56274944->56279039] (req 56274944->56279039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.450609:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21e3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56274944->56279039] (req 56274944->56279039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.450617:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21e3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56274944->56279039] (req 56274944->56279039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.450915:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554520832:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7488us (7734us total) trans 4294981048 rc 0/0 00000100:00100000:3.0:1789760749.450936:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13750 00000100:00100000:3.0:1789760749.450945:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692670071169 time 692689028162 state 0x2 ref 0 00000100:00100000:3.0:1789760749.488737:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554521088 00000100:00100000:3.0:1789760749.488753:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554521088 00000100:00100000:3.0:1789760749.488807:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692689288578 time 692713725811 state 0x0 ref 1 00000100:00100000:3.0:1789760749.488822:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692713986227 time 692713725811 state 0x1 ref 1 00000100:00100000:3.0:1789760749.488853:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692713986227 time 692734471177 state 0x0 ref 1 00000100:00100000:3.0:1789760749.488865:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554521088:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760749.488912:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56283136, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0x1ad80 00010000:00010000:3.0:1789760749.488921:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21f1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.488950:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760749.488955:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.488962:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56283136->56287231] (req 56283136->56287231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.488976:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c21f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56283136->56287231] (req 56283136->56287231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.515416:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554521344 00000100:00100000:3.0:1789760749.515426:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554521344 00000100:00100000:3.0:1789760749.515471:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692734731593 time 692745701696 state 0x0 ref 1 00000100:00100000:3.0:1789760749.515487:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692745962112 time 692745701696 state 0x1 ref 1 00000100:00100000:3.0:1789760749.515513:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692745962112 time 692761132435 state 0x0 ref 1 00000100:00100000:3.0:1789760749.515524:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554521344:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760749.515566:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56291328, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0x1ad90 00010000:00010000:3.0:1789760749.515574:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c21ff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.515589:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760749.515594:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.515600:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c21ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56291328->56295423] (req 56291328->56295423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.515613:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c21ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56291328->56295423] (req 56291328->56295423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760749.526467:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56291328 -> 56295424 00002000:00010000:3.0:1789760749.526474:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 109968 -> 109976 00010000:00010000:3.0:1789760749.526487:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c21ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56291328->56295423] (req 56291328->56295423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.526500:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c21ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56291328->56295423] (req 56291328->56295423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.526509:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c21ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56291328->56295423] (req 56291328->56295423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.526516:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760749.526522:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c21ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56291328->56295423] (req 56291328->56295423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.526531:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c21ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56291328->56295423] (req 56291328->56295423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.526549:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c21ff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56291328->56295423] (req 56291328->56295423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.526556:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c21ff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56291328->56295423] (req 56291328->56295423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.526640:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554521344:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11113us (11623us total) trans 4294981052 rc 0/0 00000100:00100000:3.0:1789760749.526652:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13754 00000100:00100000:3.0:1789760749.526661:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692745962112 time 692761132435 state 0x2 ref 0 00000100:00100000:3.0:1789760749.546894:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692778450428 time 692792510565 state 0x0 ref 1 00000100:00100000:3.0:1789760749.546911:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554521600:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760749.546967:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56299520, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0x1ada0 00010000:00010000:3.0:1789760749.546977:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c220d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.546993:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760749.546998:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.547003:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c220d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56299520->56303615] (req 56299520->56303615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.547017:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c220d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56299520->56303615] (req 56299520->56303615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760749.554433:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56299520 -> 56303616 00002000:00010000:3.0:1789760749.554541:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 109984 -> 109992 00010000:00010000:3.0:1789760749.554554:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c220d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56299520->56303615] (req 56299520->56303615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.554568:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c220d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56299520->56303615] (req 56299520->56303615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.554583:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c220d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56299520->56303615] (req 56299520->56303615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.554591:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760749.554596:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c220d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56299520->56303615] (req 56299520->56303615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.554605:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c220d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56299520->56303615] (req 56299520->56303615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.554623:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c220d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56299520->56303615] (req 56299520->56303615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.554631:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c220d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56299520->56303615] (req 56299520->56303615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.554938:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554521600:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8019us (9002us total) trans 4294981054 rc 0/0 00000100:00100000:3.0:1789760749.554949:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13756 00000100:00100000:3.0:1789760749.554959:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692778450428 time 692792510565 state 0x2 ref 0 00000100:00100000:3.0:1789760749.616123:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554521984:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12010us (12512us total) trans 4294981057 rc 0/0 00000100:00100000:3.0:1789760749.616134:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13759 00000100:00100000:3.0:1789760749.616143:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692832248983 time 692849711284 state 0x2 ref 0 00000100:00100000:3.0:1789760749.621821:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554522112 00000100:00100000:3.0:1789760749.621831:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554522112 00000100:00100000:3.0:1789760749.621887:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692832248983 time 692849711284 state 0x0 ref 1 00000100:00100000:3.0:1789760749.621972:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692849971700 time 692849711284 state 0x1 ref 1 00000100:00100000:3.0:1789760749.622096:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692849971700 time 692867713204 state 0x0 ref 1 00000100:00100000:3.0:1789760749.622110:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554522112:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760749.622159:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56315904, mtime: 0x6aad94ec, atime: 0x0, ctime: 0x6aad94ec, blocks: 0x1adc0 00010000:00010000:3.0:1789760749.622168:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c2229 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.622185:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760749.622189:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.622196:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c2229 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56315904->56319999] (req 56315904->56319999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.622211:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c2229 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56315904->56319999] (req 56315904->56319999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760749.631728:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56315904 -> 56320000 00002000:00010000:3.0:1789760749.631736:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 110016 -> 110024 00010000:00010000:3.0:1789760749.631749:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c2229 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56315904->56319999] (req 56315904->56319999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.631769:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c2229 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56315904->56319999] (req 56315904->56319999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.631777:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c2229 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56315904->56319999] (req 56315904->56319999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.631784:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760749.631789:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c2229 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56315904->56319999] (req 56315904->56319999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.631799:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c2229 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56315904->56319999] (req 56315904->56319999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.631819:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c2229 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56315904->56319999] (req 56315904->56319999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.631826:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c2229 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56315904->56319999] (req 56315904->56319999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.631974:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554522112:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9861us (10315us total) trans 4294981058 rc 0/0 00000100:00100000:3.0:1789760749.631985:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13760 00000100:00100000:3.0:1789760749.631995:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692849971700 time 692867713204 state 0x2 ref 0 00000100:00100000:3.0:1789760749.637239:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760749.670200:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554522496 00000100:00100000:3.0:1789760749.670210:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554522496 00000100:00100000:3.0:1789760749.670267:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692883372782 time 692903043299 state 0x0 ref 1 00000100:00100000:3.0:1789760749.670282:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692903303715 time 692903043299 state 0x1 ref 1 00000100:00100000:3.0:1789760749.670321:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692903303715 time 692915939547 state 0x0 ref 1 00000100:00100000:3.0:1789760749.670333:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554522496:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760749.670408:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56328192, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0x1add8 00010000:00010000:3.0:1789760749.670558:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c223e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.670578:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=692s v=(0 0) 00010000:00010000:3.0:1789760749.670582:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.670589:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c223e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56328192->56332287] (req 56328192->56332287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.670604:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c223e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56328192->56332287] (req 56328192->56332287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.682473:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760749.757442:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554523136 00000100:00100000:3.0:1789760749.757452:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554523136 00000100:00100000:3.0:1789760749.757501:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692962783640 time 692986904449 state 0x0 ref 1 00000100:00100000:3.0:1789760749.757517:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692987164865 time 692986904449 state 0x1 ref 1 00000100:00100000:3.0:1789760749.757553:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 692987164865 time 693003171924 state 0x0 ref 1 00000100:00100000:3.0:1789760749.757566:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554523136:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760749.757608:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56348672, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0x1ae00 00010000:00010000:3.0:1789760749.757615:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c2261 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.757629:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760749.757633:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.757640:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c2261 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56348672->56352767] (req 56348672->56352767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.757655:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c2261 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56348672->56352767] (req 56348672->56352767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760749.772296:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56348672 -> 56352768 00002000:00010000:3.0:1789760749.772373:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 110080 -> 110088 00010000:00010000:3.0:1789760749.772649:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c2261 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56348672->56352767] (req 56348672->56352767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.772663:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c2261 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56348672->56352767] (req 56348672->56352767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.772673:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c2261 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56348672->56352767] (req 56348672->56352767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.772685:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760749.772690:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c2261 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56348672->56352767] (req 56348672->56352767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.772698:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c2261 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56348672->56352767] (req 56348672->56352767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.772717:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c2261 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56348672->56352767] (req 56348672->56352767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.772794:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c2261 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56348672->56352767] (req 56348672->56352767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760749.838038:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56360960 -> 56365056 00002000:00010000:3.0:1789760749.838044:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 110104 -> 110112 00010000:00010000:3.0:1789760749.838057:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c2276 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56360960->56365055] (req 56360960->56365055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.838070:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c2276 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56360960->56365055] (req 56360960->56365055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.838079:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c2276 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56360960->56365055] (req 56360960->56365055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.838086:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760749.838092:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c2276 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56360960->56365055] (req 56360960->56365055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.838100:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c2276 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56360960->56365055] (req 56360960->56365055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.838233:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c2276 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56360960->56365055] (req 56360960->56365055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.838243:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c2276 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56360960->56365055] (req 56360960->56365055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.838480:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09314380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554523520:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11522us (16245us total) trans 4294981069 rc 0/0 00000100:00100000:3.0:1789760749.838490:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13771 00000100:00100000:3.0:1789760749.838500:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693045721467 time 693072560028 state 0x2 ref 0 00000100:00100000:3.0:1789760749.867856:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760749.914606:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554524032 00000100:00100000:3.0:1789760749.914618:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554524032 00000100:00100000:3.0:1789760749.914674:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693114536498 time 693138253235 state 0x0 ref 1 00000100:00100000:3.0:1789760749.914689:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693138513651 time 693138253235 state 0x1 ref 1 00000100:00100000:3.0:1789760749.914721:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693138513651 time 693160339767 state 0x0 ref 1 00000100:00100000:3.0:1789760749.914734:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06490a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554524032:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760749.914783:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56377344, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0x1ae38 00010000:00010000:3.0:1789760749.914793:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c2292 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.914810:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760749.914814:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760749.914820:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c2292 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56377344->56381439] (req 56377344->56381439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.914835:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c2292 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56377344->56381439] (req 56377344->56381439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.969900:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:3.0:1789760749.981212:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56389632 -> 56393728 00002000:00010000:3.0:1789760749.981218:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 110160 -> 110168 00010000:00010000:3.0:1789760749.981230:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c22a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56389632->56393727] (req 56389632->56393727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.981241:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c22a7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56389632->56393727] (req 56389632->56393727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.981249:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c22a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56389632->56393727] (req 56389632->56393727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.981256:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760749.981261:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c22a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56389632->56393727] (req 56389632->56393727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.981272:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c22a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56389632->56393727] (req 56389632->56393727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.981292:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c22a7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56389632->56393727] (req 56389632->56393727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760749.981364:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c22a7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56389632->56393727] (req 56389632->56393727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760749.981639:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04560000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554524416:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9724us (11744us total) trans 4294981076 rc 0/0 00000100:00100000:3.0:1789760749.981652:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13778 00000100:00100000:3.0:1789760749.981662:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693202777502 time 693217520150 state 0x2 ref 0 00000100:00100000:3.0:1789760750.032867:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760750.046523:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554524928 00000100:00100000:3.0:1789760750.046533:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554524928 00000100:00100000:3.0:1789760750.046602:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693262206667 time 693278687295 state 0x0 ref 1 00000100:00100000:3.0:1789760750.046618:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693278947711 time 693278687295 state 0x1 ref 1 00000100:00100000:3.0:1789760750.046691:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693278947711 time 693292307831 state 0x0 ref 1 00000100:00100000:3.0:1789760750.046706:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554524928:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760750.046754:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56406016, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0x1ae70 00010000:00010000:3.0:1789760750.046766:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c22c3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.046782:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760750.046786:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.046794:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c22c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56406016->56410111] (req 56406016->56410111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.046809:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c22c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56406016->56410111] (req 56406016->56410111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760750.056957:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56406016 -> 56410112 00002000:00010000:3.0:1789760750.056962:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 110192 -> 110200 00010000:00010000:3.0:1789760750.056976:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c22c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56406016->56410111] (req 56406016->56410111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.056997:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c22c3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56406016->56410111] (req 56406016->56410111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.057007:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c22c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56406016->56410111] (req 56406016->56410111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.057015:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760750.057020:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c22c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56406016->56410111] (req 56406016->56410111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.057029:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c22c3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56406016->56410111] (req 56406016->56410111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.057115:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c22c3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56406016->56410111] (req 56406016->56410111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.057125:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c22c3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56406016->56410111] (req 56406016->56410111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.057228:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554524928:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10519us (11245us total) trans 4294981080 rc 0/0 00000100:00100000:3.0:1789760750.057238:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13782 00000100:00100000:3.0:1789760750.057247:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693278947711 time 693292307831 state 0x2 ref 0 00000100:00100000:3.0:1789760750.106410:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554525440 00000100:00100000:3.0:1789760750.106420:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554525440 00000100:00100000:3.0:1789760750.106473:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693322190982 time 693336905453 state 0x0 ref 1 00000100:00100000:3.0:1789760750.106493:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693337165869 time 693336905453 state 0x1 ref 1 00000100:00100000:3.0:1789760750.106514:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693337165869 time 693352133007 state 0x0 ref 1 00000100:00100000:3.0:1789760750.106525:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554525440:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760750.106572:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56422400, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0x1ae90 00010000:00010000:3.0:1789760750.106590:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c22df lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.106606:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760750.106611:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.106618:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c22df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56422400->56426495] (req 56422400->56426495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.106632:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c22df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56422400->56426495] (req 56422400->56426495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760750.114094:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56422400 -> 56426496 00002000:00010000:3.0:1789760750.114099:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 110224 -> 110232 00010000:00010000:3.0:1789760750.114112:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c22df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56422400->56426495] (req 56422400->56426495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.114124:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c22df lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56422400->56426495] (req 56422400->56426495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.114133:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c22df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56422400->56426495] (req 56422400->56426495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.114141:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760750.114145:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c22df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56422400->56426495] (req 56422400->56426495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.114155:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c22df lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56422400->56426495] (req 56422400->56426495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.114175:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c22df lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56422400->56426495] (req 56422400->56426495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.114189:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c22df lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56422400->56426495] (req 56422400->56426495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.114415:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554525440:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7852us (8101us total) trans 4294981084 rc 0/0 00000100:00100000:3.0:1789760750.114428:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13786 00000100:00100000:3.0:1789760750.114436:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693337165869 time 693352133007 state 0x2 ref 0 00000100:00100000:3.0:1789760750.120707:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00010000:00010000:3.0:1789760750.161768:0:8453:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760750.163063:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760750.209067:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981090 is committed 00000100:00100000:3.0:1789760750.225359:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760750.235597:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981091 is committed 00000100:00100000:3.0:1789760750.244030:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554526464 00000100:00100000:3.0:1789760750.244039:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554526464 00000100:00100000:3.0:1789760750.244096:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693447083255 time 693471381988 state 0x0 ref 1 00000100:00100000:3.0:1789760750.244111:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693471642404 time 693471381988 state 0x1 ref 1 00000100:00100000:3.0:1789760750.244145:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693471642404 time 693489764044 state 0x0 ref 1 00000100:00100000:3.0:1789760750.244158:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554526464:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760750.244204:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56455168, mtime: 0x6aad94ed, atime: 0x0, ctime: 0x6aad94ed, blocks: 0x1aed0 00010000:00010000:3.0:1789760750.244213:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c2317 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.244230:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760750.244235:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.244241:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c2317 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56455168->56459263] (req 56455168->56459263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.244254:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c2317 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56455168->56459263] (req 56455168->56459263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760750.254219:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981092 is committed 00000001:00080000:3.0:1789760750.270007:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981093 is committed 00000001:00080000:3.0:1789760750.290188:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981094 is committed 00000001:00080000:3.0:1789760750.309398:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981095 is committed 00000001:00080000:3.0:1789760750.327974:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981096 is committed 00000001:00080000:3.0:1789760750.348507:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981097 is committed 00000001:00080000:3.0:1789760750.370188:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981098 is committed 00000001:00080000:3.0:1789760750.395861:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981099 is committed 00000100:00100000:3.0:1789760750.419038:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760750.427048:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981100 is committed 00000001:00080000:3.0:1789760750.457466:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981101 is committed 00000100:00100000:3.0:1789760750.463648:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760750.481263:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760750.550304:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981108 is committed 00000001:00080000:3.0:1789760750.571011:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981109 is committed 00000100:00100000:3.0:1789760750.647174:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554529408 00000100:00100000:3.0:1789760750.647184:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554529408 00000100:00100000:3.0:1789760750.647254:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693865483303 time 693879627586 state 0x0 ref 1 00000100:00100000:3.0:1789760750.647272:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693879888002 time 693879627586 state 0x1 ref 1 00000100:00100000:3.0:1789760750.647605:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693879888002 time 693893221762 state 0x0 ref 1 00000100:00100000:3.0:1789760750.647619:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06490e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554529408:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760750.647666:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56549376, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0x1af88 00010000:00010000:3.0:1789760750.647676:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c23b8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.647692:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760750.647696:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.647703:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c23b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56549376->56553471] (req 56549376->56553471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.647717:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c23b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56549376->56553471] (req 56549376->56553471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760750.653538:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56549376 -> 56553472 00002000:00010000:3.0:1789760750.653543:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 110472 -> 110480 00010000:00010000:3.0:1789760750.653556:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c23b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56549376->56553471] (req 56549376->56553471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.653568:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c23b8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56549376->56553471] (req 56549376->56553471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.653575:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c23b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56549376->56553471] (req 56549376->56553471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.653583:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760750.653588:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c23b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56549376->56553471] (req 56549376->56553471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.653597:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c23b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56549376->56553471] (req 56549376->56553471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.653617:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c23b8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56549376->56553471] (req 56549376->56553471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.653626:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c23b8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56549376->56553471] (req 56549376->56553471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.653718:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06490e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554529408:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6097us (6674us total) trans 4294981115 rc 0/0 00000100:00100000:3.0:1789760750.653729:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13817 00000100:00100000:3.0:1789760750.653739:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693879888002 time 693893221762 state 0x2 ref 0 00000100:00100000:3.0:1789760750.657630:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554529536 00000100:00100000:3.0:1789760750.657640:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554529536 00000100:00100000:3.0:1789760750.657752:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693879888002 time 693893221762 state 0x0 ref 1 00000100:00100000:3.0:1789760750.657769:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693893482178 time 693893221762 state 0x1 ref 1 00000100:00100000:3.0:1789760750.657845:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693893482178 time 693903462844 state 0x0 ref 1 00000100:00100000:3.0:1789760750.657862:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06491c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554529536:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760750.657908:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56553472, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0x1af90 00010000:00010000:3.0:1789760750.657924:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23bf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.657939:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760750.657943:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.657949:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56553472->56557567] (req 56553472->56557567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.657964:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56553472->56557567] (req 56553472->56557567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760750.663057:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56553472 -> 56557568 00002000:00010000:3.0:1789760750.663067:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 110480 -> 110488 00010000:00010000:3.0:1789760750.663080:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56553472->56557567] (req 56553472->56557567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.663092:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23bf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56553472->56557567] (req 56553472->56557567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.663100:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56553472->56557567] (req 56553472->56557567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.663107:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760750.663112:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56553472->56557567] (req 56553472->56557567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.663120:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23bf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56553472->56557567] (req 56553472->56557567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.663140:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23bf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56553472->56557567] (req 56553472->56557567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.663148:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23bf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56553472->56557567] (req 56553472->56557567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.663588:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06491c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554529536:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5720us (6292us total) trans 4294981116 rc 0/0 00000100:00100000:3.0:1789760750.663599:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13818 00000100:00100000:3.0:1789760750.663608:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693893482178 time 693903462844 state 0x2 ref 0 00000100:00100000:3.0:1789760750.695026:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554529920 00000100:00100000:3.0:1789760750.695035:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554529920 00000100:00100000:3.0:1789760750.695083:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693913818043 time 693927689699 state 0x0 ref 1 00000100:00100000:3.0:1789760750.695098:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693927950115 time 693927689699 state 0x1 ref 1 00000100:00100000:3.0:1789760750.695128:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693927950115 time 693940746956 state 0x0 ref 1 00000100:00100000:3.0:1789760750.695139:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554529920:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760750.695183:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56565760, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0x1afa8 00010000:00010000:3.0:1789760750.695192:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23d4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.695218:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760750.695222:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.695229:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56565760->56569855] (req 56565760->56569855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.695242:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56565760->56569855] (req 56565760->56569855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760750.700661:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56565760 -> 56569856 00002000:00010000:3.0:1789760750.700667:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 110504 -> 110512 00010000:00010000:3.0:1789760750.700743:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56565760->56569855] (req 56565760->56569855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.700758:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56565760->56569855] (req 56565760->56569855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.700768:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56565760->56569855] (req 56565760->56569855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.700775:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760750.700780:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56565760->56569855] (req 56565760->56569855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.700789:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23d4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56565760->56569855] (req 56565760->56569855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.700808:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23d4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56565760->56569855] (req 56565760->56569855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.700816:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23d4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56565760->56569855] (req 56565760->56569855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.700998:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554529920:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5854us (6058us total) trans 4294981119 rc 0/0 00000100:00100000:3.0:1789760750.701008:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13821 00000100:00100000:3.0:1789760750.701017:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693927950115 time 693940746956 state 0x2 ref 0 00000100:00100000:3.0:1789760750.705872:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554530048 00000100:00100000:3.0:1789760750.705880:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554530048 00000100:00100000:3.0:1789760750.705942:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693927950115 time 693940746956 state 0x0 ref 1 00000100:00100000:3.0:1789760750.705956:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693941007372 time 693940746956 state 0x1 ref 1 00000100:00100000:3.0:1789760750.706027:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693941007372 time 693951644758 state 0x0 ref 1 00000100:00100000:3.0:1789760750.706039:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554530048:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760750.706084:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56569856, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0x1afb0 00010000:00010000:3.0:1789760750.706094:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23db lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.706109:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=693s v=(0 0) 00010000:00010000:3.0:1789760750.706114:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.706121:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56569856->56573951] (req 56569856->56573951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.706134:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56569856->56573951] (req 56569856->56573951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760750.717332:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56569856 -> 56573952 00002000:00010000:3.0:1789760750.717338:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 110512 -> 110520 00010000:00010000:3.0:1789760750.717351:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56569856->56573951] (req 56569856->56573951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.717364:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56569856->56573951] (req 56569856->56573951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.717374:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56569856->56573951] (req 56569856->56573951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.717411:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760750.717416:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56569856->56573951] (req 56569856->56573951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.717425:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23db lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56569856->56573951] (req 56569856->56573951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.717445:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23db lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56569856->56573951] (req 56569856->56573951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.717459:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23db lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56569856->56573951] (req 56569856->56573951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.717556:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554530048:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11513us (11770us total) trans 4294981120 rc 0/0 00000100:00100000:3.0:1789760750.717567:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13822 00000100:00100000:3.0:1789760750.717576:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693941007372 time 693951644758 state 0x2 ref 0 00000100:00100000:3.0:1789760750.761536:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554530432 00000100:00100000:3.0:1789760750.761548:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554530432 00000100:00100000:3.0:1789760750.761619:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693970865239 time 693986565331 state 0x0 ref 1 00000100:00100000:3.0:1789760750.761638:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693986825747 time 693986565331 state 0x1 ref 1 00000100:00100000:3.0:1789760750.761681:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693986825747 time 694007299127 state 0x0 ref 1 00000100:00100000:3.0:1789760750.761697:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554530432:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760750.761759:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56582144, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0x1afc8 00010000:00010000:3.0:1789760750.761771:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23f0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.761791:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760750.761796:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.761803:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56582144->56586239] (req 56582144->56586239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.761829:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56582144->56586239] (req 56582144->56586239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760750.771951:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56582144 -> 56586240 00002000:00010000:3.0:1789760750.771964:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 110536 -> 110544 00010000:00010000:3.0:1789760750.772071:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56582144->56586239] (req 56582144->56586239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.772087:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23f0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56582144->56586239] (req 56582144->56586239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.772098:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56582144->56586239] (req 56582144->56586239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.772105:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760750.772111:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56582144->56586239] (req 56582144->56586239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.772120:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23f0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56582144->56586239] (req 56582144->56586239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.772143:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23f0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56582144->56586239] (req 56582144->56586239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.772151:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23f0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56582144->56586239] (req 56582144->56586239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.772346:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554530432:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10644us (11096us total) trans 4294981123 rc 0/0 00000100:00100000:3.0:1789760750.772359:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13825 00000100:00100000:3.0:1789760750.772372:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693986825747 time 694007299127 state 0x2 ref 0 00000100:00100000:3.0:1789760750.784712:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554530560 00000100:00100000:3.0:1789760750.784725:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554530560 00000100:00100000:3.0:1789760750.784796:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 693986825747 time 694007299127 state 0x0 ref 1 00000100:00100000:3.0:1789760750.784815:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694007559543 time 694007299127 state 0x1 ref 1 00000100:00100000:3.0:1789760750.784864:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694007559543 time 694030481888 state 0x0 ref 1 00000100:00100000:3.0:1789760750.784878:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554530560:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760750.784953:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56586240, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0x1afd0 00010000:00010000:3.0:1789760750.784967:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23f7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.784989:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760750.784994:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.785003:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56586240->56590335] (req 56586240->56590335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.785021:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56586240->56590335] (req 56586240->56590335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760750.798984:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56586240 -> 56590336 00002000:00010000:3.0:1789760750.798992:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 110544 -> 110552 00010000:00010000:3.0:1789760750.799011:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56586240->56590335] (req 56586240->56590335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.799028:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23f7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56586240->56590335] (req 56586240->56590335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.799040:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56586240->56590335] (req 56586240->56590335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.799049:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760750.799054:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56586240->56590335] (req 56586240->56590335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.799065:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23f7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56586240->56590335] (req 56586240->56590335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.799093:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23f7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56586240->56590335] (req 56586240->56590335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.799103:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c23f7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56586240->56590335] (req 56586240->56590335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.799546:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554530560:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14660us (15462us total) trans 4294981124 rc 0/0 00000100:00100000:3.0:1789760750.799562:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13826 00000100:00100000:3.0:1789760750.799574:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694007559543 time 694030481888 state 0x2 ref 0 00000100:00100000:3.0:1789760750.832913:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554530944 00000100:00100000:3.0:1789760750.832935:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554530944 00000100:00100000:3.0:1789760750.833006:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694051073031 time 694065893921 state 0x0 ref 1 00000100:00100000:3.0:1789760750.833025:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694066154337 time 694065893921 state 0x1 ref 1 00000100:00100000:3.0:1789760750.833091:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694066154337 time 694078707869 state 0x0 ref 1 00000100:00100000:3.0:1789760750.833105:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554530944:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760750.833166:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56598528, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0x1afe8 00010000:00010000:3.0:1789760750.833177:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c240c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.833197:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760750.833202:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.833210:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c240c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56598528->56602623] (req 56598528->56602623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.833227:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c240c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56598528->56602623] (req 56598528->56602623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760750.840141:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56598528 -> 56602624 00002000:00010000:3.0:1789760750.840147:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 110568 -> 110576 00010000:00010000:3.0:1789760750.840166:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c240c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56598528->56602623] (req 56598528->56602623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.840181:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c240c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56598528->56602623] (req 56598528->56602623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.840191:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c240c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56598528->56602623] (req 56598528->56602623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.840199:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760750.840204:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c240c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56598528->56602623] (req 56598528->56602623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.840214:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c240c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56598528->56602623] (req 56598528->56602623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.840239:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c240c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56598528->56602623] (req 56598528->56602623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.840248:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c240c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56598528->56602623] (req 56598528->56602623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.843088:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554530944:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9965us (10391us total) trans 4294981127 rc 0/0 00000100:00100000:3.0:1789760750.843112:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13829 00000100:00100000:3.0:1789760750.843125:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694066154337 time 694078707869 state 0x2 ref 0 00000100:00100000:3.0:1789760750.850691:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554531072 00000100:00100000:3.0:1789760750.850705:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554531072 00000100:00100000:3.0:1789760750.850780:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694066154337 time 694078707869 state 0x0 ref 1 00000100:00100000:3.0:1789760750.850798:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694078968285 time 694078707869 state 0x1 ref 1 00000100:00100000:3.0:1789760750.850837:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694078968285 time 694096455579 state 0x0 ref 1 00000100:00100000:3.0:1789760750.850985:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554531072:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760750.851068:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56602624, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0x1aff0 00010000:00010000:3.0:1789760750.851080:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2413 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.851100:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760750.851106:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.851114:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2413 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56602624->56606719] (req 56602624->56606719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.851132:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2413 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56602624->56606719] (req 56602624->56606719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760750.862087:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56602624 -> 56606720 00002000:00010000:3.0:1789760750.862098:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 110576 -> 110584 00010000:00010000:3.0:1789760750.862115:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2413 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56602624->56606719] (req 56602624->56606719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.862136:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2413 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56602624->56606719] (req 56602624->56606719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.862146:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2413 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56602624->56606719] (req 56602624->56606719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.862154:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760750.862160:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2413 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56602624->56606719] (req 56602624->56606719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.862170:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2413 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56602624->56606719] (req 56602624->56606719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.862195:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2413 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56602624->56606719] (req 56602624->56606719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.862204:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2413 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56602624->56606719] (req 56602624->56606719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.862342:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554531072:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11354us (11865us total) trans 4294981128 rc 0/0 00000100:00100000:3.0:1789760750.862355:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13830 00000100:00100000:3.0:1789760750.862369:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694078968285 time 694096455579 state 0x2 ref 0 00000001:00080000:3.0:1789760750.879488:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981129 is committed 00000001:00080000:3.0:1789760750.894262:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981130 is committed 00000100:00100000:3.0:1789760750.901517:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:3.0:1789760750.912612:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56614912 -> 56619008 00002000:00010000:3.0:1789760750.912619:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 110600 -> 110608 00010000:00010000:3.0:1789760750.912637:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c2428 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56614912->56619007] (req 56614912->56619007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.912653:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c2428 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56614912->56619007] (req 56614912->56619007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.912664:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c2428 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56614912->56619007] (req 56614912->56619007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.912672:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760750.912678:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c2428 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56614912->56619007] (req 56614912->56619007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.912689:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c2428 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56614912->56619007] (req 56614912->56619007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.912715:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c2428 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56614912->56619007] (req 56614912->56619007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.912722:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c2428 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56614912->56619007] (req 56614912->56619007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.913053:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04563800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554531456:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10853us (11543us total) trans 4294981131 rc 0/0 00000100:00100000:3.0:1789760750.913068:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13833 00000100:00100000:3.0:1789760750.913093:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694130760321 time 694147798355 state 0x2 ref 0 00000100:00100000:3.0:1789760750.923087:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554531584 00000100:00100000:3.0:1789760750.923104:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554531584 00000100:00100000:3.0:1789760750.957842:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760750.971075:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554531968 00000100:00100000:3.0:1789760750.971088:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554531968 00000100:00100000:3.0:1789760750.971167:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694187891063 time 694204138934 state 0x0 ref 1 00000100:00100000:3.0:1789760750.971186:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694204399350 time 694204138934 state 0x1 ref 1 00000100:00100000:3.0:1789760750.971227:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694204399350 time 694216845214 state 0x0 ref 1 00000100:00100000:3.0:1789760750.971240:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a272300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554531968:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760750.971305:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56631296, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0x1b028 00010000:00010000:3.0:1789760750.971317:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2444 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.971338:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760750.971343:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760750.971350:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2444 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56631296->56635391] (req 56631296->56635391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.971366:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2444 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56631296->56635391] (req 56631296->56635391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760750.980614:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56631296 -> 56635392 00002000:00010000:3.0:1789760750.980621:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 110632 -> 110640 00010000:00010000:3.0:1789760750.980639:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2444 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56631296->56635391] (req 56631296->56635391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.980655:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2444 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56631296->56635391] (req 56631296->56635391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.980665:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2444 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56631296->56635391] (req 56631296->56635391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.980740:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760750.980747:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2444 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56631296->56635391] (req 56631296->56635391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.980758:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2444 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56631296->56635391] (req 56631296->56635391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.980789:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2444 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56631296->56635391] (req 56631296->56635391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760750.980798:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2444 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56631296->56635391] (req 56631296->56635391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760750.980979:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a272300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554531968:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9727us (10396us total) trans 4294981135 rc 0/0 00000100:00100000:3.0:1789760750.980994:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13837 00000100:00100000:3.0:1789760750.981006:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694204399350 time 694216845214 state 0x2 ref 0 00000100:00100000:3.0:1789760751.030013:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554532480 00000100:00100000:3.0:1789760751.030037:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554532480 00000100:00100000:3.0:1789760751.030113:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694244964552 time 694258942934 state 0x0 ref 1 00000100:00100000:3.0:1789760751.030133:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694259203350 time 694258942934 state 0x1 ref 1 00000100:00100000:3.0:1789760751.030171:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694259203350 time 694275788211 state 0x0 ref 1 00000100:00100000:3.0:1789760751.030194:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554532480:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760751.030260:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56647680, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0x1b048 00010000:00010000:3.0:1789760751.030272:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2460 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.030305:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760751.030311:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.030319:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2460 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56647680->56651775] (req 56647680->56651775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.030338:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2460 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56647680->56651775] (req 56647680->56651775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760751.041125:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56647680 -> 56651776 00002000:00010000:3.0:1789760751.041133:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 110664 -> 110672 00010000:00010000:3.0:1789760751.041152:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2460 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56647680->56651775] (req 56647680->56651775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.041168:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2460 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56647680->56651775] (req 56647680->56651775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.041177:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2460 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56647680->56651775] (req 56647680->56651775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.041253:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760751.041260:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2460 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56647680->56651775] (req 56647680->56651775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.041271:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2460 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56647680->56651775] (req 56647680->56651775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.041305:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2460 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56647680->56651775] (req 56647680->56651775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.041313:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2460 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56647680->56651775] (req 56647680->56651775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.041476:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554532480:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11286us (11591us total) trans 4294981139 rc 0/0 00000100:00100000:3.0:1789760751.041490:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13841 00000100:00100000:3.0:1789760751.041502:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694259203350 time 694275788211 state 0x2 ref 0 00002000:00010000:3.0:1789760751.114883:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56659968 -> 56664064 00002000:00010000:3.0:1789760751.114892:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 110688 -> 110696 00010000:00010000:3.0:1789760751.114911:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c2475 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56659968->56664063] (req 56659968->56664063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.115898:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c2475 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56659968->56664063] (req 56659968->56664063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.115909:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c2475 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56659968->56664063] (req 56659968->56664063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.115933:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760751.115939:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c2475 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56659968->56664063] (req 56659968->56664063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.115950:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c2475 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56659968->56664063] (req 56659968->56664063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.115987:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c2475 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56659968->56664063] (req 56659968->56664063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.115997:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c2475 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56659968->56664063] (req 56659968->56664063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.116134:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554532864:12345-192.168.203.22@tcp:4:dd.0 Request processed in 25150us (26301us total) trans 4294981142 rc 0/0 00000100:00100000:3.0:1789760751.116151:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13844 00000100:00100000:3.0:1789760751.116163:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694309677974 time 694336568886 state 0x2 ref 0 00000100:00100000:3.0:1789760751.121923:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554532992 00000100:00100000:3.0:1789760751.121937:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554532992 00000100:00100000:3.0:1789760751.125854:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694309677974 time 694336568886 state 0x0 ref 1 00000100:00100000:3.0:1789760751.125894:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694336829302 time 694336568886 state 0x1 ref 1 00000100:00100000:3.0:1789760751.125968:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694336829302 time 694371585059 state 0x0 ref 1 00000100:00100000:3.0:1789760751.125989:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f84000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554532992:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760751.126059:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56664064, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0x1b068 00010000:00010000:3.0:1789760751.126071:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c247c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.126093:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760751.126098:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.127093:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c247c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56664064->56668159] (req 56664064->56668159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.127562:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c247c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56664064->56668159] (req 56664064->56668159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760751.139956:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56664064 -> 56668160 00002000:00010000:3.0:1789760751.139963:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 110696 -> 110704 00010000:00010000:3.0:1789760751.140053:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c247c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56664064->56668159] (req 56664064->56668159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.140077:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c247c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56664064->56668159] (req 56664064->56668159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.140088:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c247c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56664064->56668159] (req 56664064->56668159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.140096:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760751.140102:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c247c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56664064->56668159] (req 56664064->56668159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.140113:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c247c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56664064->56668159] (req 56664064->56668159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.140139:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c247c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56664064->56668159] (req 56664064->56668159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.140148:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c247c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56664064->56668159] (req 56664064->56668159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.140335:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f84000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554532992:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14348us (18560us total) trans 4294981143 rc 0/0 00000100:00100000:3.0:1789760751.140349:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13845 00000100:00100000:3.0:1789760751.140360:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694336829302 time 694371585059 state 0x2 ref 0 00000100:00100000:3.0:1789760751.186101:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554533376 00000100:00100000:3.0:1789760751.186116:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554533376 00000100:00100000:3.0:1789760751.186181:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694393518796 time 694414186728 state 0x0 ref 1 00000100:00100000:3.0:1789760751.186197:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694414447144 time 694414186728 state 0x1 ref 1 00000100:00100000:3.0:1789760751.186301:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694414447144 time 694431917763 state 0x0 ref 1 00000100:00100000:3.0:1789760751.186316:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f60e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554533376:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760751.186365:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56676352, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0x1b080 00010000:00010000:3.0:1789760751.186375:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2491 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.186456:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760751.186463:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.186473:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2491 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56676352->56680447] (req 56676352->56680447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.186489:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2491 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56676352->56680447] (req 56676352->56680447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.200492:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554533504 00000100:00100000:3.0:1789760751.200506:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554533504 00000100:00100000:3.0:1789760751.200565:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694414447144 time 694431917763 state 0x0 ref 1 00000100:00100000:3.0:1789760751.200581:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694432178179 time 694431917763 state 0x1 ref 1 00000100:00100000:3.0:1789760751.200607:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694432178179 time 694446226347 state 0x0 ref 1 00000100:00100000:3.0:1789760751.200621:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554533504:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760751.200667:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56680448, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0x1b088 00010000:00010000:3.0:1789760751.200681:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2498 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.200698:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760751.200702:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.200709:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2498 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56680448->56684543] (req 56680448->56684543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.200723:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2498 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56680448->56684543] (req 56680448->56684543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760751.227987:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56680448 -> 56684544 00002000:00010000:3.0:1789760751.227997:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 110728 -> 110736 00010000:00010000:3.0:1789760751.228010:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2498 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56680448->56684543] (req 56680448->56684543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.228023:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2498 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56680448->56684543] (req 56680448->56684543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.228033:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2498 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56680448->56684543] (req 56680448->56684543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.228047:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760751.228052:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2498 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56680448->56684543] (req 56680448->56684543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.228062:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2498 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56680448->56684543] (req 56680448->56684543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.228080:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2498 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56680448->56684543] (req 56680448->56684543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.228088:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2498 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56680448->56684543] (req 56680448->56684543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.228544:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554533504:12345-192.168.203.22@tcp:4:dd.0 Request processed in 27919us (28363us total) trans 4294981147 rc 0/0 00000100:00100000:3.0:1789760751.228556:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13849 00000100:00100000:3.0:1789760751.228566:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694432178179 time 694446226347 state 0x2 ref 0 00000001:00080000:3.0:1789760751.290607:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981150 is committed 00000100:00100000:3.0:1789760751.301065:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554534016 00000100:00100000:3.0:1789760751.301079:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554534016 00000100:00100000:3.0:1789760751.301136:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694497605099 time 694525136717 state 0x0 ref 1 00000100:00100000:3.0:1789760751.301151:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694525397133 time 694525136717 state 0x1 ref 1 00000100:00100000:3.0:1789760751.301236:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694525397133 time 694546853843 state 0x0 ref 1 00000100:00100000:3.0:1789760751.301250:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554534016:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760751.301297:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56696832, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0x1b0a8 00010000:00010000:3.0:1789760751.301306:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c24b4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.301322:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760751.301327:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.301333:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c24b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56696832->56700927] (req 56696832->56700927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.301347:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c24b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56696832->56700927] (req 56696832->56700927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760751.313493:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981151 is committed 00000001:00080000:3.0:1789760751.331780:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981152 is committed 00000001:00080000:3.0:1789760751.352457:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981153 is committed 00000001:00080000:3.0:1789760751.374108:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981154 is committed 00000100:00100000:3.0:1789760751.381208:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554534528 00000100:00100000:3.0:1789760751.381217:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554534528 00000100:00100000:3.0:1789760751.381271:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694587109434 time 694606649911 state 0x0 ref 1 00000100:00100000:3.0:1789760751.381292:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694606910327 time 694606649911 state 0x1 ref 1 00000100:00100000:3.0:1789760751.381329:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694606910327 time 694626947519 state 0x0 ref 1 00000100:00100000:3.0:1789760751.381341:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09314380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554534528:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760751.381428:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56713216, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0x1b0c8 00010000:00010000:3.0:1789760751.381437:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c24d0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.381453:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760751.381458:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.381464:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c24d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56713216->56717311] (req 56713216->56717311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.381478:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c24d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56713216->56717311] (req 56713216->56717311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760751.391766:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56713216 -> 56717312 00002000:00010000:3.0:1789760751.391776:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 110792 -> 110800 00010000:00010000:3.0:1789760751.391790:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c24d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56713216->56717311] (req 56713216->56717311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.391803:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c24d0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56713216->56717311] (req 56713216->56717311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.391813:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c24d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56713216->56717311] (req 56713216->56717311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.391820:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760751.391826:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c24d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56713216->56717311] (req 56713216->56717311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.391840:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c24d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56713216->56717311] (req 56713216->56717311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.391860:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c24d0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56713216->56717311] (req 56713216->56717311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.391868:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c24d0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56713216->56717311] (req 56713216->56717311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.391980:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09314380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554534528:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10635us (10874us total) trans 4294981155 rc 0/0 00000100:00100000:3.0:1789760751.391991:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13857 00000100:00100000:3.0:1789760751.392000:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694606910327 time 694626947519 state 0x2 ref 0 00000001:00080000:3.0:1789760751.419609:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981157 is committed 00000001:00080000:3.0:1789760751.439683:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981158 is committed 00000100:00100000:3.0:1789760751.446594:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554535040 00000100:00100000:3.0:1789760751.446604:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554535040 00000100:00100000:3.0:1789760751.446658:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694658346907 time 694676655166 state 0x0 ref 1 00000100:00100000:3.0:1789760751.446683:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694676915582 time 694676655166 state 0x1 ref 1 00000100:00100000:3.0:1789760751.446712:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694676915582 time 694692330167 state 0x0 ref 1 00000100:00100000:3.0:1789760751.446725:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554535040:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760751.446774:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56729600, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0x1b0e8 00010000:00010000:3.0:1789760751.446784:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c24ec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.446801:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760751.446805:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.446812:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c24ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56729600->56733695] (req 56729600->56733695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.446827:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c24ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56729600->56733695] (req 56729600->56733695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760751.452157:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56729600 -> 56733696 00002000:00010000:3.0:1789760751.452162:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 110824 -> 110832 00010000:00010000:3.0:1789760751.452176:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c24ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56729600->56733695] (req 56729600->56733695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.452189:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c24ec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56729600->56733695] (req 56729600->56733695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.452198:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c24ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56729600->56733695] (req 56729600->56733695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.452206:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760751.452211:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c24ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56729600->56733695] (req 56729600->56733695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.452226:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c24ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56729600->56733695] (req 56729600->56733695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.452247:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c24ec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56729600->56733695] (req 56729600->56733695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.452255:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c24ec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56729600->56733695] (req 56729600->56733695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.452550:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554535040:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5822us (6204us total) trans 4294981159 rc 0/0 00000100:00100000:3.0:1789760751.452562:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13861 00000100:00100000:3.0:1789760751.452571:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694676915582 time 694692330167 state 0x2 ref 0 00000100:00100000:3.0:1789760751.506702:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573904768 00000100:00100000:3.0:1789760751.506706:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573904768 00000100:00100000:3.0:1789760751.506728:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573904896 00000100:00100000:3.0:1789760751.506731:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573904896 00000100:00100000:3.0:1789760751.506743:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 338 00000100:00100000:3.0:1789760751.506749:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3648:x1876699573904768:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1789760751.506769:0:8440:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:3.0:1789760751.506802:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3648:x1876699573904768:12345-0@lo:13:osp-pre-1-1.0 Request processed in 54us (664us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760751.506809:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 338 00000100:00100000:3.0:1789760751.506821:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 339 00000100:00100000:3.0:1789760751.506827:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09316680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3649:x1876699573904896:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760751.506837:0:8440:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:3.0:1789760751.506930:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09316680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3649:x1876699573904896:12345-0@lo:13:osp-pre-1-0.0 Request processed in 102us (303us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760751.506937:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 339 00000100:00100000:3.0:1789760751.520066:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554535552 00000100:00100000:3.0:1789760751.520083:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554535552 00000100:00100000:3.0:1789760751.520147:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694719987200 time 694745272506 state 0x0 ref 1 00000100:00100000:3.0:1789760751.520161:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694745532922 time 694745272506 state 0x1 ref 1 00000100:00100000:3.0:1789760751.520446:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694745532922 time 694766062957 state 0x0 ref 1 00000100:00100000:3.0:1789760751.520463:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554535552:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760751.520522:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56745984, mtime: 0x6aad94ee, atime: 0x0, ctime: 0x6aad94ee, blocks: 0x1b108 00010000:00010000:3.0:1789760751.520540:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2508 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.520557:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760751.520562:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.520568:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2508 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56745984->56750079] (req 56745984->56750079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.520582:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2508 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56745984->56750079] (req 56745984->56750079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760751.526823:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56745984 -> 56750080 00002000:00010000:3.0:1789760751.526829:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 110856 -> 110864 00010000:00010000:3.0:1789760751.526843:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2508 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56745984->56750079] (req 56745984->56750079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.526857:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2508 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56745984->56750079] (req 56745984->56750079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.526866:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2508 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56745984->56750079] (req 56745984->56750079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.526873:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760751.526878:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2508 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56745984->56750079] (req 56745984->56750079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.526888:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2508 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56745984->56750079] (req 56745984->56750079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.526921:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2508 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56745984->56750079] (req 56745984->56750079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.526930:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2508 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56745984->56750079] (req 56745984->56750079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.527026:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554535552:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6564us (10483us total) trans 4294981163 rc 0/0 00000100:00100000:3.0:1789760751.527037:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13865 00000100:00100000:3.0:1789760751.527047:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694745532922 time 694766062957 state 0x2 ref 0 00000001:00080000:3.0:1789760751.541280:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981164 is committed 00000001:00080000:3.0:1789760751.559324:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981165 is committed 00000100:00100000:3.0:1789760751.594867:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760751.662146:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760751.690063:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554536576 00000100:00100000:3.0:1789760751.690074:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554536576 00000100:00100000:3.0:1789760751.690138:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694890860435 time 694907943467 state 0x0 ref 1 00000100:00100000:3.0:1789760751.690155:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694908203883 time 694907943467 state 0x1 ref 1 00000100:00100000:3.0:1789760751.690235:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694908203883 time 694935853758 state 0x0 ref 1 00000100:00100000:3.0:1789760751.690248:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554536576:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760751.690291:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56778752, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0x1b148 00010000:00010000:3.0:1789760751.690301:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2540 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.690317:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=694s v=(0 0) 00010000:00010000:3.0:1789760751.690320:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.690327:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2540 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56778752->56782847] (req 56778752->56782847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.690341:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2540 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56778752->56782847] (req 56778752->56782847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760751.697368:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56778752 -> 56782848 00002000:00010000:3.0:1789760751.697374:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 110920 -> 110928 00010000:00010000:3.0:1789760751.697485:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2540 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56778752->56782847] (req 56778752->56782847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.697499:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2540 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56778752->56782847] (req 56778752->56782847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.697509:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2540 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56778752->56782847] (req 56778752->56782847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.697517:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760751.697522:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2540 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56778752->56782847] (req 56778752->56782847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.697532:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2540 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56778752->56782847] (req 56778752->56782847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.697551:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2540 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56778752->56782847] (req 56778752->56782847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.697559:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2540 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56778752->56782847] (req 56778752->56782847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.697811:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554536576:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7560us (7971us total) trans 4294981171 rc 0/0 00000100:00100000:3.0:1789760751.697823:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13873 00000100:00100000:3.0:1789760751.697833:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694908203883 time 694935853758 state 0x2 ref 0 00002000:00010000:3.0:1789760751.760278:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56795136 -> 56799232 00002000:00010000:3.0:1789760751.760284:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 110952 -> 110960 00010000:00010000:3.0:1789760751.760297:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c255c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56795136->56799231] (req 56795136->56799231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.760309:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c255c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56795136->56799231] (req 56795136->56799231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.760318:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c255c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56795136->56799231] (req 56795136->56799231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.760326:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760751.760331:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c255c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56795136->56799231] (req 56795136->56799231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.760340:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c255c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56795136->56799231] (req 56795136->56799231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.760361:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c255c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56795136->56799231] (req 56795136->56799231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.760369:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c255c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56795136->56799231] (req 56795136->56799231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.760550:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04544a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554537088:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10080us (10454us total) trans 4294981175 rc 0/0 00000100:00100000:3.0:1789760751.760564:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13877 00000100:00100000:3.0:1789760751.760573:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 694983685921 time 694996072191 state 0x2 ref 0 00000100:00100000:3.0:1789760751.764206:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760751.798135:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760751.819206:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981180 is committed 00000100:00100000:3.0:1789760751.837634:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760751.854923:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554538112 00000100:00100000:3.0:1789760751.854933:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554538112 00000100:00100000:3.0:1789760751.854989:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 695070930223 time 695083460194 state 0x0 ref 1 00000100:00100000:3.0:1789760751.855005:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 695083720610 time 695083460194 state 0x1 ref 1 00000100:00100000:3.0:1789760751.855160:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 695083720610 time 695100776482 state 0x0 ref 1 00000100:00100000:3.0:1789760751.855174:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554538112:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760751.855220:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56827904, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0x1b1a8 00010000:00010000:3.0:1789760751.855229:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2594 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.855246:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760751.855251:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760751.855257:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2594 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56827904->56831999] (req 56827904->56831999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760751.855270:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2594 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56827904->56831999] (req 56827904->56831999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760751.944936:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760751.981264:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:3.0:1789760752.016832:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 56864768 -> 56868864 00002000:00010000:3.0:1789760752.016838:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 111088 -> 111096 00010000:00010000:3.0:1789760752.016852:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c25d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56864768->56868863] (req 56864768->56868863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.016865:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c25d3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56864768->56868863] (req 56864768->56868863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.016873:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c25d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56864768->56868863] (req 56864768->56868863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.016881:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760752.016886:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c25d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56864768->56868863] (req 56864768->56868863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.016896:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c25d3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56864768->56868863] (req 56864768->56868863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.016924:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c25d3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56864768->56868863] (req 56864768->56868863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.016932:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c25d3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56864768->56868863] (req 56864768->56868863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.017038:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554539264:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11561us (11828us total) trans 4294981192 rc 0/0 00000100:00100000:3.0:1789760752.017049:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13894 00000100:00100000:3.0:1789760752.017059:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 695227979286 time 695251081018 state 0x2 ref 0 00000100:00100000:3.0:1789760752.061138:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554539648 00000100:00100000:3.0:1789760752.061158:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554539648 00000100:00100000:3.0:1789760752.061214:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 695273715865 time 695290892866 state 0x0 ref 1 00000100:00100000:3.0:1789760752.061229:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 695291153282 time 695290892866 state 0x1 ref 1 00000100:00100000:3.0:1789760752.061263:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 695291153282 time 695306881665 state 0x0 ref 1 00000100:00100000:3.0:1789760752.061274:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a270700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554539648:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760752.061325:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56877056, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0x1b208 00010000:00010000:3.0:1789760752.061335:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c25e8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.061349:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760752.061353:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.061360:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c25e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56877056->56881151] (req 56877056->56881151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.061373:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c25e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56877056->56881151] (req 56877056->56881151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.070947:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981195 is committed 00000100:00100000:3.0:1789760752.082980:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554539776 00000100:00100000:3.0:1789760752.082992:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554539776 00000100:00100000:3.0:1789760752.083121:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 695291153282 time 695306881665 state 0x0 ref 1 00000100:00100000:3.0:1789760752.083137:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 695307142081 time 695306881665 state 0x1 ref 1 00000100:00100000:3.0:1789760752.083213:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 695307142081 time 695328831042 state 0x0 ref 1 00000100:00100000:3.0:1789760752.083227:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04547800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554539776:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760752.083276:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56881152, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0x1b210 00010000:00010000:3.0:1789760752.083286:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c25ef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.083303:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760752.083307:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.083313:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c25ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56881152->56885247] (req 56881152->56885247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.083326:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c25ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56881152->56885247] (req 56881152->56885247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.133796:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981198 is committed 00000100:00100000:3.0:1789760752.154473:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760752.200011:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760752.269550:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981206 is committed 00000100:00100000:3.0:1789760752.279643:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554541184 00000100:00100000:3.0:1789760752.279654:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554541184 00000100:00100000:3.0:1789760752.279718:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 695488957162 time 695500821986 state 0x0 ref 1 00000100:00100000:3.0:1789760752.279733:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 695501082402 time 695500821986 state 0x1 ref 1 00000100:00100000:3.0:1789760752.279765:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 695501082402 time 695525383374 state 0x0 ref 1 00000100:00100000:3.0:1789760752.279777:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554541184:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760752.279831:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56926208, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0x1b268 00010000:00010000:3.0:1789760752.279840:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c263c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.279856:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760752.279861:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.279866:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c263c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56926208->56930303] (req 56926208->56930303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.279880:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c263c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56926208->56930303] (req 56926208->56930303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.289362:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981207 is committed 00000001:00080000:3.0:1789760752.314963:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981209 is committed 00000001:00080000:3.0:1789760752.344599:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981210 is committed 00000001:00080000:3.0:1789760752.363182:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981211 is committed 00000001:00080000:3.0:1789760752.377064:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981212 is committed 00000100:00100000:3.0:1789760752.472841:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760752.518318:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760752.545620:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554543232 00000100:00100000:3.0:1789760752.545633:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554543232 00000100:00100000:3.0:1789760752.545691:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 695764396877 time 695780741126 state 0x0 ref 1 00000100:00100000:3.0:1789760752.545706:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 695781001542 time 695780741126 state 0x1 ref 1 00000100:00100000:3.0:1789760752.545739:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 695781001542 time 695791357430 state 0x0 ref 1 00000100:00100000:3.0:1789760752.545754:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554543232:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760752.545925:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 56991744, mtime: 0x6aad94ef, atime: 0x0, ctime: 0x6aad94ef, blocks: 0x1b2e8 00010000:00010000:3.0:1789760752.545934:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c26ac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.545950:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=695s v=(0 0) 00010000:00010000:3.0:1789760752.545955:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.545962:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c26ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56991744->56995839] (req 56991744->56995839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.545975:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c26ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [56991744->56995839] (req 56991744->56995839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.559027:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760752.608921:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760752.626002:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760752.668600:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760752.692093:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:3.0:1789760752.719629:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57028608 -> 57032704 00002000:00010000:3.0:1789760752.719636:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 111408 -> 111416 00010000:00010000:3.0:1789760752.719649:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c26eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57028608->57032703] (req 57028608->57032703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.719662:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c26eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57028608->57032703] (req 57028608->57032703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.719671:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c26eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57028608->57032703] (req 57028608->57032703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.719679:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760752.719684:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c26eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57028608->57032703] (req 57028608->57032703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.719694:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c26eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57028608->57032703] (req 57028608->57032703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.719714:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c26eb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57028608->57032703] (req 57028608->57032703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.719722:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c26eb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57028608->57032703] (req 57028608->57032703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.719857:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554544384:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5631us (6084us total) trans 4294981232 rc 0/0 00000100:00100000:3.0:1789760752.719868:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13934 00000100:00100000:3.0:1789760752.719878:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 695938177999 time 695959829112 state 0x2 ref 0 00000100:00100000:3.0:1789760752.723975:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760752.750929:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981234 is committed 00000100:00100000:3.0:1789760752.757167:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760752.771229:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554544896 00000100:00100000:3.0:1789760752.771239:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554544896 00000100:00100000:3.0:1789760752.771559:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 695987015927 time 696003606018 state 0x0 ref 1 00000100:00100000:3.0:1789760752.771575:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 696003866434 time 696003606018 state 0x1 ref 1 00000100:00100000:3.0:1789760752.771609:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 696003866434 time 696017227293 state 0x0 ref 1 00000100:00100000:3.0:1789760752.771623:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554544896:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760752.771680:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57044992, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0x1b350 00010000:00010000:3.0:1789760752.771690:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2707 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.771705:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=696s v=(0 0) 00010000:00010000:3.0:1789760752.771709:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.771715:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2707 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57044992->57049087] (req 57044992->57049087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.771728:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2707 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57044992->57049087] (req 57044992->57049087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760752.780443:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57044992 -> 57049088 00002000:00010000:3.0:1789760752.780450:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 111440 -> 111448 00010000:00010000:3.0:1789760752.780463:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2707 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57044992->57049087] (req 57044992->57049087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.780476:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2707 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57044992->57049087] (req 57044992->57049087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.780484:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2707 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57044992->57049087] (req 57044992->57049087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.780491:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760752.780496:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2707 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57044992->57049087] (req 57044992->57049087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.780505:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2707 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57044992->57049087] (req 57044992->57049087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.780525:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2707 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57044992->57049087] (req 57044992->57049087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.780533:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2707 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57044992->57049087] (req 57044992->57049087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760752.780685:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554544896:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9060us (9564us total) trans 4294981236 rc 0/0 00000100:00100000:3.0:1789760752.780696:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13938 00000100:00100000:3.0:1789760752.780705:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 696003866434 time 696017227293 state 0x2 ref 0 00000100:00100000:3.0:1789760752.849706:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554545408 00000100:00100000:3.0:1789760752.849717:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554545408 00000100:00100000:3.0:1789760752.849773:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 696045867152 time 696061556337 state 0x0 ref 1 00000100:00100000:3.0:1789760752.849787:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 696061816753 time 696061556337 state 0x1 ref 1 00000100:00100000:3.0:1789760752.849821:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 696061816753 time 696095439339 state 0x0 ref 1 00000100:00100000:3.0:1789760752.849832:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554545408:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760752.849882:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57061376, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0x1b370 00010000:00010000:3.0:1789760752.849892:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2723 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.849917:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=696s v=(0 0) 00010000:00010000:3.0:1789760752.849921:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760752.849928:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2723 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57061376->57065471] (req 57061376->57065471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760752.849942:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2723 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57061376->57065471] (req 57061376->57065471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760752.856318:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981240 is committed 00000100:00100000:3.0:1789760752.904314:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554545792 00000100:00100000:3.0:1789760752.904321:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554545792 00000100:00100000:3.0:1789760752.904367:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 460 00000100:00100000:3.0:1789760752.904414:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc0c825850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9765eb0b-55b9-40b6-bfdf-ffee156c3226+13:2413:x1876699554545792:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:3.0:1789760752.904485:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc0c825850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9765eb0b-55b9-40b6-bfdf-ffee156c3226+13:2413:x1876699554545792:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 107us (330us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760752.904493:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 460 00000100:00100000:3.0:1789760752.904546:0:6521:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 825 00000100:00100000:3.0:1789760752.904555:0:6521:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554546048:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:3.0:1789760752.904590:0:6521:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554546048:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 35us (381us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760752.904606:0:6521:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 825 00000100:00080000:3.0:1789760752.908838:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789760752.908847:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789760752.935667:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760752.946753:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760752.951820:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981246 is committed 00000100:00100000:3.0:1789760752.972321:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760752.987254:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760753.013080:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00080000:3.0:1789760753.016528:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760753.028740:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760753.051925:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981253 is committed 00000100:00100000:3.0:1789760753.058082:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760753.066233:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981254 is committed 00000001:00080000:3.0:1789760753.083616:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981255 is committed 00000100:00100000:3.0:1789760753.096555:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554548096 00000100:00100000:3.0:1789760753.096564:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554548096 00000100:00100000:3.0:1789760753.096619:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 696306038780 time 696320424890 state 0x0 ref 1 00000100:00100000:3.0:1789760753.096635:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 696320685306 time 696320424890 state 0x1 ref 1 00000100:00100000:3.0:1789760753.096667:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 696320685306 time 696342285332 state 0x0 ref 1 00000100:00100000:3.0:1789760753.096679:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554548096:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760753.096723:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57126912, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0x1b3f0 00010000:00010000:3.0:1789760753.096732:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c2793 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.096748:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=696s v=(0 0) 00010000:00010000:3.0:1789760753.096752:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.096759:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c2793 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57126912->57131007] (req 57126912->57131007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.096771:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c2793 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57126912->57131007] (req 57126912->57131007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760753.107304:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981256 is committed 00000100:00100000:3.0:1789760753.141286:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760753.178057:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760753.226226:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760753.237130:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981263 is committed 00000001:00080000:3.0:1789760753.256244:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981264 is committed 00000001:00080000:3.0:1789760753.277922:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981265 is committed 00000001:00080000:3.0:1789760753.292610:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981266 is committed 00000001:00080000:3.0:1789760753.310963:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981267 is committed 00000100:00100000:3.0:1789760753.322123:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554549632 00000100:00100000:3.0:1789760753.322133:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554549632 00000100:00100000:3.0:1789760753.322199:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 696532434721 time 696549255211 state 0x0 ref 1 00000100:00100000:3.0:1789760753.322213:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 696549515627 time 696549255211 state 0x1 ref 1 00000100:00100000:3.0:1789760753.322255:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 696549515627 time 696567873665 state 0x0 ref 1 00000100:00100000:3.0:1789760753.322268:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554549632:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760753.322315:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57176064, mtime: 0x6aad94f0, atime: 0x0, ctime: 0x6aad94f0, blocks: 0x1b450 00010000:00010000:3.0:1789760753.322324:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c27e7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.322341:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=696s v=(0 0) 00010000:00010000:3.0:1789760753.322345:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.322352:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c27e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57176064->57180159] (req 57176064->57180159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.322366:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c27e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57176064->57180159] (req 57176064->57180159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760753.333942:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57176064 -> 57180160 00002000:00010000:3.0:1789760753.333948:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 111696 -> 111704 00010000:00010000:3.0:1789760753.333962:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c27e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57176064->57180159] (req 57176064->57180159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.333974:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c27e7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57176064->57180159] (req 57176064->57180159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.333989:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c27e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57176064->57180159] (req 57176064->57180159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.333998:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760753.334002:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c27e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57176064->57180159] (req 57176064->57180159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.334011:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c27e7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57176064->57180159] (req 57176064->57180159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.334029:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c27e7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57176064->57180159] (req 57176064->57180159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.334036:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c27e7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57176064->57180159] (req 57176064->57180159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.334255:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554549632:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11982us (12275us total) trans 4294981268 rc 0/0 00000100:00100000:3.0:1789760753.334266:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13970 00000100:00100000:3.0:1789760753.334275:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 696549515627 time 696567873665 state 0x2 ref 0 00000100:00100000:3.0:1789760753.347180:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760753.367163:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760753.377192:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981270 is committed 00000001:00080000:3.0:1789760753.396252:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981271 is committed 00000100:00100000:3.0:1789760753.401184:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760753.407370:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981272 is committed 00000100:00100000:3.0:1789760753.414237:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760753.441915:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981274 is committed 00000100:00100000:3.0:1789760753.456185:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760753.478962:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760753.505160:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981277 is committed 00000100:00100000:3.0:1789760753.515778:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760753.533815:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760753.556510:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760753.570933:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760753.598529:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760753.614360:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760753.634815:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760753.642196:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981284 is committed 00000100:00100000:3.0:1789760753.656104:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760753.666183:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760753.686031:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760753.703437:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760753.750153:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760753.773722:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760753.802351:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554552704 00000100:00100000:3.0:1789760753.802479:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554552704 00000100:00100000:3.0:1789760753.802560:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 696996551978 time 697019740744 state 0x0 ref 1 00000100:00100000:3.0:1789760753.802576:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 697020001160 time 697019740744 state 0x1 ref 1 00000100:00100000:3.0:1789760753.802673:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 697020001160 time 697048291719 state 0x0 ref 1 00000100:00100000:3.0:1789760753.802690:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554552704:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760753.802735:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57274368, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0x1b510 00010000:00010000:3.0:1789760753.802744:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c288f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.802761:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=697s v=(0 0) 00010000:00010000:3.0:1789760753.802766:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760753.802773:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c288f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57274368->57278463] (req 57274368->57278463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.802786:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c288f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57274368->57278463] (req 57274368->57278463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760753.814955:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57274368 -> 57278464 00002000:00010000:3.0:1789760753.814961:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 111888 -> 111896 00010000:00010000:3.0:1789760753.814976:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c288f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57274368->57278463] (req 57274368->57278463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.814989:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c288f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57274368->57278463] (req 57274368->57278463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.814999:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c288f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57274368->57278463] (req 57274368->57278463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.815006:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760753.815012:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c288f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57274368->57278463] (req 57274368->57278463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.815021:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c288f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57274368->57278463] (req 57274368->57278463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.815041:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c288f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57274368->57278463] (req 57274368->57278463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.815049:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c288f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57274368->57278463] (req 57274368->57278463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.815187:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554552704:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12495us (16868us total) trans 4294981292 rc 0/0 00000100:00100000:3.0:1789760753.815197:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13994 00000100:00100000:3.0:1789760753.815207:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 697020001160 time 697048291719 state 0x2 ref 0 00000100:00100000:3.0:1789760753.827426:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:3.0:1789760753.876631:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57286656 -> 57290752 00002000:00010000:3.0:1789760753.876636:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 111912 -> 111920 00010000:00010000:3.0:1789760753.876656:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c28a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57286656->57290751] (req 57286656->57290751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.876670:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c28a4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57286656->57290751] (req 57286656->57290751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.876685:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c28a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57286656->57290751] (req 57286656->57290751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.876693:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760753.876698:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c28a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57286656->57290751] (req 57286656->57290751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.876707:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c28a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57286656->57290751] (req 57286656->57290751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.876732:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c28a4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57286656->57290751] (req 57286656->57290751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760753.876740:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c28a4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57286656->57290751] (req 57286656->57290751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760753.876829:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554553088:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11564us (12052us total) trans 4294981295 rc 0/0 00000100:00100000:3.0:1789760753.876839:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 13997 00000100:00100000:3.0:1789760753.876848:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 697091970252 time 697110869389 state 0x2 ref 0 00000100:00100000:3.0:1789760753.888038:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760753.943940:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760753.994483:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760754.046741:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760754.065881:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573905024 00000100:00100000:3.0:1789760754.065888:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573905024 00000100:00100000:3.0:1789760754.065961:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 340 00000100:00100000:3.0:1789760754.065971:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3646:x1876699573905024:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1789760754.065991:0:8440:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484945600 pending 28672 free 3866247168 avail 3634294784 00000100:00100000:3.0:1789760754.066034:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3646:x1876699573905024:12345-0@lo:13:osp-pre-0-1.0 Request processed in 66us (362us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760754.066041:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 340 00000100:00100000:3.0:1789760754.066091:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb044b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573905152:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760754.066104:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cc07bcca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573905024:0@lo:13:osp-pre-0-1.0 00002000:00010000:3.0:1789760754.082144:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57335808 -> 57339904 00002000:00010000:3.0:1789760754.082148:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 112008 -> 112016 00010000:00010000:3.0:1789760754.082160:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c28f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57335808->57339903] (req 57335808->57339903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.082172:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c28f8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57335808->57339903] (req 57335808->57339903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.082181:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c28f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57335808->57339903] (req 57335808->57339903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.082188:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760754.082192:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c28f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57335808->57339903] (req 57335808->57339903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.082200:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c28f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57335808->57339903] (req 57335808->57339903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.082219:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c28f8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57335808->57339903] (req 57335808->57339903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.082228:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c28f8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57335808->57339903] (req 57335808->57339903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.082369:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554554624:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6661us (7134us total) trans 4294981307 rc 0/0 00000100:00100000:3.0:1789760754.082403:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14009 00000100:00100000:3.0:1789760754.082413:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 697309479875 time 697321310261 state 0x2 ref 0 00000100:00100000:3.0:1789760754.087816:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760754.110178:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554555008 00000100:00100000:3.0:1789760754.110187:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554555008 00000100:00100000:3.0:1789760754.110236:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 697333910538 time 697346245340 state 0x0 ref 1 00000100:00100000:3.0:1789760754.110251:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 697346505756 time 697346245340 state 0x1 ref 1 00000100:00100000:3.0:1789760754.110287:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 697346505756 time 697355905717 state 0x0 ref 1 00000100:00100000:3.0:1789760754.110298:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554555008:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760754.110350:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57348096, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0x1b5a0 00010000:00010000:3.0:1789760754.110359:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c290d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.110410:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=697s v=(0 0) 00010000:00010000:3.0:1789760754.110416:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.110426:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c290d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57348096->57352191] (req 57348096->57352191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.110442:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c290d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57348096->57352191] (req 57348096->57352191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760754.122346:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57348096 -> 57352192 00002000:00010000:3.0:1789760754.122354:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 112032 -> 112040 00010000:00010000:3.0:1789760754.122367:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c290d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57348096->57352191] (req 57348096->57352191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.122563:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c290d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57348096->57352191] (req 57348096->57352191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.122573:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c290d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57348096->57352191] (req 57348096->57352191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.122581:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760754.122586:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c290d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57348096->57352191] (req 57348096->57352191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.122594:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c290d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57348096->57352191] (req 57348096->57352191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.122612:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c290d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57348096->57352191] (req 57348096->57352191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.122620:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c290d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57348096->57352191] (req 57348096->57352191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.122796:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554555008:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12494us (12697us total) trans 4294981310 rc 0/0 00000100:00100000:3.0:1789760754.122806:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14012 00000100:00100000:3.0:1789760754.122816:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 697346505756 time 697355905717 state 0x2 ref 0 00000100:00100000:3.0:1789760754.126330:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760754.158570:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760754.178778:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760754.199511:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554555648 00000100:00100000:3.0:1789760754.199521:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554555648 00000100:00100000:3.0:1789760754.199574:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 697405406005 time 697427683152 state 0x0 ref 1 00000100:00100000:3.0:1789760754.199590:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 697427943568 time 697427683152 state 0x1 ref 1 00000100:00100000:3.0:1789760754.199732:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 697427943568 time 697445349143 state 0x0 ref 1 00000100:00100000:3.0:1789760754.199746:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554555648:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760754.199793:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57368576, mtime: 0x6aad94f1, atime: 0x0, ctime: 0x6aad94f1, blocks: 0x1b5c8 00010000:00010000:3.0:1789760754.199803:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2930 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.199820:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=697s v=(0 0) 00010000:00010000:3.0:1789760754.199824:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.199831:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2930 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57368576->57372671] (req 57368576->57372671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.199850:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2930 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57368576->57372671] (req 57368576->57372671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760754.205828:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57368576 -> 57372672 00002000:00010000:3.0:1789760754.205837:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 112072 -> 112080 00010000:00010000:3.0:1789760754.205851:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2930 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57368576->57372671] (req 57368576->57372671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.205863:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2930 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57368576->57372671] (req 57368576->57372671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.205872:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2930 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57368576->57372671] (req 57368576->57372671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.205986:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760754.205991:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2930 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57368576->57372671] (req 57368576->57372671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.206000:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2930 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57368576->57372671] (req 57368576->57372671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.206021:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2930 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57368576->57372671] (req 57368576->57372671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.206028:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2930 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57368576->57372671] (req 57368576->57372671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.206126:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554555648:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6377us (6930us total) trans 4294981315 rc 0/0 00000100:00100000:3.0:1789760754.206137:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14017 00000100:00100000:3.0:1789760754.206147:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 697427943568 time 697445349143 state 0x2 ref 0 00000100:00100000:3.0:1789760754.212201:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760754.260471:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554556032 00000100:00100000:3.0:1789760754.260488:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554556032 00000100:00100000:3.0:1789760754.278078:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760754.287563:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981319 is committed 00000001:00080000:3.0:1789760754.298066:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981320 is committed 00000001:00080000:3.0:1789760754.313151:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981321 is committed 00000100:00100000:3.0:1789760754.326970:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760754.336813:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981322 is committed 00000001:00080000:3.0:1789760754.351825:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981323 is committed 00000100:00100000:3.0:1789760754.359174:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760754.366501:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981324 is committed 00000001:00080000:3.0:1789760754.393151:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981326 is committed 00000001:00080000:3.0:1789760754.404280:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981327 is committed 00000001:00080000:3.0:1789760754.462473:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981331 is committed 00000100:00100000:3.0:1789760754.467341:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760754.488825:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981333 is committed 00000001:00080000:3.0:1789760754.505748:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981334 is committed 00000100:00100000:3.0:1789760754.511085:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760754.521029:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981335 is committed 00000001:00080000:3.0:1789760754.539559:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981336 is committed 00000001:00080000:3.0:1789760754.552438:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981337 is committed 00000100:00100000:3.0:1789760754.575485:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760754.583036:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981338 is committed 00000001:00080000:3.0:1789760754.606946:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981339 is committed 00000100:00100000:3.0:1789760754.631225:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760754.636945:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981341 is committed 00000001:00080000:3.0:1789760754.647319:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981342 is committed 00000001:00080000:3.0:1789760754.655070:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981343 is committed 00000100:00100000:3.0:1789760754.660029:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760754.695420:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554559744 00000100:00100000:3.0:1789760754.695430:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554559744 00000100:00100000:3.0:1789760754.695480:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 697916429851 time 697927076282 state 0x0 ref 1 00000100:00100000:3.0:1789760754.695495:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 697927336698 time 697927076282 state 0x1 ref 1 00000100:00100000:3.0:1789760754.695524:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 697927336698 time 697941143327 state 0x0 ref 1 00000100:00100000:3.0:1789760754.695536:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06401500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554559744:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760754.695584:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57499648, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x1b6c8 00010000:00010000:3.0:1789760754.695594:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a10 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.695609:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=697s v=(0 0) 00010000:00010000:3.0:1789760754.695613:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.695620:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57499648->57503743] (req 57499648->57503743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.695633:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57499648->57503743] (req 57499648->57503743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760754.702349:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57499648 -> 57503744 00002000:00010000:3.0:1789760754.702354:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 112328 -> 112336 00010000:00010000:3.0:1789760754.702367:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57499648->57503743] (req 57499648->57503743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.702406:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a10 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57499648->57503743] (req 57499648->57503743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.702416:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57499648->57503743] (req 57499648->57503743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.702424:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760754.702429:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57499648->57503743] (req 57499648->57503743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.702438:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a10 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57499648->57503743] (req 57499648->57503743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.702456:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a10 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57499648->57503743] (req 57499648->57503743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.702463:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a10 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57499648->57503743] (req 57499648->57503743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.702539:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06401500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554559744:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7000us (7204us total) trans 4294981347 rc 0/0 00000100:00100000:3.0:1789760754.702551:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14049 00000100:00100000:3.0:1789760754.702560:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 697927336698 time 697941143327 state 0x2 ref 0 00000100:00100000:3.0:1789760754.750905:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554560256 00000100:00100000:3.0:1789760754.750916:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554560256 00000100:00100000:3.0:1789760754.750967:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 697970724376 time 697983229182 state 0x0 ref 1 00000100:00100000:3.0:1789760754.750983:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 697983489598 time 697983229182 state 0x1 ref 1 00000100:00100000:3.0:1789760754.751023:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 697983489598 time 697996632625 state 0x0 ref 1 00000100:00100000:3.0:1789760754.751036:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f15f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554560256:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760754.751080:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57516032, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x1b6e8 00010000:00010000:3.0:1789760754.751089:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a2c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.751105:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=697s v=(0 0) 00010000:00010000:3.0:1789760754.751110:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.751116:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57516032->57520127] (req 57516032->57520127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.751129:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57516032->57520127] (req 57516032->57520127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760754.757388:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57516032 -> 57520128 00002000:00010000:3.0:1789760754.757394:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 112360 -> 112368 00010000:00010000:3.0:1789760754.757475:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57516032->57520127] (req 57516032->57520127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.757487:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a2c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57516032->57520127] (req 57516032->57520127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.757496:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57516032->57520127] (req 57516032->57520127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.757503:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760754.757509:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57516032->57520127] (req 57516032->57520127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.757518:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a2c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57516032->57520127] (req 57516032->57520127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.757539:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a2c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57516032->57520127] (req 57516032->57520127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.757546:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a2c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57516032->57520127] (req 57516032->57520127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.757639:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f15f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554560256:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6601us (7196us total) trans 4294981351 rc 0/0 00000100:00100000:3.0:1789760754.757649:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14053 00000100:00100000:3.0:1789760754.757658:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 697983489598 time 697996632625 state 0x2 ref 0 00000100:00100000:3.0:1789760754.795093:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554560768 00000100:00100000:3.0:1789760754.795103:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554560768 00000100:00100000:3.0:1789760754.795160:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698021721297 time 698030557814 state 0x0 ref 1 00000100:00100000:3.0:1789760754.795177:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698030818230 time 698030557814 state 0x1 ref 1 00000100:00100000:3.0:1789760754.795212:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698030818230 time 698040830637 state 0x0 ref 1 00000100:00100000:3.0:1789760754.795225:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554560768:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760754.795282:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57532416, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x1b708 00010000:00010000:3.0:1789760754.795291:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a48 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.795307:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760754.795311:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.795317:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57532416->57536511] (req 57532416->57536511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.795332:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57532416->57536511] (req 57532416->57536511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760754.800294:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57532416 -> 57536512 00002000:00010000:3.0:1789760754.800299:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 112392 -> 112400 00010000:00010000:3.0:1789760754.800313:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57532416->57536511] (req 57532416->57536511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.800329:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a48 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57532416->57536511] (req 57532416->57536511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.800337:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57532416->57536511] (req 57532416->57536511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.800345:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760754.800350:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57532416->57536511] (req 57532416->57536511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.800358:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a48 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57532416->57536511] (req 57532416->57536511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.800395:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a48 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57532416->57536511] (req 57532416->57536511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.800403:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a48 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57532416->57536511] (req 57532416->57536511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.800499:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554560768:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5272us (5879us total) trans 4294981355 rc 0/0 00000100:00100000:3.0:1789760754.800510:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14057 00000100:00100000:3.0:1789760754.800519:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698030818230 time 698040830637 state 0x2 ref 0 00000001:00080000:3.0:1789760754.833049:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981357 is committed 00000001:00080000:3.0:1789760754.847111:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981358 is committed 00000100:00100000:3.0:1789760754.853981:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554561280 00000100:00100000:3.0:1789760754.853990:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554561280 00000100:00100000:3.0:1789760754.854044:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698070810370 time 698086090081 state 0x0 ref 1 00000100:00100000:3.0:1789760754.854058:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698086350497 time 698086090081 state 0x1 ref 1 00000100:00100000:3.0:1789760754.854092:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698086350497 time 698099710618 state 0x0 ref 1 00000100:00100000:3.0:1789760754.854104:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554561280:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760754.854146:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57548800, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x1b728 00010000:00010000:3.0:1789760754.854155:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a64 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.854170:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760754.854174:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.854180:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57548800->57552895] (req 57548800->57552895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.854192:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57548800->57552895] (req 57548800->57552895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760754.859422:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57548800 -> 57552896 00002000:00010000:3.0:1789760754.859427:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 112424 -> 112432 00010000:00010000:3.0:1789760754.859440:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57548800->57552895] (req 57548800->57552895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.859459:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a64 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57548800->57552895] (req 57548800->57552895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.859467:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57548800->57552895] (req 57548800->57552895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.859474:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760754.859478:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57548800->57552895] (req 57548800->57552895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.859487:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a64 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57548800->57552895] (req 57548800->57552895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.859505:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a64 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57548800->57552895] (req 57548800->57552895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.859513:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a64 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57548800->57552895] (req 57548800->57552895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.859606:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554561280:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5499us (6094us total) trans 4294981359 rc 0/0 00000100:00100000:3.0:1789760754.859617:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14061 00000100:00100000:3.0:1789760754.859635:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698086350497 time 698099710618 state 0x2 ref 0 00000100:00100000:3.0:1789760754.863138:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760754.892640:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760754.902805:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554561792 00000100:00100000:3.0:1789760754.902815:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554561792 00000100:00100000:3.0:1789760754.902869:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698127721868 time 698138437114 state 0x0 ref 1 00000100:00100000:3.0:1789760754.902893:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698138697530 time 698138437114 state 0x1 ref 1 00000100:00100000:3.0:1789760754.903033:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698138697530 time 698148650669 state 0x0 ref 1 00000100:00100000:3.0:1789760754.903045:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554561792:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760754.903091:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57565184, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x1b748 00010000:00010000:3.0:1789760754.903100:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a80 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.903117:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760754.903121:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.903129:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57565184->57569279] (req 57565184->57569279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.903143:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57565184->57569279] (req 57565184->57569279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760754.910080:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57565184 -> 57569280 00002000:00010000:3.0:1789760754.910085:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 112456 -> 112464 00010000:00010000:3.0:1789760754.910098:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57565184->57569279] (req 57565184->57569279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.910118:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a80 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57565184->57569279] (req 57565184->57569279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.910127:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a80 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57565184->57569279] (req 57565184->57569279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.910135:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760754.910140:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a80 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57565184->57569279] (req 57565184->57569279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.910148:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a80 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57565184->57569279] (req 57565184->57569279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.910178:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a80 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57565184->57569279] (req 57565184->57569279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.910187:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a80 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57565184->57569279] (req 57565184->57569279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760754.910280:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554561792:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7233us (7559us total) trans 4294981363 rc 0/0 00000100:00100000:3.0:1789760754.910291:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14065 00000100:00100000:3.0:1789760754.910300:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698138697530 time 698148650669 state 0x2 ref 0 00000001:00080000:3.0:1789760754.952027:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981366 is committed 00000100:00100000:3.0:1789760754.963441:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554562304 00000100:00100000:3.0:1789760754.963451:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554562304 00000100:00100000:3.0:1789760754.963506:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698177214581 time 698191792354 state 0x0 ref 1 00000100:00100000:3.0:1789760754.963521:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698192052770 time 698191792354 state 0x1 ref 1 00000100:00100000:3.0:1789760754.963552:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698192052770 time 698209170909 state 0x0 ref 1 00000100:00100000:3.0:1789760754.963565:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554562304:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760754.963611:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57581568, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x1b768 00010000:00010000:3.0:1789760754.963621:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a9c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.963636:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760754.963641:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760754.963653:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57581568->57585663] (req 57581568->57585663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760754.963668:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2a9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57581568->57585663] (req 57581568->57585663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760754.971053:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981367 is committed 00000001:00080000:3.0:1789760755.012098:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981369 is committed 00000001:00080000:3.0:1789760755.035199:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981370 is committed 00000100:00100000:3.0:1789760755.058349:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554562944 00000100:00100000:3.0:1789760755.058358:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554562944 00000100:00100000:3.0:1789760755.058567:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698273818302 time 698287345325 state 0x0 ref 1 00000100:00100000:3.0:1789760755.058585:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698287605741 time 698287345325 state 0x1 ref 1 00000100:00100000:3.0:1789760755.058657:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698287605741 time 698304274336 state 0x0 ref 1 00000100:00100000:3.0:1789760755.058671:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554562944:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760755.058717:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57602048, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x1b790 00010000:00010000:3.0:1789760755.058726:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c2abf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.058742:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760755.058746:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.058752:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c2abf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57602048->57606143] (req 57602048->57606143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.058767:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c2abf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57602048->57606143] (req 57602048->57606143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.080049:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554563072 00000100:00100000:3.0:1789760755.080059:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554563072 00000100:00100000:3.0:1789760755.080112:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698287605741 time 698304274336 state 0x0 ref 1 00000100:00100000:3.0:1789760755.080127:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698304534752 time 698304274336 state 0x1 ref 1 00000100:00100000:3.0:1789760755.080161:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698304534752 time 698325778756 state 0x0 ref 1 00000100:00100000:3.0:1789760755.080172:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554563072:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760755.080221:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57606144, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x1b798 00010000:00010000:3.0:1789760755.080229:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2ac6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.080245:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760755.080249:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.080255:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2ac6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57606144->57610239] (req 57606144->57610239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.080274:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2ac6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57606144->57610239] (req 57606144->57610239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.085167:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57606144 -> 57610240 00002000:00010000:3.0:1789760755.085172:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 112536 -> 112544 00010000:00010000:3.0:1789760755.085186:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2ac6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57606144->57610239] (req 57606144->57610239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.085197:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2ac6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57606144->57610239] (req 57606144->57610239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.085207:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2ac6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57606144->57610239] (req 57606144->57610239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.085214:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760755.085219:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2ac6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57606144->57610239] (req 57606144->57610239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.085233:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2ac6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57606144->57610239] (req 57606144->57610239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.085252:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2ac6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57606144->57610239] (req 57606144->57610239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.085271:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2ac6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57606144->57610239] (req 57606144->57610239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.085356:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554563072:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5181us (5380us total) trans 4294981373 rc 0/0 00000100:00100000:3.0:1789760755.085366:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14075 00000100:00100000:3.0:1789760755.085375:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698304534752 time 698325778756 state 0x2 ref 0 00000001:00080000:3.0:1789760755.098290:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981374 is committed 00000001:00080000:3.0:1789760755.119221:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981375 is committed 00000100:00100000:3.0:1789760755.140416:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554563456 00000100:00100000:3.0:1789760755.140427:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554563456 00000100:00100000:3.0:1789760755.140483:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698339774640 time 698358335244 state 0x0 ref 1 00000100:00100000:3.0:1789760755.140498:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698358595660 time 698358335244 state 0x1 ref 1 00000100:00100000:3.0:1789760755.195937:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554563712 00000100:00100000:3.0:1789760755.196013:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554563712 00000100:00100000:3.0:1789760755.196070:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698386476089 time 698411228495 state 0x0 ref 1 00000100:00100000:3.0:1789760755.196085:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698411488911 time 698411228495 state 0x1 ref 1 00000100:00100000:3.0:1789760755.196233:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698411488911 time 698441733926 state 0x0 ref 1 00000100:00100000:3.0:1789760755.196248:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554563712:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760755.196291:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57626624, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x1b7c0 00010000:00010000:3.0:1789760755.196301:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2ae9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.196316:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760755.196331:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.196338:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2ae9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57626624->57630719] (req 57626624->57630719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.196352:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2ae9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57626624->57630719] (req 57626624->57630719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.212992:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57626624 -> 57630720 00002000:00010000:3.0:1789760755.213000:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 112576 -> 112584 00010000:00010000:3.0:1789760755.213012:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2ae9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57626624->57630719] (req 57626624->57630719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.213025:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2ae9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57626624->57630719] (req 57626624->57630719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.213033:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2ae9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57626624->57630719] (req 57626624->57630719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.213045:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760755.213050:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2ae9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57626624->57630719] (req 57626624->57630719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.213058:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2ae9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57626624->57630719] (req 57626624->57630719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.213076:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2ae9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57626624->57630719] (req 57626624->57630719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.213084:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2ae9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57626624->57630719] (req 57626624->57630719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.213689:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554563712:12345-192.168.203.22@tcp:4:dd.0 Request processed in 17436us (18328us total) trans 4294981378 rc 0/0 00000100:00100000:3.0:1789760755.213701:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14080 00000100:00100000:3.0:1789760755.213710:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698411488911 time 698441733926 state 0x2 ref 0 00000100:00100000:3.0:1789760755.300314:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554564224 00000100:00100000:3.0:1789760755.300325:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554564224 00000100:00100000:3.0:1789760755.300444:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698502679884 time 698522712563 state 0x0 ref 1 00000100:00100000:3.0:1789760755.300463:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698522972979 time 698522712563 state 0x1 ref 1 00000100:00100000:3.0:1789760755.300497:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698522972979 time 698546116092 state 0x0 ref 1 00000100:00100000:3.0:1789760755.300513:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554564224:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760755.300696:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57643008, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x1b7e0 00010000:00010000:3.0:1789760755.300706:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b05 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.300722:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760755.300727:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.300733:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57643008->57647103] (req 57643008->57647103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.300746:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57643008->57647103] (req 57643008->57647103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.309026:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57643008 -> 57647104 00002000:00010000:3.0:1789760755.309031:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 112608 -> 112616 00010000:00010000:3.0:1789760755.309044:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57643008->57647103] (req 57643008->57647103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.309056:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b05 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57643008->57647103] (req 57643008->57647103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.309065:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57643008->57647103] (req 57643008->57647103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.309072:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760755.309078:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57643008->57647103] (req 57643008->57647103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.309087:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b05 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57643008->57647103] (req 57643008->57647103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.309106:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b05 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57643008->57647103] (req 57643008->57647103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.309114:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b05 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57643008->57647103] (req 57643008->57647103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.309211:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554564224:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8695us (8982us total) trans 4294981382 rc 0/0 00000100:00100000:3.0:1789760755.309222:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14084 00000100:00100000:3.0:1789760755.309231:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698522972979 time 698546116092 state 0x2 ref 0 00000001:00080000:3.0:1789760755.378194:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981385 is committed 00000100:00100000:3.0:1789760755.392404:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554564736 00000100:00100000:3.0:1789760755.392414:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554564736 00000100:00100000:3.0:1789760755.392476:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698590210478 time 698613943751 state 0x0 ref 1 00000100:00100000:3.0:1789760755.392490:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698614204167 time 698613943751 state 0x1 ref 1 00000100:00100000:3.0:1789760755.392519:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698614204167 time 698638138530 state 0x0 ref 1 00000100:00100000:3.0:1789760755.392530:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe14050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554564736:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760755.392575:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57659392, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x1b800 00010000:00010000:3.0:1789760755.392584:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b21 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.392607:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760755.392612:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.392619:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57659392->57663487] (req 57659392->57663487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.392632:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57659392->57663487] (req 57659392->57663487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.401375:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57659392 -> 57663488 00002000:00010000:3.0:1789760755.401409:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 112640 -> 112648 00010000:00010000:3.0:1789760755.401421:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57659392->57663487] (req 57659392->57663487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.401434:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b21 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57659392->57663487] (req 57659392->57663487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.401550:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57659392->57663487] (req 57659392->57663487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.401559:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760755.401564:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57659392->57663487] (req 57659392->57663487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.401580:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b21 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57659392->57663487] (req 57659392->57663487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.401608:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b21 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57659392->57663487] (req 57659392->57663487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.401617:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b21 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57659392->57663487] (req 57659392->57663487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.401706:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe14050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554564736:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9173us (9397us total) trans 4294981386 rc 0/0 00000100:00100000:3.0:1789760755.401717:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14088 00000100:00100000:3.0:1789760755.401726:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698614204167 time 698638138530 state 0x2 ref 0 00000100:00100000:3.0:1789760755.409063:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554564864 00000100:00100000:3.0:1789760755.409073:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554564864 00000100:00100000:3.0:1789760755.409127:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698614204167 time 698638138530 state 0x0 ref 1 00000100:00100000:3.0:1789760755.409142:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698638398946 time 698638138530 state 0x1 ref 1 00000100:00100000:3.0:1789760755.409216:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698638398946 time 698654834172 state 0x0 ref 1 00000100:00100000:3.0:1789760755.409229:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554564864:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760755.409274:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57663488, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x1b808 00010000:00010000:3.0:1789760755.409283:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b28 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.409299:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760755.409303:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.409309:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57663488->57667583] (req 57663488->57667583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.409322:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57663488->57667583] (req 57663488->57667583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.415942:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57663488 -> 57667584 00002000:00010000:3.0:1789760755.415947:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 112648 -> 112656 00010000:00010000:3.0:1789760755.415962:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57663488->57667583] (req 57663488->57667583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.415976:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b28 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57663488->57667583] (req 57663488->57667583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.415984:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57663488->57667583] (req 57663488->57667583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.415993:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760755.415998:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57663488->57667583] (req 57663488->57667583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.416007:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b28 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57663488->57667583] (req 57663488->57667583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.416032:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b28 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57663488->57667583] (req 57663488->57667583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.416039:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b28 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57663488->57667583] (req 57663488->57667583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.416134:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554564864:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6902us (7160us total) trans 4294981387 rc 0/0 00000100:00100000:3.0:1789760755.416144:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14089 00000100:00100000:3.0:1789760755.416153:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698638398946 time 698654834172 state 0x2 ref 0 00000001:00080000:3.0:1789760755.430008:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981388 is committed 00000100:00100000:3.0:1789760755.447178:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554565248 00000100:00100000:3.0:1789760755.447188:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554565248 00000100:00100000:3.0:1789760755.447242:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698670661099 time 698682152849 state 0x0 ref 1 00000100:00100000:3.0:1789760755.447256:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698682413265 time 698682152849 state 0x1 ref 1 00000100:00100000:3.0:1789760755.447296:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698682413265 time 698692914763 state 0x0 ref 1 00000100:00100000:3.0:1789760755.447309:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554565248:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760755.447353:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57675776, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x1b820 00010000:00010000:3.0:1789760755.447362:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b3d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.447421:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760755.447426:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.447432:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57675776->57679871] (req 57675776->57679871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.447454:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57675776->57679871] (req 57675776->57679871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.452166:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57675776 -> 57679872 00002000:00010000:3.0:1789760755.452171:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 112672 -> 112680 00010000:00010000:3.0:1789760755.452184:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57675776->57679871] (req 57675776->57679871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.452197:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b3d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57675776->57679871] (req 57675776->57679871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.452207:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57675776->57679871] (req 57675776->57679871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.452216:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760755.452221:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57675776->57679871] (req 57675776->57679871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.452230:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b3d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57675776->57679871] (req 57675776->57679871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.452313:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b3d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57675776->57679871] (req 57675776->57679871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.452323:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b3d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57675776->57679871] (req 57675776->57679871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.452638:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554565248:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5326us (5540us total) trans 4294981390 rc 0/0 00000100:00100000:3.0:1789760755.452649:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14092 00000100:00100000:3.0:1789760755.452658:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698682413265 time 698692914763 state 0x2 ref 0 00000100:00100000:3.0:1789760755.458724:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554565376 00000100:00100000:3.0:1789760755.458734:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554565376 00000100:00100000:3.0:1789760755.458789:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698682413265 time 698692914763 state 0x0 ref 1 00000100:00100000:3.0:1789760755.458805:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698693175179 time 698692914763 state 0x1 ref 1 00000100:00100000:3.0:1789760755.458835:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698693175179 time 698704453128 state 0x0 ref 1 00000100:00100000:3.0:1789760755.458847:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554565376:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760755.458900:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57679872, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x1b828 00010000:00010000:3.0:1789760755.458910:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b44 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.458926:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760755.458932:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.458938:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57679872->57683967] (req 57679872->57683967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.458952:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57679872->57683967] (req 57679872->57683967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.463367:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57679872 -> 57683968 00002000:00010000:3.0:1789760755.463372:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 112680 -> 112688 00010000:00010000:3.0:1789760755.463403:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57679872->57683967] (req 57679872->57683967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.463416:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b44 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57679872->57683967] (req 57679872->57683967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.463425:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57679872->57683967] (req 57679872->57683967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.463433:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760755.463448:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57679872->57683967] (req 57679872->57683967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.463459:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b44 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57679872->57683967] (req 57679872->57683967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.463477:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b44 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57679872->57683967] (req 57679872->57683967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.463490:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b44 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57679872->57683967] (req 57679872->57683967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.463586:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554565376:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4736us (4946us total) trans 4294981391 rc 0/0 00000100:00100000:3.0:1789760755.463596:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14093 00000100:00100000:3.0:1789760755.463605:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698693175179 time 698704453128 state 0x2 ref 0 00000001:00080000:3.0:1789760755.470974:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981392 is committed 00000001:00080000:3.0:1789760755.481602:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981393 is committed 00000100:00100000:3.0:1789760755.488336:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554565760 00000100:00100000:3.0:1789760755.488346:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554565760 00000100:00100000:3.0:1789760755.488447:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698712741253 time 698723399375 state 0x0 ref 1 00000100:00100000:3.0:1789760755.488464:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698723659791 time 698723399375 state 0x1 ref 1 00000100:00100000:3.0:1789760755.488508:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698723659791 time 698734126515 state 0x0 ref 1 00000100:00100000:3.0:1789760755.488521:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554565760:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760755.488567:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57692160, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x1b840 00010000:00010000:3.0:1789760755.488577:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b59 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.488594:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760755.488599:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.488607:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57692160->57696255] (req 57692160->57696255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.488620:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2b59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57692160->57696255] (req 57692160->57696255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.492370:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981394 is committed 00000100:00100000:3.0:1789760755.501917:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554565888 00000100:00100000:3.0:1789760755.501928:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554565888 00000100:00100000:3.0:1789760755.501978:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698723659791 time 698734126515 state 0x0 ref 1 00000100:00100000:3.0:1789760755.501993:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698734386931 time 698734126515 state 0x1 ref 1 00000100:00100000:3.0:1789760755.502025:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698734386931 time 698747643354 state 0x0 ref 1 00000100:00100000:3.0:1789760755.502037:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554565888:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760755.502080:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57696256, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x1b848 00010000:00010000:3.0:1789760755.502089:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b60 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.502106:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760755.502110:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.502116:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57696256->57700351] (req 57696256->57700351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.502131:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57696256->57700351] (req 57696256->57700351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.507253:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57696256 -> 57700352 00002000:00010000:3.0:1789760755.507311:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 112712 -> 112720 00010000:00010000:3.0:1789760755.507325:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57696256->57700351] (req 57696256->57700351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.507339:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b60 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57696256->57700351] (req 57696256->57700351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.507348:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57696256->57700351] (req 57696256->57700351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.507356:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760755.507361:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57696256->57700351] (req 57696256->57700351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.507370:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b60 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57696256->57700351] (req 57696256->57700351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.507527:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b60 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57696256->57700351] (req 57696256->57700351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.507537:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b60 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57696256->57700351] (req 57696256->57700351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.507724:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554565888:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5683us (6522us total) trans 4294981395 rc 0/0 00000100:00100000:3.0:1789760755.507736:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14097 00000100:00100000:3.0:1789760755.507746:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698734386931 time 698747643354 state 0x2 ref 0 00000100:00100000:3.0:1789760755.559312:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554566528 00000100:00100000:3.0:1789760755.559322:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554566528 00000100:00100000:3.0:1789760755.559375:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698779230675 time 698791760409 state 0x0 ref 1 00000100:00100000:3.0:1789760755.559431:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698792020825 time 698791760409 state 0x1 ref 1 00000100:00100000:3.0:1789760755.559459:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698792020825 time 698805077754 state 0x0 ref 1 00000100:00100000:3.0:1789760755.559471:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554566528:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760755.559514:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57716736, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x1b870 00010000:00010000:3.0:1789760755.559523:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b83 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.559538:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760755.559543:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.559549:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57716736->57720831] (req 57716736->57720831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.559563:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57716736->57720831] (req 57716736->57720831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.563612:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57716736 -> 57720832 00002000:00010000:3.0:1789760755.563618:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 112752 -> 112760 00010000:00010000:3.0:1789760755.563630:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57716736->57720831] (req 57716736->57720831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.563653:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b83 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57716736->57720831] (req 57716736->57720831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.563661:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57716736->57720831] (req 57716736->57720831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.563668:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760755.563673:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57716736->57720831] (req 57716736->57720831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.563682:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b83 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57716736->57720831] (req 57716736->57720831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.563702:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b83 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57716736->57720831] (req 57716736->57720831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.563709:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b83 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57716736->57720831] (req 57716736->57720831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.563802:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554566528:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4328us (4661us total) trans 4294981400 rc 0/0 00000100:00100000:3.0:1789760755.563811:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14102 00000100:00100000:3.0:1789760755.563819:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698792020825 time 698805077754 state 0x2 ref 0 00000100:00100000:3.0:1789760755.616980:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554567040 00000100:00100000:3.0:1789760755.616992:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554567040 00000100:00100000:3.0:1789760755.617319:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698835407470 time 698851818530 state 0x0 ref 1 00000100:00100000:3.0:1789760755.617337:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698852078946 time 698851818530 state 0x1 ref 1 00000100:00100000:3.0:1789760755.617365:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698852078946 time 698862983551 state 0x0 ref 1 00000100:00100000:3.0:1789760755.617556:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06449f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554567040:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760755.617603:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57733120, mtime: 0x6aad94f2, atime: 0x0, ctime: 0x6aad94f2, blocks: 0x1b890 00010000:00010000:3.0:1789760755.617612:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b9f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.617627:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760755.617631:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.617707:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57733120->57737215] (req 57733120->57737215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.617722:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57733120->57737215] (req 57733120->57737215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.621970:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57733120 -> 57737216 00002000:00010000:3.0:1789760755.621975:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 112784 -> 112792 00010000:00010000:3.0:1789760755.621988:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57733120->57737215] (req 57733120->57737215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.622006:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b9f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57733120->57737215] (req 57733120->57737215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.622015:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b9f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57733120->57737215] (req 57733120->57737215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.622022:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760755.622026:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b9f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57733120->57737215] (req 57733120->57737215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.622035:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b9f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57733120->57737215] (req 57733120->57737215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.622053:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b9f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57733120->57737215] (req 57733120->57737215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.622060:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2b9f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57733120->57737215] (req 57733120->57737215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.622198:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06449f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554567040:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4818us (5404us total) trans 4294981404 rc 0/0 00000100:00100000:3.0:1789760755.622208:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14106 00000100:00100000:3.0:1789760755.622217:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698852078946 time 698862983551 state 0x2 ref 0 00000100:00100000:3.0:1789760755.643249:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698876067762 time 698888864647 state 0x0 ref 1 00000100:00100000:3.0:1789760755.643283:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0644b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554567296:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760755.643330:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57741312, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x1b8a0 00010000:00010000:3.0:1789760755.643340:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2bad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.643355:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760755.643360:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.643367:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2bad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57741312->57745407] (req 57741312->57745407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.643587:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2bad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57741312->57745407] (req 57741312->57745407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.648582:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57741312 -> 57745408 00002000:00010000:3.0:1789760755.648587:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 112800 -> 112808 00010000:00010000:3.0:1789760755.648599:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2bad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57741312->57745407] (req 57741312->57745407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.648611:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2bad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57741312->57745407] (req 57741312->57745407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.648620:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2bad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57741312->57745407] (req 57741312->57745407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.648628:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760755.648634:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2bad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57741312->57745407] (req 57741312->57745407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.648642:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2bad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57741312->57745407] (req 57741312->57745407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.648660:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2bad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57741312->57745407] (req 57741312->57745407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.648668:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2bad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57741312->57745407] (req 57741312->57745407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.648761:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0644b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554567296:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5484us (7654us total) trans 4294981406 rc 0/0 00000100:00100000:3.0:1789760755.648772:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14108 00000100:00100000:3.0:1789760755.648781:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698876067762 time 698888864647 state 0x2 ref 0 00000100:00100000:3.0:1789760755.721141:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554567808 00000100:00100000:3.0:1789760755.721150:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554567808 00000100:00100000:3.0:1789760755.721203:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698915040459 time 698932796041 state 0x0 ref 1 00000100:00100000:3.0:1789760755.721217:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698933056457 time 698932796041 state 0x1 ref 1 00000100:00100000:3.0:1789760755.721254:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 698933056457 time 698966873113 state 0x0 ref 1 00000100:00100000:3.0:1789760755.721265:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb065a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554567808:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760755.721307:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57757696, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x1b8c0 00010000:00010000:3.0:1789760755.721317:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2bc9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.721331:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=698s v=(0 0) 00010000:00010000:3.0:1789760755.721334:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.721341:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2bc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57757696->57761791] (req 57757696->57761791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.721354:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2bc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57757696->57761791] (req 57757696->57761791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760755.851076:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57782272 -> 57786368 00002000:00010000:3.0:1789760755.851085:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 112880 -> 112888 00010000:00010000:3.0:1789760755.851098:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c2bf3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57782272->57786367] (req 57782272->57786367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.851179:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c2bf3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57782272->57786367] (req 57782272->57786367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.851189:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c2bf3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57782272->57786367] (req 57782272->57786367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.851197:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760755.851203:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c2bf3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57782272->57786367] (req 57782272->57786367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.851213:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c2bf3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57782272->57786367] (req 57782272->57786367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.851234:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c2bf3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57782272->57786367] (req 57782272->57786367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.851242:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c2bf3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57782272->57786367] (req 57782272->57786367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760755.851477:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554568576:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11627us (11860us total) trans 4294981416 rc 0/0 00000100:00100000:3.0:1789760755.851488:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14118 00000100:00100000:3.0:1789760755.851499:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699055662185 time 699085452824 state 0x2 ref 0 00000100:00100000:3.0:1789760755.857816:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760755.914735:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554568960:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9384us (11149us total) trans 4294981419 rc 0/0 00000100:00100000:3.0:1789760755.914747:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14121 00000100:00100000:3.0:1789760755.914757:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699127168422 time 699150951474 state 0x2 ref 0 00000100:00100000:3.0:1789760755.923153:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760755.952982:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981421 is committed 00000100:00100000:3.0:1789760755.959768:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760755.979191:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554569472 00000100:00100000:3.0:1789760755.979202:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554569472 00000100:00100000:3.0:1789760755.979265:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699181534458 time 699205586904 state 0x0 ref 1 00000100:00100000:3.0:1789760755.979281:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699205847320 time 699205586904 state 0x1 ref 1 00000100:00100000:3.0:1789760755.979311:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699205847320 time 699224928967 state 0x0 ref 1 00000100:00100000:3.0:1789760755.979323:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554569472:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760755.979368:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57810944, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x1b928 00010000:00010000:3.0:1789760755.979595:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2c24 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.979613:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760755.979617:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760755.979625:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2c24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57810944->57815039] (req 57810944->57815039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760755.979639:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c2c24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57810944->57815039] (req 57810944->57815039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760755.989693:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981423 is committed 00000001:00080000:3.0:1789760756.004594:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981424 is committed 00000100:00100000:3.0:1789760756.009178:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760756.016865:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981425 is committed 00000100:00100000:3.0:1789760756.046200:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554569984 00000100:00100000:3.0:1789760756.046210:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554569984 00000100:00100000:3.0:1789760756.046272:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699256417651 time 699266174807 state 0x0 ref 1 00000100:00100000:3.0:1789760756.046289:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699266435223 time 699266174807 state 0x1 ref 1 00000100:00100000:3.0:1789760756.046315:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699266435223 time 699291934361 state 0x0 ref 1 00000100:00100000:3.0:1789760756.046327:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554569984:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760756.046368:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57827328, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x1b948 00010000:00010000:3.0:1789760756.046400:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c40 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.046417:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760756.046422:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.046429:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57827328->57831423] (req 57827328->57831423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.046442:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57827328->57831423] (req 57827328->57831423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.052156:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57827328 -> 57831424 00002000:00010000:3.0:1789760756.052161:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 112968 -> 112976 00010000:00010000:3.0:1789760756.052174:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57827328->57831423] (req 57827328->57831423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.052192:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c40 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57827328->57831423] (req 57827328->57831423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.052200:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57827328->57831423] (req 57827328->57831423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.052207:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760756.052213:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57827328->57831423] (req 57827328->57831423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.052225:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c40 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57827328->57831423] (req 57827328->57831423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.052243:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c40 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57827328->57831423] (req 57827328->57831423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.052261:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c40 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57827328->57831423] (req 57827328->57831423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.052417:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554569984:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6086us (6726us total) trans 4294981427 rc 0/0 00000100:00100000:3.0:1789760756.052428:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14129 00000100:00100000:3.0:1789760756.053943:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699266435223 time 699291934361 state 0x2 ref 0 00000100:00100000:3.0:1789760756.105659:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554570496 00000100:00100000:3.0:1789760756.105668:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554570496 00000100:00100000:3.0:1789760756.105717:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699324709368 time 699339216064 state 0x0 ref 1 00000100:00100000:3.0:1789760756.105731:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699339476480 time 699339216064 state 0x1 ref 1 00000100:00100000:3.0:1789760756.105759:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699339476480 time 699351377512 state 0x0 ref 1 00000100:00100000:3.0:1789760756.105770:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554570496:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760756.105816:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57843712, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x1b968 00010000:00010000:3.0:1789760756.105826:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c5c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.105839:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760756.105843:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.105849:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57843712->57847807] (req 57843712->57847807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.105862:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57843712->57847807] (req 57843712->57847807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.115372:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57843712 -> 57847808 00002000:00010000:3.0:1789760756.115404:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 113000 -> 113008 00010000:00010000:3.0:1789760756.115418:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57843712->57847807] (req 57843712->57847807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.115431:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c5c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57843712->57847807] (req 57843712->57847807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.115440:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57843712->57847807] (req 57843712->57847807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.115448:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760756.115453:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57843712->57847807] (req 57843712->57847807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.115462:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c5c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57843712->57847807] (req 57843712->57847807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.115482:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c5c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57843712->57847807] (req 57843712->57847807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.115490:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c5c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57843712->57847807] (req 57843712->57847807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.115642:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554570496:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9861us (10064us total) trans 4294981431 rc 0/0 00000100:00100000:3.0:1789760756.115653:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14133 00000100:00100000:3.0:1789760756.115664:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699339476480 time 699351377512 state 0x2 ref 0 00000100:00100000:3.0:1789760756.152728:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760756.170611:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554571008 00000100:00100000:3.0:1789760756.170622:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554571008 00000100:00100000:3.0:1789760756.170687:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699384968034 time 699398541185 state 0x0 ref 1 00000100:00100000:3.0:1789760756.170703:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699398801601 time 699398541185 state 0x1 ref 1 00000100:00100000:3.0:1789760756.170765:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699398801601 time 699416382740 state 0x0 ref 1 00000100:00100000:3.0:1789760756.170779:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554571008:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760756.170823:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57860096, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x1b988 00010000:00010000:3.0:1789760756.170831:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c78 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.170851:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760756.170855:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.170867:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57860096->57864191] (req 57860096->57864191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.170889:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57860096->57864191] (req 57860096->57864191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.177561:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57860096 -> 57864192 00002000:00010000:3.0:1789760756.177567:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 113032 -> 113040 00010000:00010000:3.0:1789760756.177579:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57860096->57864191] (req 57860096->57864191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.177592:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c78 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57860096->57864191] (req 57860096->57864191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.177601:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57860096->57864191] (req 57860096->57864191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.177608:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760756.177613:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57860096->57864191] (req 57860096->57864191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.177621:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c78 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57860096->57864191] (req 57860096->57864191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.177724:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c78 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57860096->57864191] (req 57860096->57864191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.177732:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c78 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57860096->57864191] (req 57860096->57864191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.177858:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554571008:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7077us (7915us total) trans 4294981435 rc 0/0 00000100:00100000:3.0:1789760756.177868:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14137 00000100:00100000:3.0:1789760756.177895:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699398801601 time 699416382740 state 0x2 ref 0 00000100:00100000:3.0:1789760756.226860:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554571520 00000100:00100000:3.0:1789760756.226870:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554571520 00000100:00100000:3.0:1789760756.227053:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699445821432 time 699458527510 state 0x0 ref 1 00000100:00100000:3.0:1789760756.227202:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699458787926 time 699458527510 state 0x1 ref 1 00000100:00100000:3.0:1789760756.227235:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699458787926 time 699472853248 state 0x0 ref 1 00000100:00100000:3.0:1789760756.227248:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d9c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554571520:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760756.227491:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57876480, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x1b9a8 00010000:00010000:3.0:1789760756.227500:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c94 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.227515:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760756.227521:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.227527:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57876480->57880575] (req 57876480->57880575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.227543:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57876480->57880575] (req 57876480->57880575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.237629:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57876480 -> 57880576 00002000:00010000:3.0:1789760756.237694:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 113064 -> 113072 00010000:00010000:3.0:1789760756.237708:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57876480->57880575] (req 57876480->57880575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.237721:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c94 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57876480->57880575] (req 57876480->57880575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.237730:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57876480->57880575] (req 57876480->57880575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.237737:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760756.237742:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57876480->57880575] (req 57876480->57880575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.237752:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c94 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57876480->57880575] (req 57876480->57880575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.237769:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c94 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57876480->57880575] (req 57876480->57880575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.237783:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2c94 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57876480->57880575] (req 57876480->57880575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.238022:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d9c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554571520:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10772us (11317us total) trans 4294981439 rc 0/0 00000100:00100000:3.0:1789760756.238033:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14141 00000100:00100000:3.0:1789760756.238042:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699458787926 time 699472853248 state 0x2 ref 0 00000100:00100000:3.0:1789760756.308664:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554572032 00000100:00100000:3.0:1789760756.308676:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554572032 00000100:00100000:3.0:1789760756.308731:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699513105488 time 699534545451 state 0x0 ref 1 00000100:00100000:3.0:1789760756.308746:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699534805867 time 699534545451 state 0x1 ref 1 00000100:00100000:3.0:1789760756.308781:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699534805867 time 699554399447 state 0x0 ref 1 00000100:00100000:3.0:1789760756.308793:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe17850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554572032:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760756.308841:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57892864, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x1b9c8 00010000:00010000:3.0:1789760756.308851:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2cb0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.308867:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760756.308872:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.308888:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2cb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57892864->57896959] (req 57892864->57896959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.308909:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2cb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57892864->57896959] (req 57892864->57896959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.325083:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57892864 -> 57896960 00002000:00010000:3.0:1789760756.325092:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 113096 -> 113104 00010000:00010000:3.0:1789760756.325105:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2cb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57892864->57896959] (req 57892864->57896959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.325118:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2cb0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57892864->57896959] (req 57892864->57896959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.325126:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2cb0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57892864->57896959] (req 57892864->57896959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.325133:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760756.325138:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2cb0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57892864->57896959] (req 57892864->57896959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.325147:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2cb0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57892864->57896959] (req 57892864->57896959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.325165:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2cb0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57892864->57896959] (req 57892864->57896959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.325173:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2cb0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57892864->57896959] (req 57892864->57896959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.325279:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe17850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554572032:12345-192.168.203.22@tcp:4:dd.0 Request processed in 16482us (16766us total) trans 4294981443 rc 0/0 00000100:00100000:3.0:1789760756.325290:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14145 00000100:00100000:3.0:1789760756.325299:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699534805867 time 699554399447 state 0x2 ref 0 00002000:00010000:3.0:1789760756.443945:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57921536 -> 57925632 00002000:00010000:3.0:1789760756.443950:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 113152 -> 113160 00010000:00010000:3.0:1789760756.443962:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c2ce1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57921536->57925631] (req 57921536->57925631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.443975:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c2ce1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57921536->57925631] (req 57921536->57925631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.443985:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c2ce1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57921536->57925631] (req 57921536->57925631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.443993:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760756.443998:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c2ce1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57921536->57925631] (req 57921536->57925631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.444007:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c2ce1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57921536->57925631] (req 57921536->57925631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.444025:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c2ce1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57921536->57925631] (req 57921536->57925631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.444039:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c2ce1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57921536->57925631] (req 57921536->57925631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.444350:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554572928:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13908us (14742us total) trans 4294981450 rc 0/0 00000100:00100000:3.0:1789760756.444361:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14152 00000100:00100000:3.0:1789760756.444371:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699655984096 time 699676043358 state 0x2 ref 0 00002000:00010000:3.0:1789760756.458892:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57925632 -> 57929728 00002000:00010000:3.0:1789760756.458898:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 113160 -> 113168 00010000:00010000:3.0:1789760756.458911:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c2ce8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57925632->57929727] (req 57925632->57929727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.459052:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c2ce8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57925632->57929727] (req 57925632->57929727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.459115:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c2ce8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57925632->57929727] (req 57925632->57929727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.459125:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760756.459130:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c2ce8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57925632->57929727] (req 57925632->57929727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.459139:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c2ce8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57925632->57929727] (req 57925632->57929727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.459165:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c2ce8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57925632->57929727] (req 57925632->57929727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.459172:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c2ce8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57925632->57929727] (req 57925632->57929727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.459367:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e56a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554573056:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10322us (10649us total) trans 4294981451 rc 0/0 00000100:00100000:3.0:1789760756.459421:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14153 00000100:00100000:3.0:1789760756.459432:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699676303774 time 699694644770 state 0x2 ref 0 00002000:00010000:3.0:1789760756.475033:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57929728 -> 57933824 00002000:00010000:3.0:1789760756.475040:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 113168 -> 113176 00010000:00010000:3.0:1789760756.475053:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c2cef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57929728->57933823] (req 57929728->57933823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.475066:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c2cef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57929728->57933823] (req 57929728->57933823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.475075:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c2cef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57929728->57933823] (req 57929728->57933823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.475082:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760756.475086:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c2cef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57929728->57933823] (req 57929728->57933823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.475099:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c2cef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57929728->57933823] (req 57929728->57933823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.475120:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c2cef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57929728->57933823] (req 57929728->57933823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.475128:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c2cef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57929728->57933823] (req 57929728->57933823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.475270:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e57800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554573184:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8209us (8475us total) trans 4294981452 rc 0/0 00000100:00100000:3.0:1789760756.475282:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14154 00000100:00100000:3.0:1789760756.475291:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699694905186 time 699712664292 state 0x2 ref 0 00000100:00100000:3.0:1789760756.498316:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554573440 00000100:00100000:3.0:1789760756.498326:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554573440 00000100:00100000:3.0:1789760756.498411:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699712924708 time 699731736518 state 0x0 ref 1 00000100:00100000:3.0:1789760756.498426:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699731996934 time 699731736518 state 0x1 ref 1 00000100:00100000:3.0:1789760756.498455:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699731996934 time 699744074253 state 0x0 ref 1 00000100:00100000:3.0:1789760756.498467:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554573440:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760756.498513:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57937920, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x1ba20 00010000:00010000:3.0:1789760756.498522:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2cfd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.498538:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760756.498543:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.498549:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2cfd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57937920->57942015] (req 57937920->57942015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.498561:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2cfd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57937920->57942015] (req 57937920->57942015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.515491:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554573568 00000100:00100000:3.0:1789760756.515501:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554573568 00000100:00100000:3.0:1789760756.585344:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554574208 00000100:00100000:3.0:1789760756.585354:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554574208 00000100:00100000:3.0:1789760756.585447:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699799395115 time 699815800685 state 0x0 ref 1 00000100:00100000:3.0:1789760756.585464:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699816061101 time 699815800685 state 0x1 ref 1 00000100:00100000:3.0:1789760756.585494:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699816061101 time 699831113124 state 0x0 ref 1 00000100:00100000:3.0:1789760756.585507:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554574208:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760756.585549:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57962496, mtime: 0x6aad94f3, atime: 0x0, ctime: 0x6aad94f3, blocks: 0x1ba50 00010000:00010000:3.0:1789760756.585558:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d27 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.585573:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760756.585578:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.585584:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57962496->57966591] (req 57962496->57966591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.585597:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57962496->57966591] (req 57962496->57966591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.592141:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57962496 -> 57966592 00002000:00010000:3.0:1789760756.592146:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 113232 -> 113240 00010000:00010000:3.0:1789760756.592160:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57962496->57966591] (req 57962496->57966591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.592174:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d27 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57962496->57966591] (req 57962496->57966591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.592185:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d27 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57962496->57966591] (req 57962496->57966591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.592192:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760756.592197:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d27 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57962496->57966591] (req 57962496->57966591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.592206:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d27 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57962496->57966591] (req 57962496->57966591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.592225:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d27 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57962496->57966591] (req 57962496->57966591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.592233:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d27 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57962496->57966591] (req 57962496->57966591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.592335:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554574208:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6825us (7147us total) trans 4294981460 rc 0/0 00000100:00100000:3.0:1789760756.592346:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14162 00000100:00100000:3.0:1789760756.592355:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699816061101 time 699831113124 state 0x2 ref 0 00000100:00100000:3.0:1789760756.626128:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cc07bcf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3646:1876699573905536:0@lo:400:kworker.0 00000100:00100000:3.0:1789760756.626177:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760756.626213:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cc07bcca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573906048:0@lo:400:kworker.0 00000100:00100000:3.0:1789760756.627033:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760756.627063:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb064ece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573906560:0@lo:400:kworker.0 00000100:00100000:3.0:1789760756.627099:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760756.627124:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb064eea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573907072:0@lo:400:kworker.0 00000100:00100000:3.0:1789760756.627155:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760756.627192:0:6520:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 831 00000100:00100000:3.0:1789760756.627204:0:6520:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb064efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3649:x1876699573906816:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760756.627261:0:6520:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb064efb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3649:x1876699573906816:12345-0@lo:400:kworker.0 Request processed in 60us (1192us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760756.627275:0:6520:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 831 00000100:00100000:3.0:1789760756.627298:0:6520:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 832 00000100:00100000:3.0:1789760756.627310:0:6520:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3646:x1876699573905536:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760756.628572:0:6520:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3646:x1876699573905536:12345-0@lo:400:kworker.0 Request processed in 1128us (2264us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760756.628584:0:6520:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 832 00000100:00100000:3.0:1789760756.628646:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573905792 00000100:00100000:3.0:1789760756.628653:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573905792 00000100:00100000:3.0:1789760756.628680:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573907200 00000100:00100000:3.0:1789760756.628685:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573907200 00000100:00100000:3.0:1789760756.628698:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573906048 00000100:00100000:3.0:1789760756.628779:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573906048 00000100:00100000:3.0:1789760756.628803:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 368 00000100:00100000:3.0:1789760756.628809:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb064eca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3649:x1876699573905792:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760756.628917:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb064eca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3649:x1876699573905792:12345-0@lo:400:kworker.0 Request processed in 107us (2918us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760756.628924:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 368 00000100:00100000:3.0:1789760756.628936:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 369 00000100:00100000:3.0:1789760756.628943:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3647:x1876699573907200:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1789760756.628963:0:6526:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446227968 avail 1299988480 00000100:00100000:3.0:1789760756.628995:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3647:x1876699573907200:12345-0@lo:41:osp-pre-0-1.0 Request processed in 53us (2430us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760756.629001:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 369 00000100:00100000:3.0:1789760756.629012:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 370 00000100:00100000:3.0:1789760756.629018:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573906048:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760756.629163:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573906048:12345-0@lo:400:kworker.0 Request processed in 145us (2132us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760756.629170:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 370 00000100:00100000:3.0:1789760756.629196:0:8436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284 00000100:00100000:3.0:1789760756.629210:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573907072:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760756.629247:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573907072:12345-0@lo:400:kworker.0 Request processed in 42us (2094us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760756.629255:0:8436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284 00000100:00100000:3.0:1789760756.629298:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cc07bcf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3646:1876699573905536:0@lo:400:kworker.0 00000100:00100000:3.0:1789760756.629318:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cc07bcca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573906048:0@lo:400:kworker.0 00000100:00100000:3.0:1789760756.629338:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb064ece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573906560:0@lo:400:kworker.0 00000100:00100000:3.0:1789760756.629356:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb064eea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573907072:0@lo:400:kworker.0 00002000:00010000:3.0:1789760756.643406:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57974784 -> 57978880 00002000:00010000:3.0:1789760756.643410:0:8442:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760755 -> 1789760756 00002000:00010000:3.0:1789760756.643413:0:8442:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760755 -> 1789760756 00002000:00010000:3.0:1789760756.643415:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 113256 -> 113264 00010000:00010000:3.0:1789760756.643428:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c2d3c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57974784->57978879] (req 57974784->57978879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.643440:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c2d3c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57974784->57978879] (req 57974784->57978879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.643449:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c2d3c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57974784->57978879] (req 57974784->57978879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.643456:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760756.643461:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c2d3c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57974784->57978879] (req 57974784->57978879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.643470:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c2d3c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57974784->57978879] (req 57974784->57978879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.643489:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c2d3c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57974784->57978879] (req 57974784->57978879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.643498:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c2d3c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57974784->57978879] (req 57974784->57978879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.643588:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554574592:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5364us (5572us total) trans 4294981463 rc 0/0 00000100:00100000:3.0:1789760756.643599:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14165 00000100:00100000:3.0:1789760756.643608:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699875476154 time 699883827106 state 0x2 ref 0 00000100:00100000:3.0:1789760756.647971:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554574720 00000100:00100000:3.0:1789760756.647981:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554574720 00000100:00100000:3.0:1789760756.648038:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699875476154 time 699883827106 state 0x0 ref 1 00000100:00100000:3.0:1789760756.648054:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699884087522 time 699883827106 state 0x1 ref 1 00000100:00100000:3.0:1789760756.648091:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699884087522 time 699893708530 state 0x0 ref 1 00000100:00100000:3.0:1789760756.648104:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554574720:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760756.648155:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57978880, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x1ba70 00010000:00010000:3.0:1789760756.648165:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d43 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.648182:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760756.648186:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.648193:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57978880->57982975] (req 57978880->57982975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.648212:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57978880->57982975] (req 57978880->57982975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.652706:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57978880 -> 57982976 00002000:00010000:3.0:1789760756.652711:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 113264 -> 113272 00010000:00010000:3.0:1789760756.652724:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57978880->57982975] (req 57978880->57982975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.652737:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d43 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57978880->57982975] (req 57978880->57982975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.652746:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d43 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57978880->57982975] (req 57978880->57982975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.652753:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760756.652758:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d43 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57978880->57982975] (req 57978880->57982975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.652777:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d43 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57978880->57982975] (req 57978880->57982975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.652797:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d43 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57978880->57982975] (req 57978880->57982975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.652804:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d43 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57978880->57982975] (req 57978880->57982975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.652949:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554574720:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4841us (5072us total) trans 4294981464 rc 0/0 00000100:00100000:3.0:1789760756.652960:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14166 00000100:00100000:3.0:1789760756.652969:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699884087522 time 699893708530 state 0x2 ref 0 00000100:00100000:3.0:1789760756.699224:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554575104 00000100:00100000:3.0:1789760756.699234:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554575104 00000100:00100000:3.0:1789760756.699348:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699904512922 time 699930400614 state 0x0 ref 1 00000100:00100000:3.0:1789760756.699364:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699930661030 time 699930400614 state 0x1 ref 1 00000100:00100000:3.0:1789760756.699480:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699930661030 time 699945097473 state 0x0 ref 1 00000100:00100000:3.0:1789760756.699494:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb064eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554575104:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760756.699549:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57991168, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x1ba88 00010000:00010000:3.0:1789760756.699557:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d58 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.699573:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760756.699578:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.699583:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57991168->57995263] (req 57991168->57995263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.699596:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57991168->57995263] (req 57991168->57995263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760756.704496:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 57991168 -> 57995264 00002000:00010000:3.0:1789760756.704502:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 113288 -> 113296 00010000:00010000:3.0:1789760756.704514:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57991168->57995263] (req 57991168->57995263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.704528:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d58 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57991168->57995263] (req 57991168->57995263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.704537:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57991168->57995263] (req 57991168->57995263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.704545:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760756.704549:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57991168->57995263] (req 57991168->57995263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.704560:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d58 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57991168->57995263] (req 57991168->57995263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.704579:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d58 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57991168->57995263] (req 57991168->57995263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.704586:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d58 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57991168->57995263] (req 57991168->57995263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.704709:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb064eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554575104:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5212us (6035us total) trans 4294981467 rc 0/0 00000100:00100000:3.0:1789760756.704720:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14169 00000100:00100000:3.0:1789760756.704730:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699930661030 time 699945097473 state 0x2 ref 0 00000100:00100000:3.0:1789760756.709528:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554575232 00000100:00100000:3.0:1789760756.709538:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554575232 00000100:00100000:3.0:1789760756.709591:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699930661030 time 699945097473 state 0x0 ref 1 00000100:00100000:3.0:1789760756.709607:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699945357889 time 699945097473 state 0x1 ref 1 00000100:00100000:3.0:1789760756.709674:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699945357889 time 699955292074 state 0x0 ref 1 00000100:00100000:3.0:1789760756.709687:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554575232:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760756.709733:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 57995264, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x1ba90 00010000:00010000:3.0:1789760756.709742:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c2d5f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.709758:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760756.709763:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.709769:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c2d5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57995264->57999359] (req 57995264->57999359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.709783:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c2d5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [57995264->57999359] (req 57995264->57999359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760756.753538:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554575616 00000100:00100000:3.0:1789760756.753548:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554575616 00000100:00100000:3.0:1789760756.753601:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699968657611 time 699980225935 state 0x0 ref 1 00000100:00100000:3.0:1789760756.753676:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699980486351 time 699980225935 state 0x1 ref 1 00000100:00100000:3.0:1789760756.753717:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 699980486351 time 699999335365 state 0x0 ref 1 00000100:00100000:3.0:1789760756.753731:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554575616:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760756.753777:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58007552, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x1baa8 00010000:00010000:3.0:1789760756.753786:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d74 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.753802:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=699s v=(0 0) 00010000:00010000:3.0:1789760756.753807:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760756.753813:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58007552->58011647] (req 58007552->58011647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760756.753827:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2d74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58007552->58011647] (req 58007552->58011647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760756.829797:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981475 is committed 00000001:00080000:3.0:1789760756.844192:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981476 is committed 00000001:00080000:3.0:1789760756.859197:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981477 is committed 00000001:00080000:3.0:1789760756.869556:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981478 is committed 00000100:00100000:3.0:1789760757.192915:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554579328 00000100:00100000:3.0:1789760757.192925:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554579328 00000100:00100000:3.0:1789760757.192980:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700398649552 time 700415267678 state 0x0 ref 1 00000100:00100000:3.0:1789760757.192994:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700415528094 time 700415267678 state 0x1 ref 1 00000100:00100000:3.0:1789760757.193029:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700415528094 time 700438646895 state 0x0 ref 1 00000100:00100000:3.0:1789760757.193051:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe14c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554579328:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760757.193101:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58126336, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x1bb90 00010000:00010000:3.0:1789760757.193111:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2e3f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.193128:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760757.193143:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.193151:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2e3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58126336->58130431] (req 58126336->58130431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.193166:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2e3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58126336->58130431] (req 58126336->58130431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760757.202136:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58126336 -> 58130432 00002000:00010000:3.0:1789760757.202142:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 113552 -> 113560 00010000:00010000:3.0:1789760757.202156:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2e3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58126336->58130431] (req 58126336->58130431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.202169:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2e3f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58126336->58130431] (req 58126336->58130431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.202178:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2e3f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58126336->58130431] (req 58126336->58130431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.202191:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760757.202196:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2e3f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58126336->58130431] (req 58126336->58130431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.202206:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2e3f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58126336->58130431] (req 58126336->58130431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.202227:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2e3f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58126336->58130431] (req 58126336->58130431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.202236:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2e3f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58126336->58130431] (req 58126336->58130431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.202447:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe14c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554579328:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9380us (9695us total) trans 4294981500 rc 0/0 00000100:00100000:3.0:1789760757.202458:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14202 00000100:00100000:3.0:1789760757.202468:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700415528094 time 700438646895 state 0x2 ref 0 00000100:00100000:3.0:1789760757.257790:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700473918049 time 700503402803 state 0x0 ref 1 00000100:00100000:3.0:1789760757.257812:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03c1b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554579712:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760757.257857:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58138624, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x1bba8 00010000:00010000:3.0:1789760757.257865:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2e54 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.257895:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760757.257899:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.257906:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2e54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58138624->58142719] (req 58138624->58142719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.257923:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2e54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58138624->58142719] (req 58138624->58142719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760757.264759:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58138624 -> 58142720 00002000:00010000:3.0:1789760757.264763:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 113576 -> 113584 00010000:00010000:3.0:1789760757.264776:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2e54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58138624->58142719] (req 58138624->58142719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.264789:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2e54 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58138624->58142719] (req 58138624->58142719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.264798:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2e54 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58138624->58142719] (req 58138624->58142719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.264806:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760757.264811:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2e54 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58138624->58142719] (req 58138624->58142719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.264820:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2e54 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58138624->58142719] (req 58138624->58142719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.264843:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2e54 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58138624->58142719] (req 58138624->58142719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.264852:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2e54 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58138624->58142719] (req 58138624->58142719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.264969:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03c1b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554579712:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7159us (8525us total) trans 4294981503 rc 0/0 00000100:00100000:3.0:1789760757.264979:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14205 00000100:00100000:3.0:1789760757.264988:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700473918049 time 700503402803 state 0x2 ref 0 00000100:00100000:3.0:1789760757.273504:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554579840 00000100:00100000:3.0:1789760757.273519:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554579840 00000100:00100000:3.0:1789760757.273577:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700473918049 time 700503402803 state 0x0 ref 1 00000100:00100000:3.0:1789760757.273592:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700503663219 time 700503402803 state 0x1 ref 1 00000100:00100000:3.0:1789760757.273675:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700503663219 time 700519292306 state 0x0 ref 1 00000100:00100000:3.0:1789760757.273690:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554579840:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760757.273741:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58142720, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x1bbb0 00010000:00010000:3.0:1789760757.273752:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2e5b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.273770:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760757.273776:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.273783:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2e5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58142720->58146815] (req 58142720->58146815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.273801:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2e5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58142720->58146815] (req 58142720->58146815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.280205:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981504 is committed 00000001:00080000:3.0:1789760757.298006:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981505 is committed 00000001:00080000:3.0:1789760757.310511:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981506 is committed 00000100:00100000:3.0:1789760757.324186:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554580224 00000100:00100000:3.0:1789760757.324200:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554580224 00000100:00100000:3.0:1789760757.324815:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700537399905 time 700551720579 state 0x0 ref 1 00000100:00100000:3.0:1789760757.324830:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700551980995 time 700551720579 state 0x1 ref 1 00000100:00100000:3.0:1789760757.324920:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700551980995 time 700570537632 state 0x0 ref 1 00000100:00100000:3.0:1789760757.324936:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554580224:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760757.324982:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58155008, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x1bbc8 00010000:00010000:3.0:1789760757.324991:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c2e70 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.325007:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760757.325011:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.325018:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c2e70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58155008->58159103] (req 58155008->58159103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.325039:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c2e70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58155008->58159103] (req 58155008->58159103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.330953:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981507 is committed 00000100:00100000:3.0:1789760757.348328:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554580352 00000100:00100000:3.0:1789760757.348339:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554580352 00000100:00100000:3.0:1789760757.348432:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700551980995 time 700570537632 state 0x0 ref 1 00000100:00100000:3.0:1789760757.348447:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700570798048 time 700570537632 state 0x1 ref 1 00000100:00100000:3.0:1789760757.348469:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700570798048 time 700594088535 state 0x0 ref 1 00000100:00100000:3.0:1789760757.348481:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554580352:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760757.348529:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58159104, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x1bbd0 00010000:00010000:3.0:1789760757.348537:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c2e77 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.348553:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760757.348557:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.348564:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c2e77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58159104->58163199] (req 58159104->58163199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.348578:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c2e77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58159104->58163199] (req 58159104->58163199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760757.359146:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58159104 -> 58163200 00002000:00010000:3.0:1789760757.359152:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 113616 -> 113624 00010000:00010000:3.0:1789760757.359165:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c2e77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58159104->58163199] (req 58159104->58163199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.359178:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c2e77 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58159104->58163199] (req 58159104->58163199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.359188:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c2e77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58159104->58163199] (req 58159104->58163199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.359196:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760757.359200:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c2e77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58159104->58163199] (req 58159104->58163199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.359210:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c2e77 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58159104->58163199] (req 58159104->58163199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.359230:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c2e77 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58159104->58163199] (req 58159104->58163199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.359238:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c2e77 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58159104->58163199] (req 58159104->58163199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.360290:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554580352:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11805us (12666us total) trans 4294981508 rc 0/0 00000100:00100000:3.0:1789760757.360301:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14210 00000100:00100000:3.0:1789760757.360310:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700570798048 time 700594088535 state 0x2 ref 0 00000100:00100000:3.0:1789760757.368784:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760757.394159:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981510 is committed 00000001:00080000:3.0:1789760757.404067:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981511 is committed 00000100:00100000:3.0:1789760757.412747:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760757.424798:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981512 is committed 00000001:00080000:3.0:1789760757.453962:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981514 is committed 00000100:00100000:3.0:1789760757.460993:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554581248 00000100:00100000:3.0:1789760757.461003:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554581248 00000100:00100000:3.0:1789760757.461058:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700678529598 time 700694739059 state 0x0 ref 1 00000100:00100000:3.0:1789760757.461074:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700694999475 time 700694739059 state 0x1 ref 1 00000100:00100000:3.0:1789760757.461108:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700694999475 time 700706725917 state 0x0 ref 1 00000100:00100000:3.0:1789760757.461129:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04544380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554581248:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760757.461177:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58187776, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x1bc08 00010000:00010000:3.0:1789760757.461185:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c2ea8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.461201:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760757.461205:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.461212:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c2ea8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58187776->58191871] (req 58187776->58191871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.461232:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c2ea8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58187776->58191871] (req 58187776->58191871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.470838:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554581376 00000100:00100000:3.0:1789760757.470849:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554581376 00000100:00100000:3.0:1789760757.470910:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700694999475 time 700706725917 state 0x0 ref 1 00000100:00100000:3.0:1789760757.470925:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700706986333 time 700706725917 state 0x1 ref 1 00000100:00100000:3.0:1789760757.470988:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700706986333 time 700716606796 state 0x0 ref 1 00000100:00100000:3.0:1789760757.471001:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554581376:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760757.471046:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58191872, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x1bc10 00010000:00010000:3.0:1789760757.471056:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c2eaf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.471073:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760757.471077:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.471083:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c2eaf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58191872->58195967] (req 58191872->58195967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.471097:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c2eaf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58191872->58195967] (req 58191872->58195967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.477769:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981516 is committed 00000001:00080000:3.0:1789760757.487748:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981517 is committed 00000001:00080000:3.0:1789760757.498405:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981518 is committed 00000001:00080000:3.0:1789760757.514457:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981519 is committed 00000001:00080000:3.0:1789760757.528097:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981520 is committed 00000100:00100000:3.0:1789760757.542094:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554582016 00000100:00100000:3.0:1789760757.542107:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554582016 00000100:00100000:3.0:1789760757.542165:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700754724305 time 700766883114 state 0x0 ref 1 00000100:00100000:3.0:1789760757.542181:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700767143530 time 700766883114 state 0x1 ref 1 00000100:00100000:3.0:1789760757.542222:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700767143530 time 700787840448 state 0x0 ref 1 00000100:00100000:3.0:1789760757.542236:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554582016:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760757.542295:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58212352, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x1bc38 00010000:00010000:3.0:1789760757.542304:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2ed2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.542320:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760757.542324:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.542330:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2ed2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58212352->58216447] (req 58212352->58216447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.542345:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2ed2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58212352->58216447] (req 58212352->58216447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760757.552773:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58212352 -> 58216448 00002000:00010000:3.0:1789760757.552779:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 113720 -> 113728 00010000:00010000:3.0:1789760757.552792:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2ed2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58212352->58216447] (req 58212352->58216447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.552806:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2ed2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58212352->58216447] (req 58212352->58216447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.552816:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2ed2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58212352->58216447] (req 58212352->58216447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.552824:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760757.552829:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2ed2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58212352->58216447] (req 58212352->58216447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.552839:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2ed2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58212352->58216447] (req 58212352->58216447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.552858:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2ed2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58212352->58216447] (req 58212352->58216447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.552877:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2ed2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58212352->58216447] (req 58212352->58216447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.552978:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554582016:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10741us (11851us total) trans 4294981521 rc 0/0 00000100:00100000:3.0:1789760757.552989:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14223 00000100:00100000:3.0:1789760757.552998:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700767143530 time 700787840448 state 0x2 ref 0 00000001:00080000:3.0:1789760757.569841:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981522 is committed 00000001:00080000:3.0:1789760757.592309:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981523 is committed 00000001:00080000:3.0:1789760757.619784:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981524 is committed 00000100:00100000:3.0:1789760757.628254:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554582528 00000100:00100000:3.0:1789760757.628263:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554582528 00000100:00100000:3.0:1789760757.628316:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700824185577 time 700857387455 state 0x0 ref 1 00000100:00100000:3.0:1789760757.628330:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700857647871 time 700857387455 state 0x1 ref 1 00000100:00100000:3.0:1789760757.628360:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700857647871 time 700873978939 state 0x0 ref 1 00000100:00100000:3.0:1789760757.628371:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554582528:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760757.628447:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58228736, mtime: 0x6aad94f4, atime: 0x0, ctime: 0x6aad94f4, blocks: 0x1bc58 00010000:00010000:3.0:1789760757.628456:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2eee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.628470:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760757.628475:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.628481:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2eee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58228736->58232831] (req 58228736->58232831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.628493:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c2eee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58228736->58232831] (req 58228736->58232831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.640154:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981525 is committed 00000001:00080000:3.0:1789760757.654779:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981526 is committed 00000001:00080000:3.0:1789760757.670790:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981527 is committed 00000100:00100000:3.0:1789760757.676306:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554582912 00000100:00100000:3.0:1789760757.676315:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554582912 00000100:00100000:3.0:1789760757.676362:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700893938961 time 700908421983 state 0x0 ref 1 00000100:00100000:3.0:1789760757.680204:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700908682399 time 700908421983 state 0x1 ref 1 00000100:00100000:3.0:1789760757.680429:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700908682399 time 700926040853 state 0x0 ref 1 00000100:00100000:3.0:1789760757.680454:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554582912:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760757.680510:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58241024, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0x1bc70 00010000:00010000:3.0:1789760757.680520:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2f03 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.680536:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760757.680541:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.680548:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2f03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58241024->58245119] (req 58241024->58245119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.680563:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2f03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58241024->58245119] (req 58241024->58245119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760757.686091:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58241024 -> 58245120 00002000:00010000:3.0:1789760757.686096:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 113776 -> 113784 00010000:00010000:3.0:1789760757.686109:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2f03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58241024->58245119] (req 58241024->58245119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.686123:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2f03 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58241024->58245119] (req 58241024->58245119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.686139:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2f03 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58241024->58245119] (req 58241024->58245119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.686146:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760757.686151:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2f03 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58241024->58245119] (req 58241024->58245119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.686160:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2f03 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58241024->58245119] (req 58241024->58245119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.686179:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2f03 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58241024->58245119] (req 58241024->58245119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.686187:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2f03 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58241024->58245119] (req 58241024->58245119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.686280:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554582912:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5834us (10101us total) trans 4294981528 rc 0/0 00000100:00100000:3.0:1789760757.686290:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14230 00000100:00100000:3.0:1789760757.686299:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700908682399 time 700926040853 state 0x2 ref 0 00000001:00080000:3.0:1789760757.699971:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981529 is committed 00000100:00100000:3.0:1789760757.747067:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554583424 00000100:00100000:3.0:1789760757.747077:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554583424 00000100:00100000:3.0:1789760757.747131:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700951030670 time 700966894618 state 0x0 ref 1 00000100:00100000:3.0:1789760757.747150:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700967155034 time 700966894618 state 0x1 ref 1 00000100:00100000:3.0:1789760757.747205:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700967155034 time 700992823679 state 0x0 ref 1 00000100:00100000:3.0:1789760757.747218:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554583424:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760757.747270:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58257408, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0x1bc90 00010000:00010000:3.0:1789760757.747279:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2f1f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.747293:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=700s v=(0 0) 00010000:00010000:3.0:1789760757.747298:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.747304:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2f1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58257408->58261503] (req 58257408->58261503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.747318:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2f1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58257408->58261503] (req 58257408->58261503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760757.754668:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58257408 -> 58261504 00002000:00010000:3.0:1789760757.754673:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 113808 -> 113816 00010000:00010000:3.0:1789760757.754685:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2f1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58257408->58261503] (req 58257408->58261503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.754697:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2f1f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58257408->58261503] (req 58257408->58261503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.754715:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2f1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58257408->58261503] (req 58257408->58261503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.754723:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760757.754728:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2f1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58257408->58261503] (req 58257408->58261503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.754737:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2f1f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58257408->58261503] (req 58257408->58261503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.754754:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2f1f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58257408->58261503] (req 58257408->58261503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.754761:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2f1f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58257408->58261503] (req 58257408->58261503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.754855:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554583424:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7635us (7886us total) trans 4294981532 rc 0/0 00000100:00100000:3.0:1789760757.754875:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14234 00000100:00100000:3.0:1789760757.754885:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700967155034 time 700992823679 state 0x2 ref 0 00000100:00100000:3.0:1789760757.759997:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554583552 00000100:00100000:3.0:1789760757.760005:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554583552 00000100:00100000:3.0:1789760757.760056:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700967155034 time 700992823679 state 0x0 ref 1 00000100:00100000:3.0:1789760757.760070:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700993084095 time 700992823679 state 0x1 ref 1 00000100:00100000:3.0:1789760757.760135:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 700993084095 time 701005753274 state 0x0 ref 1 00000100:00100000:3.0:1789760757.760151:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554583552:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760757.760196:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58261504, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0x1bc98 00010000:00010000:3.0:1789760757.760204:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c2f26 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.760220:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:3.0:1789760757.760224:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.760231:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c2f26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58261504->58265599] (req 58261504->58265599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.760257:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c2f26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58261504->58265599] (req 58261504->58265599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760757.794480:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981535 is committed 00000100:00100000:3.0:1789760757.805625:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554583936 00000100:00100000:3.0:1789760757.805636:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554583936 00000100:00100000:3.0:1789760757.805690:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 701019211263 time 701033804500 state 0x0 ref 1 00000100:00100000:3.0:1789760757.805704:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 701034064916 time 701033804500 state 0x1 ref 1 00000100:00100000:3.0:1789760757.805730:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 701034064916 time 701051348716 state 0x0 ref 1 00000100:00100000:3.0:1789760757.805743:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04545500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554583936:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760757.805792:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58273792, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0x1bcb0 00010000:00010000:3.0:1789760757.805801:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2f3b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.805824:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:3.0:1789760757.805828:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760757.805835:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2f3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58273792->58277887] (req 58273792->58277887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.805853:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c2f3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58273792->58277887] (req 58273792->58277887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.859803:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760757.901454:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:3.0:1789760757.940133:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58302464 -> 58306560 00002000:00010000:3.0:1789760757.940144:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 113896 -> 113904 00010000:00010000:3.0:1789760757.940158:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c2f6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58302464->58306559] (req 58302464->58306559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.940171:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c2f6c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58302464->58306559] (req 58302464->58306559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.940181:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c2f6c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58302464->58306559] (req 58302464->58306559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.940189:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760757.940194:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c2f6c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58302464->58306559] (req 58302464->58306559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.940203:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c2f6c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58302464->58306559] (req 58302464->58306559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.940222:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c2f6c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58302464->58306559] (req 58302464->58306559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760757.940230:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c2f6c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58302464->58306559] (req 58302464->58306559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760757.940338:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04545f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554584832:12345-192.168.203.22@tcp:4:dd.0 Request processed in 16361us (16787us total) trans 4294981543 rc 0/0 00000100:00100000:3.0:1789760757.940349:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14245 00000100:00100000:3.0:1789760757.940359:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 701147674499 time 701169579645 state 0x2 ref 0 00000100:00100000:3.0:1789760758.003051:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554585344 00000100:00100000:3.0:1789760758.003061:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554585344 00000100:00100000:3.0:1789760758.003109:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 701213797996 time 701228694799 state 0x0 ref 1 00000100:00100000:3.0:1789760758.003124:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 701228955215 time 701228694799 state 0x1 ref 1 00000100:00100000:3.0:1789760758.003152:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 701228955215 time 701248770808 state 0x0 ref 1 00000100:00100000:3.0:1789760758.003163:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03c1aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554585344:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760758.003204:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58318848, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0x1bd08 00010000:00010000:3.0:1789760758.003213:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2f88 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.003229:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:3.0:1789760758.003234:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.003558:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2f88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58318848->58322943] (req 58318848->58322943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.003574:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2f88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58318848->58322943] (req 58318848->58322943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760758.012592:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58318848 -> 58322944 00002000:00010000:3.0:1789760758.012598:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 113928 -> 113936 00010000:00010000:3.0:1789760758.012611:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2f88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58318848->58322943] (req 58318848->58322943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.012625:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2f88 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58318848->58322943] (req 58318848->58322943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.012634:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2f88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58318848->58322943] (req 58318848->58322943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.012641:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760758.012645:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2f88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58318848->58322943] (req 58318848->58322943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.012654:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2f88 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58318848->58322943] (req 58318848->58322943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.012673:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2f88 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58318848->58322943] (req 58318848->58322943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.012687:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2f88 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58318848->58322943] (req 58318848->58322943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.012783:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03c1aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554585344:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9616us (9875us total) trans 4294981547 rc 0/0 00000100:00100000:3.0:1789760758.012794:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14249 00000100:00100000:3.0:1789760758.012802:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 701228955215 time 701248770808 state 0x2 ref 0 00000100:00100000:3.0:1789760758.020512:0:11089:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554585600 00000100:00100000:3.0:1789760758.020518:0:11089:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554585600 00000100:00100000:3.0:1789760758.020604:0:11089:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554585856 00000100:00100000:3.0:1789760758.020608:0:11089:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554585856 00000100:00100000:3.0:1789760758.020890:0:11089:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 835 00000100:00100000:3.0:1789760758.020898:0:11089:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03c1a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+8:2413:x1876699554585600:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:3.0:1789760758.020939:0:11089:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03c1a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+8:2413:x1876699554585600:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 42us (2808us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760758.020946:0:11089:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 835 00000100:00100000:3.0:1789760758.020961:0:11089:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 836 00000100:00100000:3.0:1789760758.020968:0:11089:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03c19f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2413:x1876699554585856:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:3.0:1789760758.020993:0:11089:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03c19f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2413:x1876699554585856:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 26us (1931us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760758.020999:0:11089:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 836 00000100:00100000:3.0:1789760758.022053:0:11089:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 837 00000100:00100000:3.0:1789760758.022062:0:11089:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03c1b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554585728:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:3.0:1789760758.022108:0:11089:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03c1b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554585728:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 47us (2054us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760758.022114:0:11089:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 837 00000100:00100000:3.0:1789760758.022130:0:11089:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 838 00000100:00100000:3.0:1789760758.022136:0:11089:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03c1ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2413:x1876699554585984:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:3.0:1789760758.022160:0:11089:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03c1ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2413:x1876699554585984:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 25us (464us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760758.022166:0:11089:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 838 00000100:00100000:3.0:1789760758.053249:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760758.066502:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554586496 00000100:00100000:3.0:1789760758.066512:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554586496 00000100:00100000:3.0:1789760758.066566:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 701285634171 time 701299099834 state 0x0 ref 1 00000100:00100000:3.0:1789760758.066581:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 701299360250 time 701299099834 state 0x1 ref 1 00000100:00100000:3.0:1789760758.066612:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 701299360250 time 701312230786 state 0x0 ref 1 00000100:00100000:3.0:1789760758.066633:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05b8c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554586496:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760758.066678:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58335232, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0x1bd28 00010000:00010000:3.0:1789760758.066688:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2fa4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.066704:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:3.0:1789760758.066708:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.066715:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2fa4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58335232->58339327] (req 58335232->58339327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.066729:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2fa4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58335232->58339327] (req 58335232->58339327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760758.078232:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58335232 -> 58339328 00002000:00010000:3.0:1789760758.078249:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 113960 -> 113968 00010000:00010000:3.0:1789760758.078262:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2fa4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58335232->58339327] (req 58335232->58339327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.078275:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2fa4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58335232->58339327] (req 58335232->58339327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.078285:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2fa4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58335232->58339327] (req 58335232->58339327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.078293:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760758.078297:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2fa4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58335232->58339327] (req 58335232->58339327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.078306:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2fa4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58335232->58339327] (req 58335232->58339327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.078325:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2fa4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58335232->58339327] (req 58335232->58339327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.078333:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2fa4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58335232->58339327] (req 58335232->58339327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.078465:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05b8c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554586496:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11839us (12129us total) trans 4294981551 rc 0/0 00000100:00100000:3.0:1789760758.078476:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14253 00000100:00100000:3.0:1789760758.078486:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 701299360250 time 701312230786 state 0x2 ref 0 00000100:00100000:3.0:1789760758.130115:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760758.180143:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760758.204325:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554587520 00000100:00100000:3.0:1789760758.204335:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554587520 00000100:00100000:3.0:1789760758.204428:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 701410809476 time 701426381670 state 0x0 ref 1 00000100:00100000:3.0:1789760758.204444:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 701426642086 time 701426381670 state 0x1 ref 1 00000100:00100000:3.0:1789760758.204479:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 701426642086 time 701450097026 state 0x0 ref 1 00000100:00100000:3.0:1789760758.204492:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05b8c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554587520:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760758.205375:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58368000, mtime: 0x6aad94f5, atime: 0x0, ctime: 0x6aad94f5, blocks: 0x1bd68 00010000:00010000:3.0:1789760758.205478:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2fdc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.205495:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=701s v=(0 0) 00010000:00010000:3.0:1789760758.205500:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.205506:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2fdc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58368000->58372095] (req 58368000->58372095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.205690:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c2fdc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58368000->58372095] (req 58368000->58372095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.245590:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760758.274782:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760758.287540:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760758.336459:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981566 is committed 00000100:00100000:3.0:1789760758.345173:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760758.598050:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981583 is committed 00000001:00080000:3.0:1789760758.624446:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981584 is committed 00000100:00100000:3.0:1789760758.654330:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760758.767179:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981593 is committed 00000001:00080000:3.0:1789760758.782870:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981594 is committed 00000100:00100000:3.0:1789760758.792302:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760758.824173:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554592384 00000100:00100000:3.0:1789760758.824182:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554592384 00000100:00100000:3.0:1789760758.824237:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702038382550 time 702057868874 state 0x0 ref 1 00000100:00100000:3.0:1789760758.824251:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702058129290 time 702057868874 state 0x1 ref 1 00000100:00100000:3.0:1789760758.824274:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702058129290 time 702069893004 state 0x0 ref 1 00000100:00100000:3.0:1789760758.824285:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554592384:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760758.824326:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58523648, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0x1be98 00010000:00010000:3.0:1789760758.824334:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c30e6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.824349:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760758.824353:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.824360:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c30e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58523648->58527743] (req 58523648->58527743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.824373:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c30e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58523648->58527743] (req 58523648->58527743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.840877:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760758.849463:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981598 is committed 00000001:00080000:3.0:1789760758.864912:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981599 is committed 00000100:00100000:3.0:1789760758.936937:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554593280 00000100:00100000:3.0:1789760758.936946:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554593280 00000100:00100000:3.0:1789760758.936996:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702153392791 time 702169490812 state 0x0 ref 1 00000100:00100000:3.0:1789760758.937010:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702169751228 time 702169490812 state 0x1 ref 1 00000100:00100000:3.0:1789760758.937040:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702169751228 time 702182658349 state 0x0 ref 1 00000100:00100000:3.0:1789760758.937050:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554593280:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760758.937095:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58552320, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0x1bed0 00010000:00010000:3.0:1789760758.937104:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3117 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.937120:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760758.937126:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760758.937132:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3117 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58552320->58556415] (req 58552320->58556415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.937146:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3117 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58552320->58556415] (req 58552320->58556415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760758.948145:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58552320 -> 58556416 00002000:00010000:3.0:1789760758.948162:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 114384 -> 114392 00010000:00010000:3.0:1789760758.948176:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3117 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58552320->58556415] (req 58552320->58556415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.948189:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3117 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58552320->58556415] (req 58552320->58556415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.948199:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3117 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58552320->58556415] (req 58552320->58556415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.948208:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760758.948213:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3117 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58552320->58556415] (req 58552320->58556415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.948222:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3117 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58552320->58556415] (req 58552320->58556415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.948242:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3117 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58552320->58556415] (req 58552320->58556415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760758.948255:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3117 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58552320->58556415] (req 58552320->58556415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760758.948435:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554593280:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11380us (11897us total) trans 4294981604 rc 0/0 00000100:00100000:3.0:1789760758.948446:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14306 00000100:00100000:3.0:1789760758.948455:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702169751228 time 702182658349 state 0x2 ref 0 00000100:00100000:3.0:1789760759.017224:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760759.027715:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554593792 00000100:00100000:3.0:1789760759.027727:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554593792 00000100:00100000:3.0:1789760759.027790:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702251587051 time 702263348357 state 0x0 ref 1 00000100:00100000:3.0:1789760759.027805:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702263608773 time 702263348357 state 0x1 ref 1 00000100:00100000:3.0:1789760759.027879:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702263608773 time 702273496554 state 0x0 ref 1 00000100:00100000:3.0:1789760759.027893:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554593792:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760759.027939:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58568704, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0x1bef0 00010000:00010000:3.0:1789760759.027948:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3133 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.027963:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760759.027968:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.027975:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3133 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58568704->58572799] (req 58568704->58572799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.027989:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3133 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58568704->58572799] (req 58568704->58572799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760759.037968:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58568704 -> 58572800 00002000:00010000:3.0:1789760759.037973:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 114416 -> 114424 00010000:00010000:3.0:1789760759.037985:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3133 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58568704->58572799] (req 58568704->58572799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.037998:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3133 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58568704->58572799] (req 58568704->58572799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.038017:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3133 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58568704->58572799] (req 58568704->58572799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.038026:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760759.038030:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3133 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58568704->58572799] (req 58568704->58572799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.038040:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3133 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58568704->58572799] (req 58568704->58572799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.038060:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3133 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58568704->58572799] (req 58568704->58572799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.038068:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3133 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58568704->58572799] (req 58568704->58572799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.038160:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554593792:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10266us (11086us total) trans 4294981608 rc 0/0 00000100:00100000:3.0:1789760759.038170:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14310 00000100:00100000:3.0:1789760759.038179:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702263608773 time 702273496554 state 0x2 ref 0 00000001:00080000:3.0:1789760759.052684:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981609 is committed 00000001:00080000:3.0:1789760759.075207:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981610 is committed 00000001:00080000:3.0:1789760759.099418:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981611 is committed 00000100:00100000:3.0:1789760759.104900:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554594304 00000100:00100000:3.0:1789760759.104909:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554594304 00000100:00100000:3.0:1789760759.104969:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702309974941 time 702338842782 state 0x0 ref 1 00000100:00100000:3.0:1789760759.104986:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702339103198 time 702338842782 state 0x1 ref 1 00000100:00100000:3.0:1789760759.105021:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702339103198 time 702350639434 state 0x0 ref 1 00000100:00100000:3.0:1789760759.105033:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06569c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554594304:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760759.105081:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58585088, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0x1bf10 00010000:00010000:3.0:1789760759.105090:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c314f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.105105:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760759.105110:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.105116:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c314f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58585088->58589183] (req 58585088->58589183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.105136:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c314f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58585088->58589183] (req 58585088->58589183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760759.110290:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58585088 -> 58589184 00002000:00010000:3.0:1789760759.110295:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 114448 -> 114456 00010000:00010000:3.0:1789760759.110310:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c314f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58585088->58589183] (req 58585088->58589183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.110323:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c314f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58585088->58589183] (req 58585088->58589183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.110333:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c314f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58585088->58589183] (req 58585088->58589183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.110341:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760759.110347:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c314f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58585088->58589183] (req 58585088->58589183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.110356:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c314f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58585088->58589183] (req 58585088->58589183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.110375:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c314f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58585088->58589183] (req 58585088->58589183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.110457:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c314f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58585088->58589183] (req 58585088->58589183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.126727:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981613 is committed 00000100:00100000:3.0:1789760759.160446:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760759.177127:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981617 is committed 00002000:00010000:3.0:1789760759.189327:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58609664 -> 58613760 00002000:00010000:3.0:1789760759.189332:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 114496 -> 114504 00010000:00010000:3.0:1789760759.189345:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c3179 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58609664->58613759] (req 58609664->58613759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.189358:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c3179 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58609664->58613759] (req 58609664->58613759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.189368:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c3179 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58609664->58613759] (req 58609664->58613759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.189402:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760759.189408:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c3179 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58609664->58613759] (req 58609664->58613759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.189419:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c3179 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58609664->58613759] (req 58609664->58613759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.189439:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c3179 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58609664->58613759] (req 58609664->58613759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.189448:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c3179 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58609664->58613759] (req 58609664->58613759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.189528:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06569880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554595072:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6893us (7089us total) trans 4294981618 rc 0/0 00000100:00100000:3.0:1789760759.189538:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14320 00000100:00100000:3.0:1789760759.189547:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702417223011 time 702428237476 state 0x2 ref 0 00000100:00100000:3.0:1789760759.193908:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb04547b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573907584:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789760759.237301:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760759.282716:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760759.305907:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760759.320272:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:3.0:1789760759.326432:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58650624 -> 58654720 00002000:00010000:3.0:1789760759.326438:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 114576 -> 114584 00010000:00010000:3.0:1789760759.326451:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c31bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58650624->58654719] (req 58650624->58654719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.326464:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c31bf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58650624->58654719] (req 58650624->58654719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.326473:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c31bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58650624->58654719] (req 58650624->58654719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.326479:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760759.326484:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c31bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58650624->58654719] (req 58650624->58654719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.326498:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c31bf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58650624->58654719] (req 58650624->58654719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.326517:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c31bf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58650624->58654719] (req 58650624->58654719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.326524:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c31bf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58650624->58654719] (req 58650624->58654719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.326923:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554596352:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6397us (6657us total) trans 4294981628 rc 0/0 00000100:00100000:3.0:1789760759.326935:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14330 00000100:00100000:3.0:1789760759.326943:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702551941056 time 702566130092 state 0x2 ref 0 00000100:00100000:3.0:1789760759.330548:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:3.0:1789760759.346670:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58658816 -> 58662912 00002000:00010000:3.0:1789760759.346675:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 114592 -> 114600 00010000:00010000:3.0:1789760759.346689:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c31cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58658816->58662911] (req 58658816->58662911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.346702:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c31cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58658816->58662911] (req 58658816->58662911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.346712:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c31cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58658816->58662911] (req 58658816->58662911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.346726:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760759.346731:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c31cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58658816->58662911] (req 58658816->58662911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.346740:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c31cd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58658816->58662911] (req 58658816->58662911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.346761:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c31cd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58658816->58662911] (req 58658816->58662911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.346769:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c31cd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58658816->58662911] (req 58658816->58662911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.346866:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06569500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554596608:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5096us (5544us total) trans 4294981630 rc 0/0 00000100:00100000:3.0:1789760759.346878:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14332 00000100:00100000:3.0:1789760759.346887:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702576904744 time 702587372251 state 0x2 ref 0 00000100:00100000:3.0:1789760759.350580:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760759.362084:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554596864 00000100:00100000:3.0:1789760759.362093:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554596864 00000100:00100000:3.0:1789760759.362140:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702587632667 time 702596757129 state 0x0 ref 1 00000100:00100000:3.0:1789760759.362164:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702597017545 time 702596757129 state 0x1 ref 1 00000100:00100000:3.0:1789760759.362223:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702597017545 time 702607841838 state 0x0 ref 1 00000100:00100000:3.0:1789760759.362237:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554596864:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760759.362291:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58667008, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0x1bfb0 00010000:00010000:3.0:1789760759.362300:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c31db lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.362315:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760759.362319:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.362326:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c31db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58667008->58671103] (req 58667008->58671103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.362341:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c31db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58667008->58671103] (req 58667008->58671103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760759.366596:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58667008 -> 58671104 00002000:00010000:3.0:1789760759.366601:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 114608 -> 114616 00010000:00010000:3.0:1789760759.366614:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c31db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58667008->58671103] (req 58667008->58671103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.366627:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c31db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58667008->58671103] (req 58667008->58671103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.366636:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c31db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58667008->58671103] (req 58667008->58671103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.366643:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760759.366649:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c31db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58667008->58671103] (req 58667008->58671103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.366658:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c31db lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58667008->58671103] (req 58667008->58671103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.366677:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c31db lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58667008->58671103] (req 58667008->58671103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.366686:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c31db lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58667008->58671103] (req 58667008->58671103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.366813:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554596864:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4573us (4883us total) trans 4294981632 rc 0/0 00000100:00100000:3.0:1789760759.366824:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14334 00000100:00100000:3.0:1789760759.366832:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702597017545 time 702607841838 state 0x2 ref 0 00000100:00100000:3.0:1789760759.383805:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554597120 00000100:00100000:3.0:1789760759.383815:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554597120 00000100:00100000:3.0:1789760759.383879:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702608102254 time 702616820155 state 0x0 ref 1 00000100:00100000:3.0:1789760759.383894:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702617080571 time 702616820155 state 0x1 ref 1 00000100:00100000:3.0:1789760759.383927:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702617080571 time 702629545875 state 0x0 ref 1 00000100:00100000:3.0:1789760759.383950:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04545c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554597120:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760759.384008:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58675200, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0x1bfc0 00010000:00010000:3.0:1789760759.384018:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c31e9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.384034:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760759.384039:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.384045:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c31e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58675200->58679295] (req 58675200->58679295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.384059:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c31e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58675200->58679295] (req 58675200->58679295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760759.389983:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58675200 -> 58679296 00002000:00010000:3.0:1789760759.389988:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 114624 -> 114632 00010000:00010000:3.0:1789760759.390002:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c31e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58675200->58679295] (req 58675200->58679295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.390016:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c31e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58675200->58679295] (req 58675200->58679295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.390025:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c31e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58675200->58679295] (req 58675200->58679295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.390032:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760759.390037:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c31e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58675200->58679295] (req 58675200->58679295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.390046:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c31e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58675200->58679295] (req 58675200->58679295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.390068:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c31e9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58675200->58679295] (req 58675200->58679295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.390075:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c31e9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58675200->58679295] (req 58675200->58679295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.390216:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04545c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554597120:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6273us (6554us total) trans 4294981634 rc 0/0 00000100:00100000:3.0:1789760759.390236:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14336 00000100:00100000:3.0:1789760759.390246:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702617080571 time 702629545875 state 0x2 ref 0 00000100:00100000:3.0:1789760759.432194:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554597632 00000100:00100000:3.0:1789760759.432203:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554597632 00000100:00100000:3.0:1789760759.432255:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702652800485 time 702667250829 state 0x0 ref 1 00000100:00100000:3.0:1789760759.432270:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702667511245 time 702667250829 state 0x1 ref 1 00000100:00100000:3.0:1789760759.432302:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702667511245 time 702677920420 state 0x0 ref 1 00000100:00100000:3.0:1789760759.432313:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554597632:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760759.432363:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58691584, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0x1bfe0 00010000:00010000:3.0:1789760759.432372:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3205 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.432427:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760759.432432:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.432438:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3205 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58691584->58695679] (req 58691584->58695679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.432453:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3205 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58691584->58695679] (req 58691584->58695679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760759.439466:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58691584 -> 58695680 00002000:00010000:3.0:1789760759.439474:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 114656 -> 114664 00010000:00010000:3.0:1789760759.439487:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3205 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58691584->58695679] (req 58691584->58695679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.439499:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3205 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58691584->58695679] (req 58691584->58695679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.439508:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3205 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58691584->58695679] (req 58691584->58695679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.439515:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760759.439520:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3205 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58691584->58695679] (req 58691584->58695679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.439528:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3205 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58691584->58695679] (req 58691584->58695679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.439553:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3205 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58691584->58695679] (req 58691584->58695679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.439567:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3205 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58691584->58695679] (req 58691584->58695679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.439777:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554597632:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7459us (7666us total) trans 4294981638 rc 0/0 00000100:00100000:3.0:1789760759.439787:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14340 00000100:00100000:3.0:1789760759.439795:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702667511245 time 702677920420 state 0x2 ref 0 00000100:00100000:3.0:1789760759.450575:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760759.458449:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981639 is committed 00000100:00100000:3.0:1789760759.462555:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760759.467584:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981640 is committed 00000001:00080000:3.0:1789760759.485315:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981641 is committed 00000100:00100000:3.0:1789760759.491746:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760759.499861:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981642 is committed 00000100:00100000:3.0:1789760759.507816:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760759.523695:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554598400 00000100:00100000:3.0:1789760759.523705:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554598400 00000100:00100000:3.0:1789760759.523759:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702737940558 time 702754239600 state 0x0 ref 1 00000100:00100000:3.0:1789760759.523774:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702754500016 time 702754239600 state 0x1 ref 1 00000100:00100000:3.0:1789760759.523807:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702754500016 time 702769425576 state 0x0 ref 1 00000100:00100000:3.0:1789760759.523818:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554598400:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760759.523875:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58716160, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0x1c010 00010000:00010000:3.0:1789760759.523884:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c322f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.523899:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760759.523904:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.523910:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c322f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58716160->58720255] (req 58716160->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.524108:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c322f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58716160->58720255] (req 58716160->58720255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760759.531082:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58716160 -> 58720256 00002000:00010000:3.0:1789760759.531090:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 114704 -> 114712 00010000:00010000:3.0:1789760759.531103:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c322f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58716160->58720255] (req 58716160->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.531117:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c322f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58716160->58720255] (req 58716160->58720255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.531126:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c322f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58716160->58720255] (req 58716160->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.531131:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760759.531135:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c322f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58716160->58720255] (req 58716160->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.531145:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c322f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58716160->58720255] (req 58716160->58720255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.531159:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c322f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58716160->58720255] (req 58716160->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.531164:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c322f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58716160->58720255] (req 58716160->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.531307:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554598400:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7485us (7715us total) trans 4294981644 rc 0/0 00000100:00100000:3.0:1789760759.531319:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14346 00000100:00100000:3.0:1789760759.531327:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702754500016 time 702769425576 state 0x2 ref 0 00000100:00100000:3.0:1789760759.536365:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760759.582892:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554598912 00000100:00100000:3.0:1789760759.582901:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554598912 00000100:00100000:3.0:1789760759.582961:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702803780803 time 702819314110 state 0x0 ref 1 00000100:00100000:3.0:1789760759.582975:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702819574526 time 702819314110 state 0x1 ref 1 00000100:00100000:3.0:1789760759.583006:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702819574526 time 702828624220 state 0x0 ref 1 00000100:00100000:3.0:1789760759.583018:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554598912:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760759.583062:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58732544, mtime: 0x6aad94f6, atime: 0x0, ctime: 0x6aad94f6, blocks: 0x1c030 00010000:00010000:3.0:1789760759.583072:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c324b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.583087:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760759.583092:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.583098:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c324b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58732544->58736639] (req 58732544->58736639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.583112:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c324b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58732544->58736639] (req 58732544->58736639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760759.589374:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58732544 -> 58736640 00002000:00010000:3.0:1789760759.589397:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 114736 -> 114744 00010000:00010000:3.0:1789760759.589410:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c324b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58732544->58736639] (req 58732544->58736639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.589423:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c324b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58732544->58736639] (req 58732544->58736639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.589433:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c324b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58732544->58736639] (req 58732544->58736639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.589441:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760759.589446:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c324b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58732544->58736639] (req 58732544->58736639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.589454:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c324b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58732544->58736639] (req 58732544->58736639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.589473:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c324b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58732544->58736639] (req 58732544->58736639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.589481:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c324b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58732544->58736639] (req 58732544->58736639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.589576:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554598912:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6554us (6776us total) trans 4294981648 rc 0/0 00000100:00100000:3.0:1789760759.589587:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14350 00000100:00100000:3.0:1789760759.589597:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702819574526 time 702828624220 state 0x2 ref 0 00000100:00100000:3.0:1789760759.592593:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760759.623952:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760759.668183:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:3.0:1789760759.675065:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58761216 -> 58765312 00002000:00010000:3.0:1789760759.675070:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 114792 -> 114800 00010000:00010000:3.0:1789760759.675084:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c327c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58761216->58765311] (req 58761216->58765311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.675097:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c327c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58761216->58765311] (req 58761216->58765311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.675105:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c327c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58761216->58765311] (req 58761216->58765311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.675118:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760759.675123:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c327c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58761216->58765311] (req 58761216->58765311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.675132:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c327c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58761216->58765311] (req 58761216->58765311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.675151:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c327c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58761216->58765311] (req 58761216->58765311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.675159:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c327c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58761216->58765311] (req 58761216->58765311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.675247:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d9c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554599808:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6148us (7070us total) trans 4294981655 rc 0/0 00000100:00100000:3.0:1789760759.675257:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14357 00000100:00100000:3.0:1789760759.675267:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702898715908 time 702914698744 state 0x2 ref 0 00000100:00100000:3.0:1789760759.703923:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760759.718209:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760759.732176:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554600320 00000100:00100000:3.0:1789760759.732186:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554600320 00000100:00100000:3.0:1789760759.732256:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702950538813 time 702964444890 state 0x0 ref 1 00000100:00100000:3.0:1789760759.732272:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702964705306 time 702964444890 state 0x1 ref 1 00000100:00100000:3.0:1789760759.732306:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702964705306 time 702977924216 state 0x0 ref 1 00000100:00100000:3.0:1789760759.732319:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06568a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554600320:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760759.732365:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58777600, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0x1c088 00010000:00010000:3.0:1789760759.732374:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3298 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.732430:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=702s v=(0 0) 00010000:00010000:3.0:1789760759.732434:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.732440:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3298 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58777600->58781695] (req 58777600->58781695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.732454:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3298 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58777600->58781695] (req 58777600->58781695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760759.737983:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58777600 -> 58781696 00002000:00010000:3.0:1789760759.737988:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 114824 -> 114832 00010000:00010000:3.0:1789760759.738001:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3298 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58777600->58781695] (req 58777600->58781695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.738014:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3298 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58777600->58781695] (req 58777600->58781695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.738022:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3298 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58777600->58781695] (req 58777600->58781695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.738033:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760759.738038:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3298 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58777600->58781695] (req 58777600->58781695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.738046:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3298 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58777600->58781695] (req 58777600->58781695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.738064:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3298 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58777600->58781695] (req 58777600->58781695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.738072:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3298 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58777600->58781695] (req 58777600->58781695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.738371:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06568a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554600320:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6049us (6541us total) trans 4294981659 rc 0/0 00000100:00100000:3.0:1789760759.738426:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14361 00000100:00100000:3.0:1789760759.738436:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 702964705306 time 702977924216 state 0x2 ref 0 00000100:00100000:3.0:1789760759.744678:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:3.0:1789760759.787638:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58789888 -> 58793984 00002000:00010000:3.0:1789760759.787643:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 114848 -> 114856 00010000:00010000:3.0:1789760759.787657:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c32ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58789888->58793983] (req 58789888->58793983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.787669:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c32ad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58789888->58793983] (req 58789888->58793983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.787678:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c32ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58789888->58793983] (req 58789888->58793983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.787685:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760759.787690:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c32ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58789888->58793983] (req 58789888->58793983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.787698:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c32ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58789888->58793983] (req 58789888->58793983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.787716:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c32ad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58789888->58793983] (req 58789888->58793983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.787724:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c32ad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58789888->58793983] (req 58789888->58793983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.788131:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06568e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554600704:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9841us (11368us total) trans 4294981662 rc 0/0 00000100:00100000:3.0:1789760759.788143:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14364 00000100:00100000:3.0:1789760759.788152:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703011277388 time 703023892432 state 0x2 ref 0 00000100:00100000:3.0:1789760759.796277:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760759.841153:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760759.850363:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981666 is committed 00000100:00100000:3.0:1789760759.877712:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554601472 00000100:00100000:3.0:1789760759.877723:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554601472 00000100:00100000:3.0:1789760759.877778:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703090571928 time 703107060767 state 0x0 ref 1 00000100:00100000:3.0:1789760759.877794:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703107321183 time 703107060767 state 0x1 ref 1 00000100:00100000:3.0:1789760759.877885:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703107321183 time 703123501681 state 0x0 ref 1 00000100:00100000:3.0:1789760759.877899:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554601472:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760759.877945:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58814464, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0x1c0d0 00010000:00010000:3.0:1789760759.877954:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c32d7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.877970:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760759.877975:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.877981:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c32d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58814464->58818559] (req 58814464->58818559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.877995:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c32d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58814464->58818559] (req 58814464->58818559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760759.883770:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58814464 -> 58818560 00002000:00010000:3.0:1789760759.883776:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 114896 -> 114904 00010000:00010000:3.0:1789760759.883789:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c32d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58814464->58818559] (req 58814464->58818559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.883803:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c32d7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58814464->58818559] (req 58814464->58818559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.883819:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c32d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58814464->58818559] (req 58814464->58818559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.883826:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760759.883832:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c32d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58814464->58818559] (req 58814464->58818559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.883841:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c32d7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58814464->58818559] (req 58814464->58818559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.883969:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c32d7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58814464->58818559] (req 58814464->58818559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.883978:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c32d7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58814464->58818559] (req 58814464->58818559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.884128:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554601472:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6227us (6584us total) trans 4294981668 rc 0/0 00000100:00100000:3.0:1789760759.884139:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14370 00000100:00100000:3.0:1789760759.884149:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703107321183 time 703123501681 state 0x2 ref 0 00000100:00100000:3.0:1789760759.897267:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760759.915908:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554601728 00000100:00100000:3.0:1789760759.915919:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554601728 00000100:00100000:3.0:1789760759.915976:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703123762097 time 703145207212 state 0x0 ref 1 00000100:00100000:3.0:1789760759.915995:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703145467628 time 703145207212 state 0x1 ref 1 00000100:00100000:3.0:1789760759.916028:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703145467628 time 703161646835 state 0x0 ref 1 00000100:00100000:3.0:1789760759.916041:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554601728:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760759.916090:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58822656, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0x1c0e0 00010000:00010000:3.0:1789760759.916100:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c32e5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.916118:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760759.916123:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.916129:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c32e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58822656->58826751] (req 58822656->58826751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.916143:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c32e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58822656->58826751] (req 58822656->58826751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760759.924226:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58822656 -> 58826752 00002000:00010000:3.0:1789760759.924232:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 114912 -> 114920 00010000:00010000:3.0:1789760759.924245:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c32e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58822656->58826751] (req 58822656->58826751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.924259:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c32e5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58822656->58826751] (req 58822656->58826751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.924268:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c32e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58822656->58826751] (req 58822656->58826751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.924276:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760759.924280:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c32e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58822656->58826751] (req 58822656->58826751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.924289:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c32e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58822656->58826751] (req 58822656->58826751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.924306:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c32e5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58822656->58826751] (req 58822656->58826751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.924314:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c32e5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58822656->58826751] (req 58822656->58826751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.924447:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554601728:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8404us (9494us total) trans 4294981670 rc 0/0 00000100:00100000:3.0:1789760759.924456:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14372 00000100:00100000:3.0:1789760759.924466:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703145467628 time 703161646835 state 0x2 ref 0 00000001:00080000:3.0:1789760759.952514:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981672 is committed 00000001:00080000:3.0:1789760759.964149:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981673 is committed 00000100:00100000:3.0:1789760759.968151:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554602240 00000100:00100000:3.0:1789760759.968161:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554602240 00000100:00100000:3.0:1789760759.968210:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703190223062 time 703204358817 state 0x0 ref 1 00000100:00100000:3.0:1789760759.968226:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703204619233 time 703204358817 state 0x1 ref 1 00000100:00100000:3.0:1789760759.968369:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703204619233 time 703213985703 state 0x0 ref 1 00000100:00100000:3.0:1789760759.968417:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554602240:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760759.968461:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58839040, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0x1c100 00010000:00010000:3.0:1789760759.968471:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3301 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.968487:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760759.968492:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.968498:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3301 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58839040->58843135] (req 58839040->58843135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.968511:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3301 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58839040->58843135] (req 58839040->58843135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760759.972789:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981674 is committed 00000100:00100000:3.0:1789760759.978805:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554602368 00000100:00100000:3.0:1789760759.978814:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554602368 00000100:00100000:3.0:1789760759.978879:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703204619233 time 703213985703 state 0x0 ref 1 00000100:00100000:3.0:1789760759.978894:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703214246119 time 703213985703 state 0x1 ref 1 00000100:00100000:3.0:1789760759.979604:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703214246119 time 703225219381 state 0x0 ref 1 00000100:00100000:3.0:1789760759.979620:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04545c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554602368:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760759.979671:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58843136, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0x1c108 00010000:00010000:3.0:1789760759.980468:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3308 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.980487:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760759.980491:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760759.980497:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3308 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58843136->58847231] (req 58843136->58847231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.980512:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3308 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58843136->58847231] (req 58843136->58847231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760759.985803:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58843136 -> 58847232 00002000:00010000:3.0:1789760759.985807:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 114952 -> 114960 00010000:00010000:3.0:1789760759.985820:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3308 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58843136->58847231] (req 58843136->58847231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.985831:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3308 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58843136->58847231] (req 58843136->58847231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.985841:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3308 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58843136->58847231] (req 58843136->58847231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.985858:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760759.985863:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3308 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58843136->58847231] (req 58843136->58847231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.985878:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3308 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58843136->58847231] (req 58843136->58847231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.985898:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3308 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58843136->58847231] (req 58843136->58847231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760759.985906:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3308 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58843136->58847231] (req 58843136->58847231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760759.987299:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04545c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554602368:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7675us (8634us total) trans 4294981675 rc 0/0 00000100:00100000:3.0:1789760759.987311:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14377 00000100:00100000:3.0:1789760759.987321:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703214246119 time 703225219381 state 0x2 ref 0 00000100:00100000:3.0:1789760759.995288:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760760.028322:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760760.033095:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981678 is committed 00000100:00100000:3.0:1789760760.038221:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760760.043897:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981679 is committed 00000100:00100000:3.0:1789760760.049034:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760760.056490:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981680 is committed 00000100:00100000:3.0:1789760760.061467:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760760.073772:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760760.084942:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760760.100877:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760760.117625:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:3.0:1789760760.126189:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58884096 -> 58888192 00002000:00010000:3.0:1789760760.126194:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 115032 -> 115040 00010000:00010000:3.0:1789760760.126206:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c334e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58884096->58888191] (req 58884096->58888191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.126272:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c334e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58884096->58888191] (req 58884096->58888191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.126284:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c334e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58884096->58888191] (req 58884096->58888191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.126291:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760760.126297:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c334e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58884096->58888191] (req 58884096->58888191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.126307:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c334e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58884096->58888191] (req 58884096->58888191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.126327:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c334e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58884096->58888191] (req 58884096->58888191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.126335:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c334e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58884096->58888191] (req 58884096->58888191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.126493:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09316680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554603648:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8532us (8875us total) trans 4294981685 rc 0/0 00000100:00100000:3.0:1789760760.126504:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14387 00000100:00100000:3.0:1789760760.126514:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703347061525 time 703363563461 state 0x2 ref 0 00000100:00100000:3.0:1789760760.131971:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760760.146018:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554603904 00000100:00100000:3.0:1789760760.146028:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554603904 00000100:00100000:3.0:1789760760.146078:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703363823877 time 703379113389 state 0x0 ref 1 00000100:00100000:3.0:1789760760.146092:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703379373805 time 703379113389 state 0x1 ref 1 00000100:00100000:3.0:1789760760.146160:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703379373805 time 703391777921 state 0x0 ref 1 00000100:00100000:3.0:1789760760.146171:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04546300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554603904:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760760.146227:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58892288, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0x1c168 00010000:00010000:3.0:1789760760.146236:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c335c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.146251:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760760.146255:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.146261:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c335c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58892288->58896383] (req 58892288->58896383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.146273:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c335c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58892288->58896383] (req 58892288->58896383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760760.155587:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981687 is committed 00000001:00080000:3.0:1789760760.168988:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981688 is committed 00000100:00100000:3.0:1789760760.173512:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760760.188747:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:3.0:1789760760.196030:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58904576 -> 58908672 00002000:00010000:3.0:1789760760.196035:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 115072 -> 115080 00010000:00010000:3.0:1789760760.196047:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c3371 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58904576->58908671] (req 58904576->58908671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.196061:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c3371 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58904576->58908671] (req 58904576->58908671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.196071:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c3371 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58904576->58908671] (req 58904576->58908671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.196078:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760760.196083:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c3371 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58904576->58908671] (req 58904576->58908671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.196093:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c3371 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58904576->58908671] (req 58904576->58908671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.196111:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c3371 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58904576->58908671] (req 58904576->58908671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.196118:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c3371 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58904576->58908671] (req 58904576->58908671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.196229:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09316300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554604288:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7198us (7479us total) trans 4294981690 rc 0/0 00000100:00100000:3.0:1789760760.196243:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14392 00000100:00100000:3.0:1789760760.196252:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703419748662 time 703434628918 state 0x2 ref 0 00000100:00100000:3.0:1789760760.204926:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554604416 00000100:00100000:3.0:1789760760.204936:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554604416 00000100:00100000:3.0:1789760760.204989:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703419748662 time 703434628918 state 0x0 ref 1 00000100:00100000:3.0:1789760760.205006:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703434889334 time 703434628918 state 0x1 ref 1 00000100:00100000:3.0:1789760760.205104:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703434889334 time 703450712295 state 0x0 ref 1 00000100:00100000:3.0:1789760760.205118:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d81500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554604416:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760760.205162:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58908672, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0x1c188 00010000:00010000:3.0:1789760760.205172:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3378 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.205188:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760760.205192:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.205199:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3378 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58908672->58912767] (req 58908672->58912767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.205222:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3378 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58908672->58912767] (req 58908672->58912767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760760.245595:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981693 is committed 00000100:00100000:3.0:1789760760.254553:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554604800 00000100:00100000:3.0:1789760760.254563:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554604800 00000100:00100000:3.0:1789760760.254615:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703463934104 time 703480729828 state 0x0 ref 1 00000100:00100000:3.0:1789760760.254630:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703480990244 time 703480729828 state 0x1 ref 1 00000100:00100000:3.0:1789760760.254692:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703480990244 time 703500310277 state 0x0 ref 1 00000100:00100000:3.0:1789760760.254704:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d83100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554604800:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760760.254749:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58920960, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0x1c1a0 00010000:00010000:3.0:1789760760.254758:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c338d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.254774:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760760.254778:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.254785:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c338d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58920960->58925055] (req 58920960->58925055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.254798:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c338d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58920960->58925055] (req 58920960->58925055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760760.260167:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58920960 -> 58925056 00002000:00010000:3.0:1789760760.260172:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 115104 -> 115112 00010000:00010000:3.0:1789760760.260185:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c338d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58920960->58925055] (req 58920960->58925055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.260197:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c338d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58920960->58925055] (req 58920960->58925055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.260211:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c338d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58920960->58925055] (req 58920960->58925055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.260228:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760760.260233:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c338d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58920960->58925055] (req 58920960->58925055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.260244:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c338d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58920960->58925055] (req 58920960->58925055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.260262:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c338d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58920960->58925055] (req 58920960->58925055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.260270:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c338d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58920960->58925055] (req 58920960->58925055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.260361:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d83100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554604800:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5654us (6053us total) trans 4294981694 rc 0/0 00000100:00100000:3.0:1789760760.260371:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14396 00000100:00100000:3.0:1789760760.260881:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703480990244 time 703500310277 state 0x2 ref 0 00000100:00100000:3.0:1789760760.278984:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554605056 00000100:00100000:3.0:1789760760.278996:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554605056 00000100:00100000:3.0:1789760760.279056:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703500570693 time 703510258060 state 0x0 ref 1 00000100:00100000:3.0:1789760760.279073:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703510518476 time 703510258060 state 0x1 ref 1 00000100:00100000:3.0:1789760760.279170:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703510518476 time 703524787275 state 0x0 ref 1 00000100:00100000:3.0:1789760760.279184:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04546680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554605056:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760760.279228:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58929152, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0x1c1b0 00010000:00010000:3.0:1789760760.279238:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c339b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.279253:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760760.279257:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.279263:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c339b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58929152->58933247] (req 58929152->58933247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.279277:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c339b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58929152->58933247] (req 58929152->58933247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760760.284708:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58929152 -> 58933248 00002000:00010000:3.0:1789760760.284714:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 115120 -> 115128 00010000:00010000:3.0:1789760760.284727:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c339b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58929152->58933247] (req 58929152->58933247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.284740:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c339b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58929152->58933247] (req 58929152->58933247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.284749:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c339b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58929152->58933247] (req 58929152->58933247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.284764:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760760.284769:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c339b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58929152->58933247] (req 58929152->58933247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.284778:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c339b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58929152->58933247] (req 58929152->58933247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.284796:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c339b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58929152->58933247] (req 58929152->58933247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.284803:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c339b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58929152->58933247] (req 58929152->58933247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.284904:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04546680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554605056:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5719us (7866us total) trans 4294981696 rc 0/0 00000100:00100000:3.0:1789760760.284914:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14398 00000100:00100000:3.0:1789760760.284924:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703510518476 time 703524787275 state 0x2 ref 0 00000001:00080000:3.0:1789760760.297854:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981697 is committed 00000001:00080000:3.0:1789760760.311913:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981698 is committed 00000100:00100000:3.0:1789760760.343296:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554605568 00000100:00100000:3.0:1789760760.343305:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554605568 00000100:00100000:3.0:1789760760.343360:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703551521673 time 703568964449 state 0x0 ref 1 00000100:00100000:3.0:1789760760.343421:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703569224865 time 703568964449 state 0x1 ref 1 00000100:00100000:3.0:1789760760.343457:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703569224865 time 703589075094 state 0x0 ref 1 00000100:00100000:3.0:1789760760.343469:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d80700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554605568:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760760.343530:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58945536, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0x1c1d0 00010000:00010000:3.0:1789760760.343540:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c33b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.343555:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760760.343559:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.343566:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c33b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58945536->58949631] (req 58945536->58949631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.343578:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c33b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58945536->58949631] (req 58945536->58949631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760760.349009:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58945536 -> 58949632 00002000:00010000:3.0:1789760760.349014:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 115152 -> 115160 00010000:00010000:3.0:1789760760.349027:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c33b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58945536->58949631] (req 58945536->58949631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.349040:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c33b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58945536->58949631] (req 58945536->58949631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.349049:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c33b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58945536->58949631] (req 58945536->58949631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.349062:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760760.349067:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c33b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58945536->58949631] (req 58945536->58949631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.349077:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c33b7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58945536->58949631] (req 58945536->58949631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.349097:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c33b7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58945536->58949631] (req 58945536->58949631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.349104:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c33b7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58945536->58949631] (req 58945536->58949631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.349187:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d80700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554605568:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5715us (5982us total) trans 4294981700 rc 0/0 00000100:00100000:3.0:1789760760.349197:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14402 00000100:00100000:3.0:1789760760.349205:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703569224865 time 703589075094 state 0x2 ref 0 00000001:00080000:3.0:1789760760.362952:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981701 is committed 00000001:00080000:3.0:1789760760.385076:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981702 is committed 00000100:00100000:3.0:1789760760.403754:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554606080 00000100:00100000:3.0:1789760760.403763:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554606080 00000100:00100000:3.0:1789760760.403817:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703623978295 time 703637668045 state 0x0 ref 1 00000100:00100000:3.0:1789760760.403832:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703637928461 time 703637668045 state 0x1 ref 1 00000100:00100000:3.0:1789760760.403871:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703637928461 time 703649489262 state 0x0 ref 1 00000100:00100000:3.0:1789760760.403883:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04547800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554606080:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760760.403934:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58961920, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0x1c1f0 00010000:00010000:3.0:1789760760.403941:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c33d3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.403957:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760760.403961:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.403967:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c33d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58961920->58966015] (req 58961920->58966015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.403980:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c33d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58961920->58966015] (req 58961920->58966015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760760.408659:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58961920 -> 58966016 00002000:00010000:3.0:1789760760.408664:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 115184 -> 115192 00010000:00010000:3.0:1789760760.408678:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c33d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58961920->58966015] (req 58961920->58966015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.408691:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c33d3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58961920->58966015] (req 58961920->58966015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.408700:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c33d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58961920->58966015] (req 58961920->58966015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.408711:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760760.408716:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c33d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58961920->58966015] (req 58961920->58966015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.408724:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c33d3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58961920->58966015] (req 58961920->58966015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.408743:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c33d3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58961920->58966015] (req 58961920->58966015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.408751:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c33d3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58961920->58966015] (req 58961920->58966015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.408856:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04547800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554606080:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4959us (5173us total) trans 4294981704 rc 0/0 00000100:00100000:3.0:1789760760.408867:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14406 00000100:00100000:3.0:1789760760.408878:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703637928461 time 703649489262 state 0x2 ref 0 00000100:00100000:3.0:1789760760.413526:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554606208 00000100:00100000:3.0:1789760760.413535:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554606208 00000100:00100000:3.0:1789760760.413588:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703637928461 time 703649489262 state 0x0 ref 1 00000100:00100000:3.0:1789760760.413613:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703649749678 time 703649489262 state 0x1 ref 1 00000100:00100000:3.0:1789760760.413643:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703649749678 time 703659261832 state 0x0 ref 1 00000100:00100000:3.0:1789760760.413658:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04545500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554606208:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760760.413714:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58966016, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0x1c1f8 00010000:00010000:3.0:1789760760.413722:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c33da lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.413737:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760760.413741:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.413747:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c33da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58966016->58970111] (req 58966016->58970111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.413761:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c33da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58966016->58970111] (req 58966016->58970111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760760.419819:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58966016 -> 58970112 00002000:00010000:3.0:1789760760.419826:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 115192 -> 115200 00010000:00010000:3.0:1789760760.419840:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c33da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58966016->58970111] (req 58966016->58970111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.419861:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c33da lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58966016->58970111] (req 58966016->58970111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.419869:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c33da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58966016->58970111] (req 58966016->58970111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.419876:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760760.419881:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c33da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58966016->58970111] (req 58966016->58970111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.419890:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c33da lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58966016->58970111] (req 58966016->58970111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.419920:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c33da lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58966016->58970111] (req 58966016->58970111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.419927:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c33da lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58966016->58970111] (req 58966016->58970111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.420008:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04545500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554606208:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6346us (6661us total) trans 4294981705 rc 0/0 00000100:00100000:3.0:1789760760.420019:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14407 00000100:00100000:3.0:1789760760.420028:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703649749678 time 703659261832 state 0x2 ref 0 00000100:00100000:3.0:1789760760.428016:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554606336 00000100:00100000:3.0:1789760760.428026:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554606336 00000100:00100000:3.0:1789760760.428081:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703649749678 time 703659261832 state 0x0 ref 1 00000100:00100000:3.0:1789760760.428102:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703659522248 time 703659261832 state 0x1 ref 1 00000100:00100000:3.0:1789760760.428135:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703659522248 time 703673753373 state 0x0 ref 1 00000100:00100000:3.0:1789760760.428147:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554606336:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760760.428194:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58970112, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0x1c200 00010000:00010000:3.0:1789760760.428205:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c33e1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.428220:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760760.428232:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.428239:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c33e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58970112->58974207] (req 58970112->58974207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.428252:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c33e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58970112->58974207] (req 58970112->58974207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760760.445552:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981707 is committed 00000100:00100000:3.0:1789760760.453875:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554606592 00000100:00100000:3.0:1789760760.453884:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554606592 00000100:00100000:3.0:1789760760.453937:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703674013789 time 703686764621 state 0x0 ref 1 00000100:00100000:3.0:1789760760.453953:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703687025037 time 703686764621 state 0x1 ref 1 00000100:00100000:3.0:1789760760.454016:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703687025037 time 703699634170 state 0x0 ref 1 00000100:00100000:3.0:1789760760.454030:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04545f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554606592:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760760.454075:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58978304, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0x1c210 00010000:00010000:3.0:1789760760.454085:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c33ef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.454100:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760760.454104:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.454113:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c33ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58978304->58982399] (req 58978304->58982399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.454126:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c33ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58978304->58982399] (req 58978304->58982399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.463319:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554606720 00000100:00100000:3.0:1789760760.463329:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554606720 00000100:00100000:3.0:1789760760.463545:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703687025037 time 703699634170 state 0x0 ref 1 00000100:00100000:3.0:1789760760.463563:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703699894586 time 703699634170 state 0x1 ref 1 00000100:00100000:3.0:1789760760.463702:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703699894586 time 703709319187 state 0x0 ref 1 00000100:00100000:3.0:1789760760.463718:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554606720:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760760.463767:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58982400, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0x1c218 00010000:00010000:3.0:1789760760.463776:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c33f6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.463791:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760760.463795:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.463800:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c33f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58982400->58986495] (req 58982400->58986495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.463813:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c33f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58982400->58986495] (req 58982400->58986495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760760.468593:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58982400 -> 58986496 00002000:00010000:3.0:1789760760.468733:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 115224 -> 115232 00010000:00010000:3.0:1789760760.468752:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c33f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58982400->58986495] (req 58982400->58986495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.468764:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c33f6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58982400->58986495] (req 58982400->58986495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.468773:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c33f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58982400->58986495] (req 58982400->58986495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.468780:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760760.468784:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c33f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58982400->58986495] (req 58982400->58986495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.468792:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c33f6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58982400->58986495] (req 58982400->58986495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.468811:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c33f6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58982400->58986495] (req 58982400->58986495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.468818:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c33f6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58982400->58986495] (req 58982400->58986495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.468957:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554606720:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5237us (5791us total) trans 4294981709 rc 0/0 00000100:00100000:3.0:1789760760.468968:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14411 00000100:00100000:3.0:1789760760.468977:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703699894586 time 703709319187 state 0x2 ref 0 00000001:00080000:3.0:1789760760.499980:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981712 is committed 00000100:00100000:3.0:1789760760.507139:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554607232 00000100:00100000:3.0:1789760760.507149:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554607232 00000100:00100000:3.0:1789760760.507202:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703731985820 time 703741393668 state 0x0 ref 1 00000100:00100000:3.0:1789760760.507222:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703741654084 time 703741393668 state 0x1 ref 1 00000100:00100000:3.0:1789760760.507262:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703741654084 time 703752880515 state 0x0 ref 1 00000100:00100000:3.0:1789760760.507275:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554607232:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760760.507326:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 58998784, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0x1c238 00010000:00010000:3.0:1789760760.507335:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3412 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.507354:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760760.507359:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.507365:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3412 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58998784->59002879] (req 58998784->59002879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.507419:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3412 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58998784->59002879] (req 58998784->59002879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760760.518973:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 58998784 -> 59002880 00002000:00010000:3.0:1789760760.518978:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 115256 -> 115264 00010000:00010000:3.0:1789760760.518991:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3412 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58998784->59002879] (req 58998784->59002879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.519004:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3412 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58998784->59002879] (req 58998784->59002879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.519013:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3412 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58998784->59002879] (req 58998784->59002879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.519020:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760760.519025:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3412 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58998784->59002879] (req 58998784->59002879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.519033:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3412 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58998784->59002879] (req 58998784->59002879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.519053:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3412 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58998784->59002879] (req 58998784->59002879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.519061:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3412 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [58998784->59002879] (req 58998784->59002879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.519196:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554607232:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11917us (12150us total) trans 4294981713 rc 0/0 00000100:00100000:3.0:1789760760.519207:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14415 00000100:00100000:3.0:1789760760.519217:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703741654084 time 703752880515 state 0x2 ref 0 00000100:00100000:3.0:1789760760.523084:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760760.558931:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981716 is committed 00000100:00100000:3.0:1789760760.565115:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760760.573898:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981717 is committed 00000001:00080000:3.0:1789760760.599215:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981719 is committed 00000100:00100000:3.0:1789760760.604158:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760760.610866:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981720 is committed 00000100:00100000:3.0:1789760760.618167:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554608256 00000100:00100000:3.0:1789760760.618176:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554608256 00000100:00100000:3.0:1789760760.618238:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703837578041 time 703850186493 state 0x0 ref 1 00000100:00100000:3.0:1789760760.618253:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703850446909 time 703850186493 state 0x1 ref 1 00000100:00100000:3.0:1789760760.618331:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703850446909 time 703863948347 state 0x0 ref 1 00000100:00100000:3.0:1789760760.618343:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554608256:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760760.618429:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59031552, mtime: 0x6aad94f7, atime: 0x0, ctime: 0x6aad94f7, blocks: 0x1c278 00010000:00010000:3.0:1789760760.618438:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c344a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.618453:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760760.618457:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.618463:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c344a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59031552->59035647] (req 59031552->59035647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.618476:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c344a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59031552->59035647] (req 59031552->59035647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760760.623555:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59031552 -> 59035648 00002000:00010000:3.0:1789760760.623560:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 115320 -> 115328 00010000:00010000:3.0:1789760760.623573:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c344a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59031552->59035647] (req 59031552->59035647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.623585:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c344a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59031552->59035647] (req 59031552->59035647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.623594:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c344a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59031552->59035647] (req 59031552->59035647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.623675:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760760.623680:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c344a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59031552->59035647] (req 59031552->59035647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.623689:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c344a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59031552->59035647] (req 59031552->59035647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.623710:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c344a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59031552->59035647] (req 59031552->59035647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.623717:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c344a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59031552->59035647] (req 59031552->59035647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.623980:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554608256:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5633us (6040us total) trans 4294981721 rc 0/0 00000100:00100000:3.0:1789760760.623992:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14423 00000100:00100000:3.0:1789760760.624002:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703850446909 time 703863948347 state 0x2 ref 0 00000001:00080000:3.0:1789760760.636481:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981722 is committed 00000001:00080000:3.0:1789760760.646979:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981723 is committed 00000001:00080000:3.0:1789760760.665923:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981724 is committed 00000100:00100000:3.0:1789760760.674747:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554608768 00000100:00100000:3.0:1789760760.674757:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554608768 00000100:00100000:3.0:1789760760.674807:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703888170083 time 703906583884 state 0x0 ref 1 00000100:00100000:3.0:1789760760.674821:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703906844300 time 703906583884 state 0x1 ref 1 00000100:00100000:3.0:1789760760.674860:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 703906844300 time 703920478132 state 0x0 ref 1 00000100:00100000:3.0:1789760760.674872:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554608768:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760760.674916:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59047936, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0x1c298 00010000:00010000:3.0:1789760760.674926:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3466 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.674942:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=703s v=(0 0) 00010000:00010000:3.0:1789760760.674946:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.674953:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3466 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59047936->59052031] (req 59047936->59052031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.674966:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3466 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59047936->59052031] (req 59047936->59052031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760760.681937:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981725 is committed 00000001:00080000:3.0:1789760760.693532:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981726 is committed 00000001:00080000:3.0:1789760760.712507:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981727 is committed 00000001:00080000:3.0:1789760760.769788:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981731 is committed 00000001:00080000:3.0:1789760760.781649:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981732 is committed 00000001:00080000:3.0:1789760760.797174:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981733 is committed 00002000:00010000:3.0:1789760760.908700:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59109376 -> 59113472 00002000:00010000:3.0:1789760760.908706:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 115472 -> 115480 00010000:00010000:3.0:1789760760.908722:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c34cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59109376->59113471] (req 59109376->59113471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.908738:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c34cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59109376->59113471] (req 59109376->59113471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.908747:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c34cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59109376->59113471] (req 59109376->59113471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.908753:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760760.908758:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c34cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59109376->59113471] (req 59109376->59113471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.908767:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c34cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59109376->59113471] (req 59109376->59113471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.908788:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c34cf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59109376->59113471] (req 59109376->59113471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.908802:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c34cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59109376->59113471] (req 59109376->59113471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.909813:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554610688:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10701us (10900us total) trans 4294981740 rc 0/0 00000100:00100000:3.0:1789760760.909827:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14442 00000100:00100000:3.0:1789760760.909836:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704132487807 time 704144711876 state 0x2 ref 0 00000100:00100000:3.0:1789760760.914242:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760760.944502:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554611200 00000100:00100000:3.0:1789760760.944512:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554611200 00000100:00100000:3.0:1789760760.944564:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704172208432 time 704181394041 state 0x0 ref 1 00000100:00100000:3.0:1789760760.944578:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704181654457 time 704181394041 state 0x1 ref 1 00000100:00100000:3.0:1789760760.944616:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704181654457 time 704190234060 state 0x0 ref 1 00000100:00100000:3.0:1789760760.944626:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554611200:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760760.944669:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59125760, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0x1c330 00010000:00010000:3.0:1789760760.944677:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c34eb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.944692:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760760.944696:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.944702:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c34eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59125760->59129855] (req 59125760->59129855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.944714:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c34eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59125760->59129855] (req 59125760->59129855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760760.952508:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59125760 -> 59129856 00002000:00010000:3.0:1789760760.952513:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 115504 -> 115512 00010000:00010000:3.0:1789760760.952527:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c34eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59125760->59129855] (req 59125760->59129855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.952539:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c34eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59125760->59129855] (req 59125760->59129855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.952549:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c34eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59125760->59129855] (req 59125760->59129855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.952557:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760760.952562:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c34eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59125760->59129855] (req 59125760->59129855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.952571:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c34eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59125760->59129855] (req 59125760->59129855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.952591:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c34eb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59125760->59129855] (req 59125760->59129855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.952599:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c34eb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59125760->59129855] (req 59125760->59129855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.952693:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554611200:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8063us (8419us total) trans 4294981744 rc 0/0 00000100:00100000:3.0:1789760760.952704:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14446 00000100:00100000:3.0:1789760760.952713:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704181654457 time 704190234060 state 0x2 ref 0 00002000:00010000:3.0:1789760760.975337:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59133952 -> 59138048 00002000:00010000:3.0:1789760760.975343:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 115520 -> 115528 00010000:00010000:3.0:1789760760.975356:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c34f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59133952->59138047] (req 59133952->59138047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.975370:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c34f9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59133952->59138047] (req 59133952->59138047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.975399:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c34f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59133952->59138047] (req 59133952->59138047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.975407:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760760.975412:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c34f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59133952->59138047] (req 59133952->59138047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.975422:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c34f9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59133952->59138047] (req 59133952->59138047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.975441:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c34f9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59133952->59138047] (req 59133952->59138047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.975455:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c34f9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59133952->59138047] (req 59133952->59138047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760760.975555:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554611456:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4939us (5247us total) trans 4294981746 rc 0/0 00000100:00100000:3.0:1789760760.975566:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14448 00000100:00100000:3.0:1789760760.975575:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704205907183 time 704216210228 state 0x2 ref 0 00000100:00100000:3.0:1789760760.994798:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554611712 00000100:00100000:3.0:1789760760.994808:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554611712 00000100:00100000:3.0:1789760760.994868:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704216470644 time 704228766601 state 0x0 ref 1 00000100:00100000:3.0:1789760760.994883:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704229027017 time 704228766601 state 0x1 ref 1 00000100:00100000:3.0:1789760760.994952:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704229027017 time 704240569121 state 0x0 ref 1 00000100:00100000:3.0:1789760760.994965:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554611712:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760760.995010:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59142144, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0x1c350 00010000:00010000:3.0:1789760760.995019:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3507 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.995035:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760760.995039:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760760.995045:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3507 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59142144->59146239] (req 59142144->59146239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760760.995063:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3507 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59142144->59146239] (req 59142144->59146239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760761.000457:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59142144 -> 59146240 00002000:00010000:3.0:1789760761.000462:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 115536 -> 115544 00010000:00010000:3.0:1789760761.000476:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3507 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59142144->59146239] (req 59142144->59146239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.000488:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3507 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59142144->59146239] (req 59142144->59146239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.000497:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3507 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59142144->59146239] (req 59142144->59146239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.000505:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760761.000510:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3507 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59142144->59146239] (req 59142144->59146239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.000519:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3507 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59142144->59146239] (req 59142144->59146239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.000537:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3507 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59142144->59146239] (req 59142144->59146239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.000544:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3507 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59142144->59146239] (req 59142144->59146239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.001010:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554611712:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6041us (6522us total) trans 4294981748 rc 0/0 00000100:00100000:3.0:1789760761.001020:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14450 00000100:00100000:3.0:1789760761.001030:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704229027017 time 704240569121 state 0x2 ref 0 00002000:00010000:3.0:1789760761.021313:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59150336 -> 59154432 00002000:00010000:3.0:1789760761.021319:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 115552 -> 115560 00010000:00010000:3.0:1789760761.021331:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c3515 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59150336->59154431] (req 59150336->59154431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.021344:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c3515 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59150336->59154431] (req 59150336->59154431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.021353:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c3515 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59150336->59154431] (req 59150336->59154431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.021360:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760761.021365:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c3515 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59150336->59154431] (req 59150336->59154431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.021373:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c3515 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59150336->59154431] (req 59150336->59154431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.021425:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c3515 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59150336->59154431] (req 59150336->59154431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.021438:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c3515 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59150336->59154431] (req 59150336->59154431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.021563:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554611968:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7655us (7850us total) trans 4294981750 rc 0/0 00000100:00100000:3.0:1789760761.021574:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14452 00000100:00100000:3.0:1789760761.021583:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704249891591 time 704259512440 state 0x2 ref 0 00000100:00100000:3.0:1789760761.043321:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554612224 00000100:00100000:3.0:1789760761.043330:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554612224 00000100:00100000:3.0:1789760761.043433:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704259772856 time 704271685329 state 0x0 ref 1 00000100:00100000:3.0:1789760761.043447:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704271945745 time 704271685329 state 0x1 ref 1 00000100:00100000:3.0:1789760761.043477:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704271945745 time 704289096022 state 0x0 ref 1 00000100:00100000:3.0:1789760761.043489:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554612224:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760761.043535:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59158528, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0x1c370 00010000:00010000:3.0:1789760761.043544:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3523 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.043557:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760761.043561:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.043566:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3523 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59158528->59162623] (req 59158528->59162623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.043582:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3523 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59158528->59162623] (req 59158528->59162623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760761.049032:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59158528 -> 59162624 00002000:00010000:3.0:1789760761.049037:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 115568 -> 115576 00010000:00010000:3.0:1789760761.049051:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3523 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59158528->59162623] (req 59158528->59162623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.049063:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3523 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59158528->59162623] (req 59158528->59162623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.049071:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3523 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59158528->59162623] (req 59158528->59162623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.049078:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760761.049082:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3523 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59158528->59162623] (req 59158528->59162623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.049091:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3523 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59158528->59162623] (req 59158528->59162623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.049110:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3523 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59158528->59162623] (req 59158528->59162623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.049116:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3523 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59158528->59162623] (req 59158528->59162623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.049500:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554612224:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6005us (6283us total) trans 4294981752 rc 0/0 00000100:00100000:3.0:1789760761.049512:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14454 00000100:00100000:3.0:1789760761.049521:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704271945745 time 704289096022 state 0x2 ref 0 00000100:00100000:3.0:1789760761.158621:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554612992 00000100:00100000:3.0:1789760761.158634:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554612992 00000100:00100000:3.0:1789760761.158694:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704356509987 time 704383357479 state 0x0 ref 1 00000100:00100000:3.0:1789760761.158709:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704383617895 time 704383357479 state 0x1 ref 1 00000100:00100000:3.0:1789760761.158800:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704383617895 time 704404417067 state 0x0 ref 1 00000100:00100000:3.0:1789760761.158814:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d80e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554612992:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760761.158876:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59183104, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0x1c3a0 00010000:00010000:3.0:1789760761.158886:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c354d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.158902:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760761.158906:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.158912:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c354d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59183104->59187199] (req 59183104->59187199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.158926:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c354d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59183104->59187199] (req 59183104->59187199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760761.166263:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59183104 -> 59187200 00002000:00010000:3.0:1789760761.166273:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 115616 -> 115624 00010000:00010000:3.0:1789760761.166287:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c354d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59183104->59187199] (req 59183104->59187199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.166298:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c354d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59183104->59187199] (req 59183104->59187199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.166306:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c354d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59183104->59187199] (req 59183104->59187199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.166312:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760761.166317:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c354d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59183104->59187199] (req 59183104->59187199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.166325:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c354d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59183104->59187199] (req 59183104->59187199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.166345:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c354d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59183104->59187199] (req 59183104->59187199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.166352:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c354d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59183104->59187199] (req 59183104->59187199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.166490:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d80e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554612992:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7675us (12005us total) trans 4294981758 rc 0/0 00000100:00100000:3.0:1789760761.166500:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14460 00000100:00100000:3.0:1789760761.166510:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704383617895 time 704404417067 state 0x2 ref 0 00000100:00100000:3.0:1789760761.228488:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554613504 00000100:00100000:3.0:1789760761.228498:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554613504 00000100:00100000:3.0:1789760761.228546:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704436891452 time 704452570922 state 0x0 ref 1 00000100:00100000:3.0:1789760761.228560:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704452831338 time 704452570922 state 0x1 ref 1 00000100:00100000:3.0:1789760761.228624:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704452831338 time 704474242381 state 0x0 ref 1 00000100:00100000:3.0:1789760761.228636:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554613504:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760761.228676:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59199488, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0x1c3c0 00010000:00010000:3.0:1789760761.228685:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3569 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.228710:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760761.228715:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.228720:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3569 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59199488->59203583] (req 59199488->59203583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.228733:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3569 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59199488->59203583] (req 59199488->59203583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760761.242327:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981762 is committed 00000001:00080000:3.0:1789760761.255536:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981763 is committed 00000001:00080000:3.0:1789760761.274975:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981764 is committed 00000100:00100000:3.0:1789760761.302800:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554614016 00000100:00100000:3.0:1789760761.302809:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554614016 00000100:00100000:3.0:1789760761.302872:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704514622719 time 704529169333 state 0x0 ref 1 00000100:00100000:3.0:1789760761.302887:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704529429749 time 704529169333 state 0x1 ref 1 00000100:00100000:3.0:1789760761.302919:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704529429749 time 704548538007 state 0x0 ref 1 00000100:00100000:3.0:1789760761.302932:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554614016:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760761.302977:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59215872, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0x1c3e0 00010000:00010000:3.0:1789760761.302987:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3585 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.303002:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760761.303007:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.303013:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3585 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59215872->59219967] (req 59215872->59219967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.303039:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3585 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59215872->59219967] (req 59215872->59219967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760761.311075:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59215872 -> 59219968 00002000:00010000:3.0:1789760761.311080:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 115680 -> 115688 00010000:00010000:3.0:1789760761.311094:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3585 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59215872->59219967] (req 59215872->59219967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.311112:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3585 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59215872->59219967] (req 59215872->59219967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.311120:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3585 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59215872->59219967] (req 59215872->59219967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.311127:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760761.311132:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3585 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59215872->59219967] (req 59215872->59219967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.311141:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3585 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59215872->59219967] (req 59215872->59219967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.311161:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3585 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59215872->59219967] (req 59215872->59219967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.311169:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3585 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59215872->59219967] (req 59215872->59219967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.311313:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554614016:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8379us (8649us total) trans 4294981766 rc 0/0 00000100:00100000:3.0:1789760761.311324:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14468 00000100:00100000:3.0:1789760761.311333:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704529429749 time 704548538007 state 0x2 ref 0 00000001:00080000:3.0:1789760761.328935:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981767 is committed 00000001:00080000:3.0:1789760761.343181:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981768 is committed 00000001:00080000:3.0:1789760761.357693:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981769 is committed 00000100:00100000:3.0:1789760761.367754:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554614528 00000100:00100000:3.0:1789760761.367763:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554614528 00000100:00100000:3.0:1789760761.367815:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704581709519 time 704593958071 state 0x0 ref 1 00000100:00100000:3.0:1789760761.367829:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704594218487 time 704593958071 state 0x1 ref 1 00000100:00100000:3.0:1789760761.368037:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704594218487 time 704613539624 state 0x0 ref 1 00000100:00100000:3.0:1789760761.368057:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554614528:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760761.368100:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59232256, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0x1c400 00010000:00010000:3.0:1789760761.368110:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35a1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.368125:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760761.368130:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.368136:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59232256->59236351] (req 59232256->59236351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.368163:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59232256->59236351] (req 59232256->59236351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760761.374551:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59232256 -> 59236352 00002000:00010000:3.0:1789760761.374556:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 115712 -> 115720 00010000:00010000:3.0:1789760761.374570:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59232256->59236351] (req 59232256->59236351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.374589:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35a1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59232256->59236351] (req 59232256->59236351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.374609:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59232256->59236351] (req 59232256->59236351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.374616:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760761.374621:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59232256->59236351] (req 59232256->59236351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.374631:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35a1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59232256->59236351] (req 59232256->59236351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.374655:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35a1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59232256->59236351] (req 59232256->59236351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.374663:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35a1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59232256->59236351] (req 59232256->59236351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.374749:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554614528:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6692us (7195us total) trans 4294981770 rc 0/0 00000100:00100000:3.0:1789760761.374766:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14472 00000100:00100000:3.0:1789760761.374776:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704594218487 time 704613539624 state 0x2 ref 0 00000100:00100000:3.0:1789760761.427450:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554615040 00000100:00100000:3.0:1789760761.427460:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554615040 00000100:00100000:3.0:1789760761.427514:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704644338773 time 704656677674 state 0x0 ref 1 00000100:00100000:3.0:1789760761.427531:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704656938090 time 704656677674 state 0x1 ref 1 00000100:00100000:3.0:1789760761.427564:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704656938090 time 704673182689 state 0x0 ref 1 00000100:00100000:3.0:1789760761.427576:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554615040:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760761.427640:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59248640, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0x1c420 00010000:00010000:3.0:1789760761.427650:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35bd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.427667:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760761.427672:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.427680:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59248640->59252735] (req 59248640->59252735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.427693:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59248640->59252735] (req 59248640->59252735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760761.434232:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59248640 -> 59252736 00002000:00010000:3.0:1789760761.434238:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 115744 -> 115752 00010000:00010000:3.0:1789760761.434251:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59248640->59252735] (req 59248640->59252735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.434264:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35bd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59248640->59252735] (req 59248640->59252735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.434280:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59248640->59252735] (req 59248640->59252735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.434288:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760761.434293:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59248640->59252735] (req 59248640->59252735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.434303:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59248640->59252735] (req 59248640->59252735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.434333:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35bd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59248640->59252735] (req 59248640->59252735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.434341:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35bd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59248640->59252735] (req 59248640->59252735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.434470:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554615040:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6882us (7138us total) trans 4294981774 rc 0/0 00000100:00100000:3.0:1789760761.434481:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14476 00000100:00100000:3.0:1789760761.434491:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704656938090 time 704673182689 state 0x2 ref 0 00000100:00100000:3.0:1789760761.490827:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554615552 00000100:00100000:3.0:1789760761.490845:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554615552 00000100:00100000:3.0:1789760761.490901:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704702802537 time 704716429817 state 0x0 ref 1 00000100:00100000:3.0:1789760761.490915:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704716690233 time 704716429817 state 0x1 ref 1 00000100:00100000:3.0:1789760761.490991:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704716690233 time 704736608560 state 0x0 ref 1 00000100:00100000:3.0:1789760761.491004:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554615552:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760761.491063:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59265024, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0x1c440 00010000:00010000:3.0:1789760761.491073:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35d9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.491089:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760761.491093:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.491100:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59265024->59269119] (req 59265024->59269119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.491115:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59265024->59269119] (req 59265024->59269119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760761.497039:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59265024 -> 59269120 00002000:00010000:3.0:1789760761.497045:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 115776 -> 115784 00010000:00010000:3.0:1789760761.497059:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59265024->59269119] (req 59265024->59269119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.497074:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35d9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59265024->59269119] (req 59265024->59269119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.497084:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59265024->59269119] (req 59265024->59269119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.497098:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760761.497103:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59265024->59269119] (req 59265024->59269119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.497114:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35d9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59265024->59269119] (req 59265024->59269119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.497132:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35d9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59265024->59269119] (req 59265024->59269119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.497141:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35d9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59265024->59269119] (req 59265024->59269119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.497483:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554615552:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6475us (7063us total) trans 4294981778 rc 0/0 00000100:00100000:3.0:1789760761.497495:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14480 00000100:00100000:3.0:1789760761.497505:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704716690233 time 704736608560 state 0x2 ref 0 00002000:00010000:3.0:1789760761.538577:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59277312 -> 59281408 00002000:00010000:3.0:1789760761.538583:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 115800 -> 115808 00010000:00010000:3.0:1789760761.538607:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c35ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59277312->59281407] (req 59277312->59281407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.538621:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c35ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59277312->59281407] (req 59277312->59281407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.538637:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c35ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59277312->59281407] (req 59277312->59281407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.538644:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760761.538649:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c35ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59277312->59281407] (req 59277312->59281407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.538660:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c35ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59277312->59281407] (req 59277312->59281407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.538681:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c35ee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59277312->59281407] (req 59277312->59281407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.538690:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c35ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59277312->59281407] (req 59277312->59281407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.538782:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554615936:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12266us (12565us total) trans 4294981781 rc 0/0 00000100:00100000:3.0:1789760761.538793:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14483 00000100:00100000:3.0:1789760761.538802:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704757385063 time 704772119252 state 0x2 ref 0 00000100:00100000:3.0:1789760761.543775:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554616064 00000100:00100000:3.0:1789760761.543784:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554616064 00000100:00100000:3.0:1789760761.543833:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704757385063 time 704772119252 state 0x0 ref 1 00000100:00100000:3.0:1789760761.543859:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704772379668 time 704772119252 state 0x1 ref 1 00000100:00100000:3.0:1789760761.543890:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704772379668 time 704789508268 state 0x0 ref 1 00000100:00100000:3.0:1789760761.543901:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09314a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554616064:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760761.543955:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59281408, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0x1c460 00010000:00010000:3.0:1789760761.543965:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35f5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.543981:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760761.543986:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.543992:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59281408->59285503] (req 59281408->59285503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.544006:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59281408->59285503] (req 59281408->59285503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760761.553919:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59281408 -> 59285504 00002000:00010000:3.0:1789760761.553924:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 115808 -> 115816 00010000:00010000:3.0:1789760761.553937:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59281408->59285503] (req 59281408->59285503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.553950:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59281408->59285503] (req 59281408->59285503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.553959:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59281408->59285503] (req 59281408->59285503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.553971:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760761.553975:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59281408->59285503] (req 59281408->59285503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.553985:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59281408->59285503] (req 59281408->59285503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.554005:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35f5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59281408->59285503] (req 59281408->59285503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.554013:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c35f5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59281408->59285503] (req 59281408->59285503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.554136:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09314a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554616064:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10231us (10436us total) trans 4294981782 rc 0/0 00000100:00100000:3.0:1789760761.554148:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14484 00000100:00100000:3.0:1789760761.554158:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704772379668 time 704789508268 state 0x2 ref 0 00000100:00100000:3.0:1789760761.588118:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554616448 00000100:00100000:3.0:1789760761.588128:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554616448 00000100:00100000:3.0:1789760761.588179:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704805695823 time 704819381060 state 0x0 ref 1 00000100:00100000:3.0:1789760761.588195:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704819641476 time 704819381060 state 0x1 ref 1 00000100:00100000:3.0:1789760761.588239:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704819641476 time 704833857821 state 0x0 ref 1 00000100:00100000:3.0:1789760761.588255:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fd5e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554616448:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760761.588309:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59293696, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0x1c478 00010000:00010000:3.0:1789760761.588320:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c360a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.588336:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760761.588340:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.588346:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c360a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59293696->59297791] (req 59293696->59297791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.588360:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c360a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59293696->59297791] (req 59293696->59297791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760761.594018:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59293696 -> 59297792 00002000:00010000:3.0:1789760761.594023:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 115832 -> 115840 00010000:00010000:3.0:1789760761.594037:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c360a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59293696->59297791] (req 59293696->59297791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.594050:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c360a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59293696->59297791] (req 59293696->59297791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.594060:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c360a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59293696->59297791] (req 59293696->59297791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.594067:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760761.594072:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c360a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59293696->59297791] (req 59293696->59297791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.594081:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c360a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59293696->59297791] (req 59293696->59297791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.594101:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c360a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59293696->59297791] (req 59293696->59297791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.594109:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c360a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59293696->59297791] (req 59293696->59297791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.594579:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fd5e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554616448:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6319us (6544us total) trans 4294981785 rc 0/0 00000100:00100000:3.0:1789760761.594602:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14487 00000100:00100000:3.0:1789760761.594612:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704819641476 time 704833857821 state 0x2 ref 0 00000100:00100000:3.0:1789760761.600011:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554616576 00000100:00100000:3.0:1789760761.600020:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554616576 00000100:00100000:3.0:1789760761.600075:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704819641476 time 704833857821 state 0x0 ref 1 00000100:00100000:3.0:1789760761.600090:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704834118237 time 704833857821 state 0x1 ref 1 00000100:00100000:3.0:1789760761.600121:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704834118237 time 704845739423 state 0x0 ref 1 00000100:00100000:3.0:1789760761.600133:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554616576:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760761.600181:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59297792, mtime: 0x6aad94f8, atime: 0x0, ctime: 0x6aad94f8, blocks: 0x1c480 00010000:00010000:3.0:1789760761.600191:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3611 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.600207:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760761.600458:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.600466:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3611 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59297792->59301887] (req 59297792->59301887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.600482:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3611 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59297792->59301887] (req 59297792->59301887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760761.627869:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981787 is committed 00000001:00080000:3.0:1789760761.650453:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981788 is committed 00000001:00080000:3.0:1789760761.662536:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981789 is committed 00000001:00080000:3.0:1789760761.671056:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981790 is committed 00000100:00100000:3.0:1789760761.748039:0:3649:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb09314380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3649:1876699573909504:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760761.748073:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760761.748126:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb064ece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573909632:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789760761.748149:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760761.748194:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573907968 00000100:00100000:3.0:1789760761.748198:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573907968 00000100:00100000:3.0:1789760761.748236:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 463 00000100:00100000:3.0:1789760761.748243:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc0c825c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5711db52-8702-4d6b-b42c-8543e9bf1cb2+24:3648:x1876699573907968:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760761.748274:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc0c825c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5711db52-8702-4d6b-b42c-8543e9bf1cb2+24:3648:x1876699573907968:12345-0@lo:400:kworker.0 Request processed in 32us (2102us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760761.748281:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 463 00000100:00100000:3.0:1789760761.748310:0:11526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573909504 00000100:00100000:3.0:1789760761.748314:0:11526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573909504 00000100:00100000:3.0:1789760761.748337:0:11526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 347 00000100:00100000:3.0:1789760761.748342:0:11526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb064eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1876699573909504:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1789760761.748355:0:11526:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:3.0:1789760761.748404:0:11526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb064eea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1876699573909504:12345-0@lo:13:osp-pre-1-1.0 Request processed in 61us (330us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760761.748410:0:11526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 347 00000100:00100000:3.0:1789760761.748424:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573909632 00000100:00100000:3.0:1789760761.748428:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573909632 00000100:00100000:3.0:1789760761.748445:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 374 00000100:00100000:3.0:1789760761.748451:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb064ed880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3646:x1876699573909632:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1789760761.748471:0:6526:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446227968 avail 1299988480 00000100:00100000:3.0:1789760761.748504:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb064ed880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3646:x1876699573909632:12345-0@lo:41:osp-pre-0-1.0 Request processed in 53us (354us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760761.748510:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 374 00000100:00100000:3.0:1789760761.748539:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb064ece00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573909632:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789760761.748564:0:3649:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb09314380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3649:1876699573909504:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760761.749109:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554617728 00000100:00100000:3.0:1789760761.749113:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554617728 00000100:00100000:3.0:1789760761.749148:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704952733574 time 704978138933 state 0x0 ref 1 00000100:00100000:3.0:1789760761.749275:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704978399349 time 704978138933 state 0x1 ref 1 00000100:00100000:3.0:1789760761.749301:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704978399349 time 704994919236 state 0x0 ref 1 00000100:00100000:3.0:1789760761.749314:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554617728:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760761.749354:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59334656, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0x1c4c8 00010000:00010000:3.0:1789760761.749362:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3650 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.749552:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789760761.749558:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.749565:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3650 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59334656->59338751] (req 59334656->59338751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.749583:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3650 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59334656->59338751] (req 59334656->59338751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760761.754952:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59334656 -> 59338752 00002000:00010000:3.0:1789760761.754956:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 115912 -> 115920 00010000:00010000:3.0:1789760761.754969:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3650 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59334656->59338751] (req 59334656->59338751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.754981:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3650 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59334656->59338751] (req 59334656->59338751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.754990:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3650 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59334656->59338751] (req 59334656->59338751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.754998:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760761.755004:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3650 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59334656->59338751] (req 59334656->59338751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.755017:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3650 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59334656->59338751] (req 59334656->59338751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.755035:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3650 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59334656->59338751] (req 59334656->59338751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.755043:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3650 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59334656->59338751] (req 59334656->59338751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.755176:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554617728:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5859us (6162us total) trans 4294981795 rc 0/0 00000100:00100000:3.0:1789760761.755186:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14497 00000100:00100000:3.0:1789760761.755196:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 704978399349 time 704994919236 state 0x2 ref 0 00000100:00100000:3.0:1789760761.826247:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554618240 00000100:00100000:3.0:1789760761.826257:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554618240 00000100:00100000:3.0:1789760761.826308:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705034288571 time 705053520627 state 0x0 ref 1 00000100:00100000:3.0:1789760761.826323:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705053781043 time 705053520627 state 0x1 ref 1 00000100:00100000:3.0:1789760761.826357:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705053781043 time 705071975048 state 0x0 ref 1 00000100:00100000:3.0:1789760761.826370:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04546300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554618240:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760761.826566:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59351040, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0x1c4e8 00010000:00010000:3.0:1789760761.826576:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c366c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.826592:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760761.826596:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.826602:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c366c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59351040->59355135] (req 59351040->59355135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.826614:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c366c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59351040->59355135] (req 59351040->59355135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760761.832153:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59351040 -> 59355136 00002000:00010000:3.0:1789760761.832159:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 115944 -> 115952 00010000:00010000:3.0:1789760761.832172:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c366c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59351040->59355135] (req 59351040->59355135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.832184:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c366c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59351040->59355135] (req 59351040->59355135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.832194:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c366c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59351040->59355135] (req 59351040->59355135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.832203:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760761.832218:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c366c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59351040->59355135] (req 59351040->59355135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.832229:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c366c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59351040->59355135] (req 59351040->59355135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.832254:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c366c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59351040->59355135] (req 59351040->59355135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.832262:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c366c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59351040->59355135] (req 59351040->59355135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760761.832448:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04546300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554618240:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6075us (6305us total) trans 4294981799 rc 0/0 00000100:00100000:3.0:1789760761.832458:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14501 00000100:00100000:3.0:1789760761.832467:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705053781043 time 705071975048 state 0x2 ref 0 00000001:00080000:3.0:1789760761.850311:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981800 is committed 00000001:00080000:3.0:1789760761.874029:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981801 is committed 00000001:00080000:3.0:1789760761.890429:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981802 is committed 00000100:00100000:3.0:1789760761.897994:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554618752 00000100:00100000:3.0:1789760761.898005:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554618752 00000100:00100000:3.0:1789760761.898061:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705114422589 time 705130475711 state 0x0 ref 1 00000100:00100000:3.0:1789760761.898075:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705130736127 time 705130475711 state 0x1 ref 1 00000100:00100000:3.0:1789760761.898110:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705130736127 time 705143728229 state 0x0 ref 1 00000100:00100000:3.0:1789760761.898123:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d83100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554618752:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760761.898168:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59367424, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0x1c508 00010000:00010000:3.0:1789760761.898178:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3688 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.898194:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760761.898199:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760761.898205:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3688 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59367424->59371519] (req 59367424->59371519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760761.898229:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3688 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59367424->59371519] (req 59367424->59371519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.024149:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981809 is committed 00000001:00080000:3.0:1789760762.060201:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981812 is committed 00000001:00080000:3.0:1789760762.079272:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981813 is committed 00000100:00100000:3.0:1789760762.329629:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554621824 00000100:00100000:3.0:1789760762.329640:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554621824 00000100:00100000:3.0:1789760762.329695:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705540747781 time 705559947685 state 0x0 ref 1 00000100:00100000:3.0:1789760762.329711:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705560208101 time 705559947685 state 0x1 ref 1 00000100:00100000:3.0:1789760762.329742:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705560208101 time 705575360954 state 0x0 ref 1 00000100:00100000:3.0:1789760762.329754:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0656ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554621824:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760762.329797:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59465728, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0x1c5c8 00010000:00010000:3.0:1789760762.329806:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3730 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.329823:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760762.329839:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.329847:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3730 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59465728->59469823] (req 59465728->59469823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.329861:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3730 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59465728->59469823] (req 59465728->59469823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760762.347439:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59465728 -> 59469824 00002000:00010000:3.0:1789760762.347445:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 116168 -> 116176 00010000:00010000:3.0:1789760762.347458:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3730 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59465728->59469823] (req 59465728->59469823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.347471:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3730 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59465728->59469823] (req 59465728->59469823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.347480:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3730 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59465728->59469823] (req 59465728->59469823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.347488:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760762.347493:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3730 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59465728->59469823] (req 59465728->59469823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.347502:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3730 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59465728->59469823] (req 59465728->59469823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.347521:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3730 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59465728->59469823] (req 59465728->59469823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.347534:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3730 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59465728->59469823] (req 59465728->59469823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.347765:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0656ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554621824:12345-192.168.203.22@tcp:4:dd.0 Request processed in 18006us (18264us total) trans 4294981827 rc 0/0 00000100:00100000:3.0:1789760762.347776:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14529 00000100:00100000:3.0:1789760762.347787:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705560208101 time 705575360954 state 0x2 ref 0 00000100:00100000:3.0:1789760762.408236:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554622336 00000100:00100000:3.0:1789760762.408247:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554622336 00000100:00100000:3.0:1789760762.408305:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705619206526 time 705640442654 state 0x0 ref 1 00000100:00100000:3.0:1789760762.408322:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705640703070 time 705640442654 state 0x1 ref 1 00000100:00100000:3.0:1789760762.408358:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705640703070 time 705653976097 state 0x0 ref 1 00000100:00100000:3.0:1789760762.408371:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554622336:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760762.408443:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59482112, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0x1c5e8 00010000:00010000:3.0:1789760762.408452:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c374c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.408468:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760762.408473:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.408479:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c374c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59482112->59486207] (req 59482112->59486207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.408498:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c374c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59482112->59486207] (req 59482112->59486207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760762.415462:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59482112 -> 59486208 00002000:00010000:3.0:1789760762.415467:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 116200 -> 116208 00010000:00010000:3.0:1789760762.415481:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c374c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59482112->59486207] (req 59482112->59486207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.415492:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c374c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59482112->59486207] (req 59482112->59486207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.415500:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c374c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59482112->59486207] (req 59482112->59486207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.415506:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760762.415511:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c374c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59482112->59486207] (req 59482112->59486207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.415519:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c374c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59482112->59486207] (req 59482112->59486207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.415539:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c374c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59482112->59486207] (req 59482112->59486207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.415547:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c374c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59482112->59486207] (req 59482112->59486207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.415766:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554622336:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7392us (8387us total) trans 4294981831 rc 0/0 00000100:00100000:3.0:1789760762.415777:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14533 00000100:00100000:3.0:1789760762.415786:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705640703070 time 705653976097 state 0x2 ref 0 00000001:00080000:3.0:1789760762.426466:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981832 is committed 00000100:00100000:3.0:1789760762.463421:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554622848 00000100:00100000:3.0:1789760762.463430:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554622848 00000100:00100000:3.0:1789760762.463484:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705677921993 time 705693369219 state 0x0 ref 1 00000100:00100000:3.0:1789760762.463500:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705693629635 time 705693369219 state 0x1 ref 1 00000100:00100000:3.0:1789760762.463531:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705693629635 time 705709149830 state 0x0 ref 1 00000100:00100000:3.0:1789760762.463544:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554622848:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760762.463592:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59498496, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0x1c608 00010000:00010000:3.0:1789760762.463601:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3768 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.463618:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760762.463622:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.463629:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3768 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59498496->59502591] (req 59498496->59502591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.463643:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3768 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59498496->59502591] (req 59498496->59502591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.546897:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554623616 00000100:00100000:3.0:1789760762.546907:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554623616 00000100:00100000:3.0:1789760762.546964:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705760872930 time 705776513800 state 0x0 ref 1 00000100:00100000:3.0:1789760762.546979:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705776774216 time 705776513800 state 0x1 ref 1 00000100:00100000:3.0:1789760762.547049:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705776774216 time 705792666701 state 0x0 ref 1 00000100:00100000:3.0:1789760762.547063:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554623616:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760762.547109:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59523072, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0x1c638 00010000:00010000:3.0:1789760762.547119:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3792 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.547134:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760762.547138:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.547145:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3792 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59523072->59527167] (req 59523072->59527167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.547158:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3792 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59523072->59527167] (req 59523072->59527167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760762.553330:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59523072 -> 59527168 00002000:00010000:3.0:1789760762.553335:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 116280 -> 116288 00010000:00010000:3.0:1789760762.553349:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3792 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59523072->59527167] (req 59523072->59527167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.553486:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3792 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59523072->59527167] (req 59523072->59527167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.553496:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3792 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59523072->59527167] (req 59523072->59527167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.553504:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760762.553508:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3792 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59523072->59527167] (req 59523072->59527167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.553518:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3792 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59523072->59527167] (req 59523072->59527167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.553538:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3792 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59523072->59527167] (req 59523072->59527167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.553547:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3792 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59523072->59527167] (req 59523072->59527167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.553716:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554623616:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6651us (6912us total) trans 4294981841 rc 0/0 00000100:00100000:3.0:1789760762.553727:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14543 00000100:00100000:3.0:1789760762.553736:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705776774216 time 705792666701 state 0x2 ref 0 00000100:00100000:3.0:1789760762.606962:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554624128 00000100:00100000:3.0:1789760762.606972:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554624128 00000100:00100000:3.0:1789760762.607025:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705820020858 time 705835560700 state 0x0 ref 1 00000100:00100000:3.0:1789760762.607040:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705835821116 time 705835560700 state 0x1 ref 1 00000100:00100000:3.0:1789760762.607072:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705835821116 time 705852690980 state 0x0 ref 1 00000100:00100000:3.0:1789760762.607085:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06569f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554624128:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760762.607129:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59539456, mtime: 0x6aad94f9, atime: 0x0, ctime: 0x6aad94f9, blocks: 0x1c658 00010000:00010000:3.0:1789760762.607138:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c37ae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.607153:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760762.607158:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.607164:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c37ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59539456->59543551] (req 59539456->59543551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.607178:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c37ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59539456->59543551] (req 59539456->59543551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760762.613301:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59539456 -> 59543552 00002000:00010000:3.0:1789760762.613306:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 116312 -> 116320 00010000:00010000:3.0:1789760762.613320:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c37ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59539456->59543551] (req 59539456->59543551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.613333:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c37ae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59539456->59543551] (req 59539456->59543551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.613346:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c37ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59539456->59543551] (req 59539456->59543551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.613354:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760762.613359:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c37ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59539456->59543551] (req 59539456->59543551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.613368:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c37ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59539456->59543551] (req 59539456->59543551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.613402:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c37ae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59539456->59543551] (req 59539456->59543551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.613411:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c37ae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59539456->59543551] (req 59539456->59543551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.613557:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06569f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554624128:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6469us (6681us total) trans 4294981845 rc 0/0 00000100:00100000:3.0:1789760762.613568:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14547 00000100:00100000:3.0:1789760762.613578:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705835821116 time 705852690980 state 0x2 ref 0 00000100:00100000:3.0:1789760762.663181:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554624640 00000100:00100000:3.0:1789760762.663190:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554624640 00000100:00100000:3.0:1789760762.663360:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705873872429 time 705886693620 state 0x0 ref 1 00000100:00100000:3.0:1789760762.663375:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705886954036 time 705886693620 state 0x1 ref 1 00000100:00100000:3.0:1789760762.663445:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705886954036 time 705909063178 state 0x0 ref 1 00000100:00100000:3.0:1789760762.663459:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554624640:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760762.663509:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59555840, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x1c678 00010000:00010000:3.0:1789760762.663519:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c37ca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.663535:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=705s v=(0 0) 00010000:00010000:3.0:1789760762.663540:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.663546:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c37ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59555840->59559935] (req 59555840->59559935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.663560:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c37ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59555840->59559935] (req 59555840->59559935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760762.749810:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59572224 -> 59576320 00002000:00010000:3.0:1789760762.749822:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 116376 -> 116384 00010000:00010000:3.0:1789760762.749846:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c37e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59572224->59576319] (req 59572224->59576319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.749927:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c37e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59572224->59576319] (req 59572224->59576319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.749937:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c37e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59572224->59576319] (req 59572224->59576319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.749949:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760762.749954:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c37e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59572224->59576319] (req 59572224->59576319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.749962:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c37e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59572224->59576319] (req 59572224->59576319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.749981:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c37e6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59572224->59576319] (req 59572224->59576319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.749988:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c37e6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59572224->59576319] (req 59572224->59576319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.750144:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554625152:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14079us (14334us total) trans 4294981853 rc 0/0 00000100:00100000:3.0:1789760762.750155:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14555 00000100:00100000:3.0:1789760762.750165:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 705961313681 time 705981668259 state 0x2 ref 0 00000100:00100000:3.0:1789760762.822964:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760762.833557:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981857 is committed 00002000:00010000:3.0:1789760762.833976:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59588608 -> 59592704 00002000:00010000:3.0:1789760762.833981:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 116408 -> 116416 00010000:00010000:3.0:1789760762.833994:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c3802 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59588608->59592703] (req 59588608->59592703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.834014:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c3802 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59588608->59592703] (req 59588608->59592703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.834022:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c3802 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59588608->59592703] (req 59588608->59592703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.834029:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760762.834034:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c3802 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59588608->59592703] (req 59588608->59592703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.834043:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c3802 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59588608->59592703] (req 59588608->59592703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.834064:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c3802 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59588608->59592703] (req 59588608->59592703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.834070:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c3802 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59588608->59592703] (req 59588608->59592703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.834201:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb064ece00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554625664:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6309us (11240us total) trans 4294981857 rc 0/0 00000100:00100000:3.0:1789760762.834320:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14559 00000100:00100000:3.0:1789760762.834330:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706051784996 time 706073486744 state 0x2 ref 0 00000001:00080000:3.0:1789760762.848189:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981858 is committed 00000100:00100000:3.0:1789760762.856125:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760762.861944:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981859 is committed 00000100:00100000:3.0:1789760762.871959:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554626048 00000100:00100000:3.0:1789760762.871969:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554626048 00000100:00100000:3.0:1789760762.872022:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706085027552 time 706102043251 state 0x0 ref 1 00000100:00100000:3.0:1789760762.872037:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706102303667 time 706102043251 state 0x1 ref 1 00000100:00100000:3.0:1789760762.872096:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706102303667 time 706117714503 state 0x0 ref 1 00000100:00100000:3.0:1789760762.872118:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06569500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554626048:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760762.872164:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59600896, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x1c6d0 00010000:00010000:3.0:1789760762.872173:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3817 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.872188:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760762.872193:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.872199:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3817 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59600896->59604991] (req 59600896->59604991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.872224:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3817 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59600896->59604991] (req 59600896->59604991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760762.882633:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59600896 -> 59604992 00002000:00010000:3.0:1789760762.882638:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 116432 -> 116440 00010000:00010000:3.0:1789760762.882652:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3817 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59600896->59604991] (req 59600896->59604991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.882669:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3817 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59600896->59604991] (req 59600896->59604991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.882678:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3817 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59600896->59604991] (req 59600896->59604991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.882685:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760762.882690:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3817 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59600896->59604991] (req 59600896->59604991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.882699:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3817 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59600896->59604991] (req 59600896->59604991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.882719:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3817 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59600896->59604991] (req 59600896->59604991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.882727:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3817 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59600896->59604991] (req 59600896->59604991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.882812:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06569500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554626048:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10701us (11023us total) trans 4294981860 rc 0/0 00000100:00100000:3.0:1789760762.882821:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14562 00000100:00100000:3.0:1789760762.882842:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706102303667 time 706117714503 state 0x2 ref 0 00000100:00100000:3.0:1789760762.889237:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554626176 00000100:00100000:3.0:1789760762.889248:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554626176 00000100:00100000:3.0:1789760762.889305:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706102303667 time 706117714503 state 0x0 ref 1 00000100:00100000:3.0:1789760762.889320:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706117974919 time 706117714503 state 0x1 ref 1 00000100:00100000:3.0:1789760762.889351:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706117974919 time 706134969022 state 0x0 ref 1 00000100:00100000:3.0:1789760762.889364:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554626176:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760762.889448:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59604992, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x1c6d8 00010000:00010000:3.0:1789760762.889458:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c381e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.889475:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760762.889479:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.889485:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c381e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59604992->59609087] (req 59604992->59609087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.889498:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c381e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59604992->59609087] (req 59604992->59609087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760762.897037:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59604992 -> 59609088 00002000:00010000:3.0:1789760762.897043:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 116440 -> 116448 00010000:00010000:3.0:1789760762.897057:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c381e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59604992->59609087] (req 59604992->59609087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.897069:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c381e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59604992->59609087] (req 59604992->59609087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.897083:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c381e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59604992->59609087] (req 59604992->59609087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.897089:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760762.897094:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c381e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59604992->59609087] (req 59604992->59609087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.897102:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c381e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59604992->59609087] (req 59604992->59609087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.897121:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c381e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59604992->59609087] (req 59604992->59609087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.897128:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c381e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59604992->59609087] (req 59604992->59609087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760762.897233:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554626176:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7866us (8069us total) trans 4294981861 rc 0/0 00000100:00100000:3.0:1789760762.897244:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14563 00000100:00100000:3.0:1789760762.897252:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706117974919 time 706134969022 state 0x2 ref 0 00000100:00100000:3.0:1789760762.902424:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554626304 00000100:00100000:3.0:1789760762.902434:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554626304 00000100:00100000:3.0:1789760762.902490:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706117974919 time 706134969022 state 0x0 ref 1 00000100:00100000:3.0:1789760762.902507:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706135229438 time 706134969022 state 0x1 ref 1 00000001:00080000:3.0:1789760762.910778:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981862 is committed 00000001:00080000:3.0:1789760762.921422:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981863 is committed 00000001:00080000:3.0:1789760762.949430:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981864 is committed 00000100:00100000:3.0:1789760762.955314:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760762.960276:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981865 is committed 00000100:00100000:3.0:1789760762.967912:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554626816 00000100:00100000:3.0:1789760762.967921:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554626816 00000100:00100000:3.0:1789760762.967976:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706188471245 time 706201299751 state 0x0 ref 1 00000100:00100000:3.0:1789760762.967990:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706201560167 time 706201299751 state 0x1 ref 1 00000100:00100000:3.0:1789760762.968068:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706201560167 time 706213685883 state 0x0 ref 1 00000100:00100000:3.0:1789760762.968080:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06568a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554626816:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760762.968139:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59625472, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x1c700 00010000:00010000:3.0:1789760762.968147:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3841 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.968162:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760762.968166:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760762.968172:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3841 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59625472->59629567] (req 59625472->59629567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760762.968185:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3841 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59625472->59629567] (req 59625472->59629567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760762.975368:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981866 is committed 00000001:00080000:3.0:1789760762.993349:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981867 is committed 00000100:00100000:3.0:1789760763.017779:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760763.022769:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981868 is committed 00000100:00100000:3.0:1789760763.033603:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554627200 00000100:00100000:3.0:1789760763.033612:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554627200 00000100:00100000:3.0:1789760763.033660:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706232729180 time 706263634118 state 0x0 ref 1 00000100:00100000:3.0:1789760763.033675:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706263894534 time 706263634118 state 0x1 ref 1 00000100:00100000:3.0:1789760763.033706:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706263894534 time 706279325000 state 0x0 ref 1 00000100:00100000:3.0:1789760763.033717:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06568e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554627200:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760763.033770:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59637760, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x1c718 00010000:00010000:3.0:1789760763.033780:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3856 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.033795:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760763.033799:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.033805:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3856 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59637760->59641855] (req 59637760->59641855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.033818:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3856 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59637760->59641855] (req 59637760->59641855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.041172:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59637760 -> 59641856 00002000:00010000:3.0:1789760763.041177:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 116504 -> 116512 00010000:00010000:3.0:1789760763.041191:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3856 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59637760->59641855] (req 59637760->59641855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.041267:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3856 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59637760->59641855] (req 59637760->59641855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.041278:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3856 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59637760->59641855] (req 59637760->59641855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.041287:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760763.041292:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3856 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59637760->59641855] (req 59637760->59641855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.041364:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3856 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59637760->59641855] (req 59637760->59641855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.041473:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3856 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59637760->59641855] (req 59637760->59641855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.041483:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3856 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59637760->59641855] (req 59637760->59641855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.041579:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06568e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554627200:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7858us (8046us total) trans 4294981869 rc 0/0 00000100:00100000:3.0:1789760763.041590:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14571 00000100:00100000:3.0:1789760763.041599:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706263894534 time 706279325000 state 0x2 ref 0 00000100:00100000:3.0:1789760763.045999:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554627328 00000100:00100000:3.0:1789760763.046010:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554627328 00000100:00100000:3.0:1789760763.046066:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706263894534 time 706279325000 state 0x0 ref 1 00000100:00100000:3.0:1789760763.046083:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706279585416 time 706279325000 state 0x1 ref 1 00000100:00100000:3.0:1789760763.046163:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706279585416 time 706291780325 state 0x0 ref 1 00000100:00100000:3.0:1789760763.046176:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554627328:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760763.046219:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59641856, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x1c720 00010000:00010000:3.0:1789760763.046228:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c385d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.046243:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760763.046247:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.046253:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c385d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59641856->59645951] (req 59641856->59645951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.046266:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c385d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59641856->59645951] (req 59641856->59645951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.095735:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554627712 00000100:00100000:3.0:1789760763.095745:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554627712 00000100:00100000:3.0:1789760763.095798:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706314077009 time 706330032094 state 0x0 ref 1 00000100:00100000:3.0:1789760763.095813:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706330292510 time 706330032094 state 0x1 ref 1 00000100:00100000:3.0:1789760763.095850:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706330292510 time 706341468238 state 0x0 ref 1 00000100:00100000:3.0:1789760763.095866:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06569c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554627712:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760763.095910:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59654144, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x1c738 00010000:00010000:3.0:1789760763.095919:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3872 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.096082:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760763.096086:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.096092:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3872 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59654144->59658239] (req 59654144->59658239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.096105:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3872 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59654144->59658239] (req 59654144->59658239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.102071:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59654144 -> 59658240 00002000:00010000:3.0:1789760763.102076:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 116536 -> 116544 00010000:00010000:3.0:1789760763.102089:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3872 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59654144->59658239] (req 59654144->59658239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.102103:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3872 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59654144->59658239] (req 59654144->59658239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.102112:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3872 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59654144->59658239] (req 59654144->59658239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.102120:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760763.102126:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3872 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59654144->59658239] (req 59654144->59658239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.102136:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3872 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59654144->59658239] (req 59654144->59658239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.102155:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3872 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59654144->59658239] (req 59654144->59658239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.102163:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3872 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59654144->59658239] (req 59654144->59658239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.102446:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06569c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554627712:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6576us (6796us total) trans 4294981873 rc 0/0 00000100:00100000:3.0:1789760763.102457:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14575 00000100:00100000:3.0:1789760763.102467:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706330292510 time 706341468238 state 0x2 ref 0 00000100:00100000:3.0:1789760763.113048:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760763.162698:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554628864 00000100:00100000:3.0:1789760763.162708:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554628864 00000100:00100000:3.0:1789760763.162759:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706370101481 time 706387050478 state 0x0 ref 1 00000100:00100000:3.0:1789760763.162774:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706387310894 time 706387050478 state 0x1 ref 1 00000100:00100000:3.0:1789760763.162801:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706387310894 time 706408420124 state 0x0 ref 1 00000100:00100000:3.0:1789760763.162811:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d81c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554628864:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760763.162870:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59670528, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x1c758 00010000:00010000:3.0:1789760763.162881:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c388e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.162898:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760763.162903:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.162910:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c388e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59670528->59674623] (req 59670528->59674623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.162925:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c388e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59670528->59674623] (req 59670528->59674623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.181543:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59670528 -> 59674624 00002000:00010000:3.0:1789760763.181555:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 116568 -> 116576 00010000:00010000:3.0:1789760763.181569:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c388e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59670528->59674623] (req 59670528->59674623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.181583:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c388e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59670528->59674623] (req 59670528->59674623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.181592:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c388e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59670528->59674623] (req 59670528->59674623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.181599:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760763.181604:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c388e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59670528->59674623] (req 59670528->59674623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.181619:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c388e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59670528->59674623] (req 59670528->59674623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.181637:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c388e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59670528->59674623] (req 59670528->59674623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.181645:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c388e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59670528->59674623] (req 59670528->59674623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.181742:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d81c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554628864:12345-192.168.203.22@tcp:4:dd.0 Request processed in 18927us (19132us total) trans 4294981877 rc 0/0 00000100:00100000:3.0:1789760763.181753:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14579 00000100:00100000:3.0:1789760763.181764:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706387310894 time 706408420124 state 0x2 ref 0 00000100:00100000:3.0:1789760763.187185:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554628992 00000100:00100000:3.0:1789760763.187197:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554628992 00000100:00100000:3.0:1789760763.187257:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706387310894 time 706408420124 state 0x0 ref 1 00000100:00100000:3.0:1789760763.187272:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706408680540 time 706408420124 state 0x1 ref 1 00000100:00100000:3.0:1789760763.188222:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706408680540 time 706433834525 state 0x0 ref 1 00000100:00100000:3.0:1789760763.188243:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03c1b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554628992:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760763.188293:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59674624, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x1c760 00010000:00010000:3.0:1789760763.188302:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3895 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.188318:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760763.188323:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.188331:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3895 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59674624->59678719] (req 59674624->59678719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.188346:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3895 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59674624->59678719] (req 59674624->59678719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.195480:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59674624 -> 59678720 00002000:00010000:3.0:1789760763.195487:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 116576 -> 116584 00010000:00010000:3.0:1789760763.195504:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3895 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59674624->59678719] (req 59674624->59678719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.195518:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3895 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59674624->59678719] (req 59674624->59678719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.195527:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3895 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59674624->59678719] (req 59674624->59678719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.195536:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760763.195542:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3895 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59674624->59678719] (req 59674624->59678719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.195552:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3895 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59674624->59678719] (req 59674624->59678719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.195584:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3895 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59674624->59678719] (req 59674624->59678719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.195593:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3895 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59674624->59678719] (req 59674624->59678719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.195709:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03c1b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554628992:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7465us (8641us total) trans 4294981878 rc 0/0 00000100:00100000:3.0:1789760763.195723:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14580 00000100:00100000:3.0:1789760763.195734:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706408680540 time 706433834525 state 0x2 ref 0 00000100:00100000:3.0:1789760763.238776:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554629376 00000100:00100000:3.0:1789760763.238786:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554629376 00000100:00100000:3.0:1789760763.238852:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706450069397 time 706467460469 state 0x0 ref 1 00000100:00100000:3.0:1789760763.238869:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706467720885 time 706467460469 state 0x1 ref 1 00000100:00100000:3.0:1789760763.238900:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706467720885 time 706484518699 state 0x0 ref 1 00000100:00100000:3.0:1789760763.238913:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03c19880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554629376:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760763.238958:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59686912, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x1c778 00010000:00010000:3.0:1789760763.238968:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c38aa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.238984:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760763.238990:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.238996:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c38aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59686912->59691007] (req 59686912->59691007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.239101:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c38aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59686912->59691007] (req 59686912->59691007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.261690:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554629504 00000100:00100000:3.0:1789760763.261704:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554629504 00000100:00100000:3.0:1789760763.262010:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706467720885 time 706484518699 state 0x0 ref 1 00000100:00100000:3.0:1789760763.262029:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706484779115 time 706484518699 state 0x1 ref 1 00000100:00100000:3.0:1789760763.262065:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706484779115 time 706507683178 state 0x0 ref 1 00000100:00100000:3.0:1789760763.262080:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554629504:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760763.262131:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59691008, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x1c780 00010000:00010000:3.0:1789760763.262142:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38b1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.262160:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760763.262166:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.262173:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59691008->59695103] (req 59691008->59695103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.262188:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59691008->59695103] (req 59691008->59695103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.268267:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59691008 -> 59695104 00002000:00010000:3.0:1789760763.268273:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 116608 -> 116616 00010000:00010000:3.0:1789760763.268286:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59691008->59695103] (req 59691008->59695103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.268299:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38b1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59691008->59695103] (req 59691008->59695103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.268309:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59691008->59695103] (req 59691008->59695103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.268316:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760763.268322:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59691008->59695103] (req 59691008->59695103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.268330:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38b1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59691008->59695103] (req 59691008->59695103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.268349:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38b1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59691008->59695103] (req 59691008->59695103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.268357:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38b1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59691008->59695103] (req 59691008->59695103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.268504:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554629504:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6423us (9308us total) trans 4294981882 rc 0/0 00000100:00100000:3.0:1789760763.268515:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14584 00000100:00100000:3.0:1789760763.268525:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706484779115 time 706507683178 state 0x2 ref 0 00000100:00100000:3.0:1789760763.323727:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554630016 00000100:00100000:3.0:1789760763.323736:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554630016 00000100:00100000:3.0:1789760763.323786:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706541726388 time 706553830762 state 0x0 ref 1 00000100:00100000:3.0:1789760763.323801:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706554091178 time 706553830762 state 0x1 ref 1 00000100:00100000:3.0:1789760763.323841:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706554091178 time 706569459144 state 0x0 ref 1 00000100:00100000:3.0:1789760763.323854:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d83100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554630016:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760763.323897:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59707392, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x1c7a0 00010000:00010000:3.0:1789760763.323907:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38cd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.323930:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760763.323935:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.323942:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59707392->59711487] (req 59707392->59711487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.323957:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59707392->59711487] (req 59707392->59711487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.337260:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59707392 -> 59711488 00002000:00010000:3.0:1789760763.337265:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 116640 -> 116648 00010000:00010000:3.0:1789760763.337279:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59707392->59711487] (req 59707392->59711487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.337292:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59707392->59711487] (req 59707392->59711487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.337301:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59707392->59711487] (req 59707392->59711487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.337309:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760763.337314:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59707392->59711487] (req 59707392->59711487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.337323:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38cd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59707392->59711487] (req 59707392->59711487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.337343:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38cd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59707392->59711487] (req 59707392->59711487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.337350:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38cd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59707392->59711487] (req 59707392->59711487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.337490:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d83100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554630016:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13633us (13849us total) trans 4294981886 rc 0/0 00000100:00100000:3.0:1789760763.337500:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14588 00000100:00100000:3.0:1789760763.337510:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706554091178 time 706569459144 state 0x2 ref 0 00000100:00100000:3.0:1789760763.401347:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554630528 00000100:00100000:3.0:1789760763.401357:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554630528 00000100:00100000:3.0:1789760763.401453:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706609723361 time 706628077506 state 0x0 ref 1 00000100:00100000:3.0:1789760763.401468:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706628337922 time 706628077506 state 0x1 ref 1 00000100:00100000:3.0:1789760763.401500:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706628337922 time 706647117974 state 0x0 ref 1 00000100:00100000:3.0:1789760763.401512:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554630528:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760763.401560:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59723776, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x1c7c0 00010000:00010000:3.0:1789760763.401569:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38e9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.401595:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760763.401599:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.401606:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59723776->59727871] (req 59723776->59727871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.401621:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59723776->59727871] (req 59723776->59727871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.411444:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59723776 -> 59727872 00002000:00010000:3.0:1789760763.411449:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 116672 -> 116680 00010000:00010000:3.0:1789760763.411462:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59723776->59727871] (req 59723776->59727871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.411480:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59723776->59727871] (req 59723776->59727871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.411490:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59723776->59727871] (req 59723776->59727871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.411497:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760763.411502:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59723776->59727871] (req 59723776->59727871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.411513:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59723776->59727871] (req 59723776->59727871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.411533:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38e9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59723776->59727871] (req 59723776->59727871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.411540:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c38e9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59723776->59727871] (req 59723776->59727871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.411712:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554630528:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10195us (10514us total) trans 4294981890 rc 0/0 00000100:00100000:3.0:1789760763.411724:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14592 00000100:00100000:3.0:1789760763.411733:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706628337922 time 706647117974 state 0x2 ref 0 00000100:00100000:3.0:1789760763.472992:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554631040 00000100:00100000:3.0:1789760763.473002:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554631040 00000100:00100000:3.0:1789760763.473057:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706684577205 time 706702986104 state 0x0 ref 1 00000100:00100000:3.0:1789760763.473075:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706703246520 time 706702986104 state 0x1 ref 1 00000100:00100000:3.0:1789760763.473155:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706703246520 time 706718771465 state 0x0 ref 1 00000100:00100000:3.0:1789760763.473168:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554631040:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760763.473226:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59740160, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x1c7e0 00010000:00010000:3.0:1789760763.473236:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3905 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.473251:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760763.473256:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.473263:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3905 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59740160->59744255] (req 59740160->59744255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.473277:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3905 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59740160->59744255] (req 59740160->59744255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760763.477976:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981894 is committed 00000100:00100000:3.0:1789760763.483925:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760763.495460:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981895 is committed 00000001:00080000:3.0:1789760763.512743:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981896 is committed 00000001:00080000:3.0:1789760763.526912:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981897 is committed 00000100:00100000:3.0:1789760763.533849:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554631552 00000100:00100000:3.0:1789760763.533859:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554631552 00000100:00100000:3.0:1789760763.533920:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706750506617 time 706763771434 state 0x0 ref 1 00000100:00100000:3.0:1789760763.533936:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706764031850 time 706763771434 state 0x1 ref 1 00000100:00100000:3.0:1789760763.533968:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706764031850 time 706779587371 state 0x0 ref 1 00000100:00100000:3.0:1789760763.533979:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04546680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554631552:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760763.534025:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59756544, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x1c800 00010000:00010000:3.0:1789760763.534034:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3921 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.534053:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760763.534057:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.534064:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3921 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59756544->59760639] (req 59756544->59760639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.534078:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3921 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59756544->59760639] (req 59756544->59760639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.539044:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59756544 -> 59760640 00002000:00010000:3.0:1789760763.539050:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 116736 -> 116744 00010000:00010000:3.0:1789760763.539063:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3921 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59756544->59760639] (req 59756544->59760639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.539076:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3921 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59756544->59760639] (req 59756544->59760639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.539086:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3921 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59756544->59760639] (req 59756544->59760639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.539094:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760763.539100:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3921 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59756544->59760639] (req 59756544->59760639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.539109:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3921 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59756544->59760639] (req 59756544->59760639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.539127:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3921 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59756544->59760639] (req 59756544->59760639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.539135:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3921 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59756544->59760639] (req 59756544->59760639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.539271:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04546680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554631552:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5287us (5559us total) trans 4294981898 rc 0/0 00000100:00100000:3.0:1789760763.539281:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14600 00000100:00100000:3.0:1789760763.539290:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706764031850 time 706779587371 state 0x2 ref 0 00000001:00080000:3.0:1789760763.550186:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981899 is committed 00000100:00100000:3.0:1789760763.574354:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760763.593835:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554632064 00000100:00100000:3.0:1789760763.593846:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554632064 00000100:00100000:3.0:1789760763.593903:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706801330986 time 706820797493 state 0x0 ref 1 00000100:00100000:3.0:1789760763.594938:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706821057909 time 706820797493 state 0x1 ref 1 00000100:00100000:3.0:1789760763.594989:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706821057909 time 706840605374 state 0x0 ref 1 00000100:00100000:3.0:1789760763.595009:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04544380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554632064:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760763.595067:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59772928, mtime: 0x6aad94fa, atime: 0x0, ctime: 0x6aad94fa, blocks: 0x1c820 00010000:00010000:3.0:1789760763.595076:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c393d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.595092:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760763.595097:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.595103:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c393d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59772928->59777023] (req 59772928->59777023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.595117:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c393d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59772928->59777023] (req 59772928->59777023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760763.601464:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59772928 -> 59777024 00002000:00010000:3.0:1789760763.601469:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 116768 -> 116776 00010000:00010000:3.0:1789760763.601484:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c393d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59772928->59777023] (req 59772928->59777023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.601496:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c393d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59772928->59777023] (req 59772928->59777023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.601504:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c393d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59772928->59777023] (req 59772928->59777023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.601517:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760763.601522:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c393d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59772928->59777023] (req 59772928->59777023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.601530:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c393d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59772928->59777023] (req 59772928->59777023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.601549:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c393d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59772928->59777023] (req 59772928->59777023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.601556:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c393d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59772928->59777023] (req 59772928->59777023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760763.602945:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04544380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554632064:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7929us (9198us total) trans 4294981902 rc 0/0 00000100:00100000:3.0:1789760763.602959:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14604 00000100:00100000:3.0:1789760763.602969:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706821057909 time 706840605374 state 0x2 ref 0 00000100:00100000:3.0:1789760763.658288:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554632576 00000100:00100000:3.0:1789760763.658356:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554632576 00000100:00100000:3.0:1789760763.658422:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706876219369 time 706890154502 state 0x0 ref 1 00000100:00100000:3.0:1789760763.658434:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706890414918 time 706890154502 state 0x1 ref 1 00000100:00100000:3.0:1789760763.658491:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 706890414918 time 706904109863 state 0x0 ref 1 00000100:00100000:3.0:1789760763.658504:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04547800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554632576:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760763.658544:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59789312, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0x1c840 00010000:00010000:3.0:1789760763.658552:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3959 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.658568:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=706s v=(0 0) 00010000:00010000:3.0:1789760763.658572:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760763.658578:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3959 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59789312->59793407] (req 59789312->59793407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760763.658591:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3959 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59789312->59793407] (req 59789312->59793407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760763.782564:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981912 is committed 00000100:00100000:3.0:1789760763.795698:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760763.802657:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981913 is committed 00000001:00080000:3.0:1789760763.823061:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981914 is committed 00000100:00100000:3.0:1789760763.833275:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760763.858515:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760763.889122:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981918 is committed 00000100:00100000:3.0:1789760763.901555:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760763.909497:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981919 is committed 00000100:00100000:3.0:1789760763.942632:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760763.998119:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760764.023727:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981926 is committed 00000001:00080000:3.0:1789760764.037456:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981927 is committed 00000100:00100000:3.0:1789760764.042282:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760764.072475:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981929 is committed 00000001:00080000:3.0:1789760764.089711:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981930 is committed 00000001:00080000:3.0:1789760764.102482:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981931 is committed 00000001:00080000:3.0:1789760764.117686:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981932 is committed 00000001:00080000:3.0:1789760764.131899:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981933 is committed 00000001:00080000:3.0:1789760764.150748:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981934 is committed 00000001:00080000:3.0:1789760764.216774:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981938 is committed 00000100:00100000:3.0:1789760764.238858:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:3.0:1789760764.264373:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59932672 -> 59936768 00002000:00010000:3.0:1789760764.264405:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 117080 -> 117088 00010000:00010000:3.0:1789760764.264419:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c3a4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59932672->59936767] (req 59932672->59936767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.264432:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c3a4e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59932672->59936767] (req 59932672->59936767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.264441:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c3a4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59932672->59936767] (req 59932672->59936767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.264448:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760764.264458:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c3a4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59932672->59936767] (req 59932672->59936767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.264468:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c3a4e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59932672->59936767] (req 59932672->59936767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.264487:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c3a4e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59932672->59936767] (req 59932672->59936767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.264494:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c3a4e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59932672->59936767] (req 59932672->59936767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.265139:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03c1b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554637056:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11597us (11945us total) trans 4294981941 rc 0/0 00000100:00100000:3.0:1789760764.265151:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14643 00000100:00100000:3.0:1789760764.265161:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 707485219031 time 707499142953 state 0x2 ref 0 00000100:00100000:3.0:1789760764.286607:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554637312 00000100:00100000:3.0:1789760764.286617:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554637312 00000100:00100000:3.0:1789760764.286664:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 707499403369 time 707517478413 state 0x0 ref 1 00000100:00100000:3.0:1789760764.286677:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 707517738829 time 707517478413 state 0x1 ref 1 00000100:00100000:3.0:1789760764.286704:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 707517738829 time 707532323687 state 0x0 ref 1 00000100:00100000:3.0:1789760764.286714:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06569c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554637312:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760764.286760:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59940864, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0x1c968 00010000:00010000:3.0:1789760764.286769:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3a5c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.286783:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760764.286787:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.286800:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3a5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59940864->59944959] (req 59940864->59944959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.286825:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3a5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59940864->59944959] (req 59940864->59944959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760764.292263:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981943 is committed 00000001:00080000:3.0:1789760764.308842:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981944 is committed 00000001:00080000:3.0:1789760764.319121:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981945 is committed 00000001:00080000:3.0:1789760764.335428:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981946 is committed 00000100:00100000:3.0:1789760764.339249:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554637824 00000100:00100000:3.0:1789760764.339259:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554637824 00000100:00100000:3.0:1789760764.339312:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 707559671197 time 707575523443 state 0x0 ref 1 00000100:00100000:3.0:1789760764.339327:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 707575783859 time 707575523443 state 0x1 ref 1 00000100:00100000:3.0:1789760764.339422:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 707575783859 time 707585039785 state 0x0 ref 1 00000100:00100000:3.0:1789760764.339436:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554637824:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760764.339481:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 59957248, mtime: 0x6aad94fb, atime: 0x0, ctime: 0x6aad94fb, blocks: 0x1c988 00010000:00010000:3.0:1789760764.339490:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3a78 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.339504:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760764.339508:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.339514:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3a78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59957248->59961343] (req 59957248->59961343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.339528:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3a78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59957248->59961343] (req 59957248->59961343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760764.346109:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981947 is committed 00000100:00100000:3.0:1789760764.373624:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:3.0:1789760764.380064:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 59965440 -> 59969536 00002000:00010000:3.0:1789760764.380069:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 117144 -> 117152 00010000:00010000:3.0:1789760764.380083:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c3a86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59965440->59969535] (req 59965440->59969535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.380095:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c3a86 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59965440->59969535] (req 59965440->59969535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.380104:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c3a86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59965440->59969535] (req 59965440->59969535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.380114:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760764.380119:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c3a86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59965440->59969535] (req 59965440->59969535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.380128:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c3a86 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59965440->59969535] (req 59965440->59969535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.380148:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c3a86 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59965440->59969535] (req 59965440->59969535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.380156:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c3a86 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [59965440->59969535] (req 59965440->59969535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.380457:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554638080:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6622us (6835us total) trans 4294981949 rc 0/0 00000100:00100000:3.0:1789760764.380468:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14651 00000100:00100000:3.0:1789760764.380477:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 707606540869 time 707619430199 state 0x2 ref 0 00000001:00080000:3.0:1789760764.395786:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981950 is committed 00000100:00100000:3.0:1789760764.413755:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760764.423496:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554638592 00000100:00100000:3.0:1789760764.423508:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554638592 00000100:00100000:3.0:1789760764.444939:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760764.464060:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981956 is committed 00000100:00100000:3.0:1789760764.488316:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760764.627021:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981963 is committed 00000001:00080000:3.0:1789760764.648684:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981964 is committed 00000001:00080000:3.0:1789760764.662096:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981965 is committed 00002000:00010000:3.0:1789760764.662891:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60030976 -> 60035072 00002000:00010000:3.0:1789760764.662899:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 117272 -> 117280 00010000:00010000:3.0:1789760764.662916:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c3af6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60030976->60035071] (req 60030976->60035071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.662931:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c3af6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60030976->60035071] (req 60030976->60035071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.662941:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c3af6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60030976->60035071] (req 60030976->60035071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.662955:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760764.662961:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c3af6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60030976->60035071] (req 60030976->60035071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.662972:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c3af6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60030976->60035071] (req 60030976->60035071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.662997:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c3af6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60030976->60035071] (req 60030976->60035071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.663006:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c3af6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60030976->60035071] (req 60030976->60035071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.663244:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554640128:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9021us (9883us total) trans 4294981965 rc 0/0 00000100:00100000:3.0:1789760764.663257:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14667 00000100:00100000:3.0:1789760764.663270:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 707886138658 time 707899816552 state 0x2 ref 0 00000100:00100000:3.0:1789760764.666372:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554640256 00000100:00100000:3.0:1789760764.666416:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554640256 00000100:00100000:3.0:1789760764.666494:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 707886138658 time 707899816552 state 0x0 ref 1 00000100:00100000:3.0:1789760764.666513:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 707900076968 time 707899816552 state 0x1 ref 1 00000100:00100000:3.0:1789760764.666539:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 707900076968 time 707912157273 state 0x0 ref 1 00000100:00100000:3.0:1789760764.666556:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554640256:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760764.666618:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60035072, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0x1ca20 00010000:00010000:3.0:1789760764.666629:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3afd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.666650:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760764.666656:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.666664:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3afd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60035072->60039167] (req 60035072->60039167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.666682:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3afd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60035072->60039167] (req 60035072->60039167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760764.676124:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60035072 -> 60039168 00002000:00010000:3.0:1789760764.676130:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 117280 -> 117288 00010000:00010000:3.0:1789760764.676148:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3afd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60035072->60039167] (req 60035072->60039167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.676164:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3afd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60035072->60039167] (req 60035072->60039167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.676175:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3afd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60035072->60039167] (req 60035072->60039167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.676182:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760764.676197:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3afd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60035072->60039167] (req 60035072->60039167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.676210:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3afd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60035072->60039167] (req 60035072->60039167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.676237:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3afd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60035072->60039167] (req 60035072->60039167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.676246:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3afd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60035072->60039167] (req 60035072->60039167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.676487:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554640256:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9923us (10226us total) trans 4294981966 rc 0/0 00000100:00100000:3.0:1789760764.676501:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14668 00000100:00100000:3.0:1789760764.676513:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 707900076968 time 707912157273 state 0x2 ref 0 00000100:00100000:3.0:1789760764.727958:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554640640 00000100:00100000:3.0:1789760764.727973:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554640640 00000100:00100000:3.0:1789760764.728333:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 707929161508 time 707946113527 state 0x0 ref 1 00000100:00100000:3.0:1789760764.728356:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 707946373943 time 707946113527 state 0x1 ref 1 00000100:00100000:3.0:1789760764.728418:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 707946373943 time 707974034988 state 0x0 ref 1 00000100:00100000:3.0:1789760764.728434:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554640640:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760764.728503:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60047360, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0x1ca38 00010000:00010000:3.0:1789760764.728518:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3b12 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.728539:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=707s v=(0 0) 00010000:00010000:3.0:1789760764.728545:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.728553:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3b12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60047360->60051455] (req 60047360->60051455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.728572:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3b12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60047360->60051455] (req 60047360->60051455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760764.745820:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60047360 -> 60051456 00002000:00010000:3.0:1789760764.745827:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 117304 -> 117312 00010000:00010000:3.0:1789760764.745845:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3b12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60047360->60051455] (req 60047360->60051455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.745861:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3b12 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60047360->60051455] (req 60047360->60051455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.745872:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3b12 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60047360->60051455] (req 60047360->60051455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.745879:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760764.745885:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3b12 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60047360->60051455] (req 60047360->60051455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.745895:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3b12 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60047360->60051455] (req 60047360->60051455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.745924:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3b12 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60047360->60051455] (req 60047360->60051455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.745933:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3b12 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60047360->60051455] (req 60047360->60051455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.746185:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554640640:12345-192.168.203.22@tcp:4:dd.0 Request processed in 17747us (18393us total) trans 4294981969 rc 0/0 00000100:00100000:3.0:1789760764.746198:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14671 00000100:00100000:3.0:1789760764.746211:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 707946373943 time 707974034988 state 0x2 ref 0 00000100:00100000:3.0:1789760764.762192:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554640768 00000100:00100000:3.0:1789760764.762207:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554640768 00000100:00100000:3.0:1789760764.762288:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 707946373943 time 707974034988 state 0x0 ref 1 00000100:00100000:3.0:1789760764.762309:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 707974295404 time 707974034988 state 0x1 ref 1 00000100:00100000:3.0:1789760764.762510:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 707974295404 time 708008098112 state 0x0 ref 1 00000100:00100000:3.0:1789760764.814692:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554641152 00000100:00100000:3.0:1789760764.814706:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554641152 00000100:00100000:3.0:1789760764.814784:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708029433147 time 708044776443 state 0x0 ref 1 00000100:00100000:3.0:1789760764.814804:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708045036859 time 708044776443 state 0x1 ref 1 00000100:00100000:3.0:1789760764.814889:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708045036859 time 708060505807 state 0x0 ref 1 00000100:00100000:3.0:1789760764.814904:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554641152:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760764.814966:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60063744, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0x1ca58 00010000:00010000:3.0:1789760764.814979:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3b2e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.815000:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760764.815006:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.815013:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3b2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60063744->60067839] (req 60063744->60067839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.815030:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3b2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60063744->60067839] (req 60063744->60067839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760764.828996:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60063744 -> 60067840 00002000:00010000:3.0:1789760764.829005:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 117336 -> 117344 00010000:00010000:3.0:1789760764.829023:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3b2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60063744->60067839] (req 60063744->60067839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.829100:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3b2e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60063744->60067839] (req 60063744->60067839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.829165:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3b2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60063744->60067839] (req 60063744->60067839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.829174:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760764.829181:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3b2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60063744->60067839] (req 60063744->60067839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.829192:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3b2e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60063744->60067839] (req 60063744->60067839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.829218:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3b2e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60063744->60067839] (req 60063744->60067839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.829226:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3b2e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60063744->60067839] (req 60063744->60067839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760764.829440:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554641152:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14531us (15378us total) trans 4294981973 rc 0/0 00000100:00100000:3.0:1789760764.829454:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14675 00000100:00100000:3.0:1789760764.829465:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708045036859 time 708060505807 state 0x2 ref 0 00000100:00100000:3.0:1789760764.886197:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554641664 00000100:00100000:3.0:1789760764.886212:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554641664 00000100:00100000:3.0:1789760764.886290:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708099782731 time 708115767288 state 0x0 ref 1 00000100:00100000:3.0:1789760764.886321:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708116027704 time 708115767288 state 0x1 ref 1 00000100:00100000:3.0:1789760764.886359:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708116027704 time 708131976349 state 0x0 ref 1 00000100:00100000:3.0:1789760764.886374:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554641664:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760764.886484:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60080128, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0x1ca78 00010000:00010000:3.0:1789760764.886496:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3b4a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.886516:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760764.886521:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.886529:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3b4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60080128->60084223] (req 60080128->60084223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.886767:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3b4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60080128->60084223] (req 60080128->60084223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760764.893090:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981977 is committed 00000001:00080000:3.0:1789760764.903074:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981978 is committed 00000001:00080000:3.0:1789760764.915126:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981979 is committed 00000001:00080000:3.0:1789760764.933460:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981980 is committed 00000100:00100000:3.0:1789760764.977203:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554642432 00000100:00100000:3.0:1789760764.977253:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554642432 00000100:00100000:3.0:1789760764.977307:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708184379048 time 708207875006 state 0x0 ref 1 00000100:00100000:3.0:1789760764.977322:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708208135422 time 708207875006 state 0x1 ref 1 00000100:00100000:3.0:1789760764.977357:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708208135422 time 708222974829 state 0x0 ref 1 00000100:00100000:3.0:1789760764.977368:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554642432:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760764.977448:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60104704, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0x1caa8 00010000:00010000:3.0:1789760764.977457:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3b74 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.977478:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760764.977482:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760764.977488:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3b74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60104704->60108799] (req 60104704->60108799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760764.977501:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3b74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60104704->60108799] (req 60104704->60108799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760764.985714:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981983 is committed 00000001:00080000:3.0:1789760765.003602:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981984 is committed 00000001:00080000:3.0:1789760765.025894:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981985 is committed 00000100:00100000:3.0:1789760765.052795:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554642944 00000100:00100000:3.0:1789760765.052824:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554642944 00000100:00100000:3.0:1789760765.052914:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708260926814 time 708282343407 state 0x0 ref 1 00000100:00100000:3.0:1789760765.052932:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708282603823 time 708282343407 state 0x1 ref 1 00000100:00100000:3.0:1789760765.052968:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708282603823 time 708298586841 state 0x0 ref 1 00000100:00100000:3.0:1789760765.052982:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554642944:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760765.053046:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60121088, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0x1cac8 00010000:00010000:3.0:1789760765.053058:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3b90 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.053078:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760765.053083:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.053092:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3b90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60121088->60125183] (req 60121088->60125183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.053111:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c3b90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60121088->60125183] (req 60121088->60125183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760765.080153:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981988 is committed 00000001:00080000:3.0:1789760765.091471:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981989 is committed 00002000:00010000:3.0:1789760765.216655:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60162048 -> 60166144 00002000:00010000:3.0:1789760765.216663:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 117528 -> 117536 00010000:00010000:3.0:1789760765.216681:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c3bd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60162048->60166143] (req 60162048->60166143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.216697:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c3bd6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60162048->60166143] (req 60162048->60166143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.216707:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c3bd6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60162048->60166143] (req 60162048->60166143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.216716:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760765.216721:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c3bd6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60162048->60166143] (req 60162048->60166143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.216731:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c3bd6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60162048->60166143] (req 60162048->60166143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.216756:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c3bd6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60162048->60166143] (req 60162048->60166143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.216765:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c3bd6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60162048->60166143] (req 60162048->60166143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.217050:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554644224:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8322us (9568us total) trans 4294981997 rc 0/0 00000100:00100000:3.0:1789760765.217064:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14699 00000100:00100000:3.0:1789760765.217077:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708436814529 time 708454325749 state 0x2 ref 0 00000001:00080000:3.0:1789760765.239528:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294981999 is committed 00000100:00100000:3.0:1789760765.268072:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554644736 00000100:00100000:3.0:1789760765.268088:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554644736 00000100:00100000:3.0:1789760765.268169:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708479992903 time 708491171364 state 0x0 ref 1 00000100:00100000:3.0:1789760765.268199:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708491431780 time 708491171364 state 0x1 ref 1 00000100:00100000:3.0:1789760765.268291:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708491431780 time 708513908818 state 0x0 ref 1 00000100:00100000:3.0:1789760765.268307:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04560e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554644736:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760765.268422:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60178432, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0x1cb38 00010000:00010000:3.0:1789760765.268433:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3bf2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.268455:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760765.268460:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.268467:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3bf2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60178432->60182527] (req 60178432->60182527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.268484:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3bf2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60178432->60182527] (req 60178432->60182527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.341310:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554645248 00000100:00100000:3.0:1789760765.341322:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554645248 00000100:00100000:3.0:1789760765.341406:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708552779370 time 708567600506 state 0x0 ref 1 00000100:00100000:3.0:1789760765.341425:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708567860922 time 708567600506 state 0x1 ref 1 00000100:00100000:3.0:1789760765.341460:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708567860922 time 708587078339 state 0x0 ref 1 00000100:00100000:3.0:1789760765.341474:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554645248:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760765.341534:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60194816, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0x1cb58 00010000:00010000:3.0:1789760765.341546:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3c0e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.341727:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760765.341734:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.341742:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3c0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60194816->60198911] (req 60194816->60198911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.341760:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3c0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60194816->60198911] (req 60194816->60198911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760765.350855:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60194816 -> 60198912 00002000:00010000:3.0:1789760765.350862:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 117592 -> 117600 00010000:00010000:3.0:1789760765.350880:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3c0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60194816->60198911] (req 60194816->60198911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.350894:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3c0e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60194816->60198911] (req 60194816->60198911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.350905:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3c0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60194816->60198911] (req 60194816->60198911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.350914:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760765.350919:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3c0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60194816->60198911] (req 60194816->60198911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.350929:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3c0e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60194816->60198911] (req 60194816->60198911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.350954:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3c0e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60194816->60198911] (req 60194816->60198911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.350963:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3c0e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60194816->60198911] (req 60194816->60198911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.351595:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554645248:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10112us (10526us total) trans 4294982005 rc 0/0 00000100:00100000:3.0:1789760765.351609:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14707 00000100:00100000:3.0:1789760765.351622:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708567860922 time 708587078339 state 0x2 ref 0 00000100:00100000:3.0:1789760765.360919:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760765.405012:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760765.437528:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760765.524500:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760765.546962:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982019 is committed 00000001:00080000:3.0:1789760765.559972:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982020 is committed 00000100:00100000:3.0:1789760765.568675:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760765.575281:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982021 is committed 00000100:00100000:3.0:1789760765.653114:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554647808 00000100:00100000:3.0:1789760765.653128:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554647808 00000100:00100000:3.0:1789760765.653285:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708854510596 time 708876148800 state 0x0 ref 1 00000100:00100000:3.0:1789760765.653306:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708876409216 time 708876148800 state 0x1 ref 1 00000100:00100000:3.0:1789760765.653345:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708876409216 time 708898962552 state 0x0 ref 1 00000100:00100000:3.0:1789760765.653361:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554647808:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760765.653452:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60276736, mtime: 0x6aad94fc, atime: 0x0, ctime: 0x6aad94fc, blocks: 0x1cbf8 00010000:00010000:3.0:1789760765.653464:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3c9a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.653484:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760765.653489:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.653497:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3c9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60276736->60280831] (req 60276736->60280831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.653513:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3c9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60276736->60280831] (req 60276736->60280831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760765.660352:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60276736 -> 60280832 00002000:00010000:3.0:1789760765.660358:0:8442:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760764 -> 1789760765 00002000:00010000:3.0:1789760765.660361:0:8442:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760764 -> 1789760765 00002000:00010000:3.0:1789760765.660364:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 117752 -> 117760 00010000:00010000:3.0:1789760765.660404:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3c9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60276736->60280831] (req 60276736->60280831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.660420:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3c9a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60276736->60280831] (req 60276736->60280831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.660430:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3c9a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60276736->60280831] (req 60276736->60280831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.660438:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760765.660444:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3c9a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60276736->60280831] (req 60276736->60280831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.660454:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3c9a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60276736->60280831] (req 60276736->60280831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.660479:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3c9a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60276736->60280831] (req 60276736->60280831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.660487:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3c9a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60276736->60280831] (req 60276736->60280831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.660718:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554647808:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7353us (8125us total) trans 4294982025 rc 0/0 00000100:00100000:3.0:1789760765.660731:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14727 00000100:00100000:3.0:1789760765.660743:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708876409216 time 708898962552 state 0x2 ref 0 00000100:00100000:3.0:1789760765.708891:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554648320 00000100:00100000:3.0:1789760765.708905:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554648320 00000100:00100000:3.0:1789760765.708981:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708925684011 time 708939576846 state 0x0 ref 1 00000100:00100000:3.0:1789760765.709000:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708939837262 time 708939576846 state 0x1 ref 1 00000100:00100000:3.0:1789760765.709067:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708939837262 time 708954685025 state 0x0 ref 1 00000100:00100000:3.0:1789760765.709081:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554648320:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760765.709149:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60293120, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x1cc18 00010000:00010000:3.0:1789760765.709160:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cb6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.709182:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=708s v=(0 0) 00010000:00010000:3.0:1789760765.709198:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.709206:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60293120->60297215] (req 60293120->60297215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.709224:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60293120->60297215] (req 60293120->60297215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760765.716510:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60293120 -> 60297216 00002000:00010000:3.0:1789760765.716518:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 117784 -> 117792 00010000:00010000:3.0:1789760765.716537:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60293120->60297215] (req 60293120->60297215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.716552:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cb6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60293120->60297215] (req 60293120->60297215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.716563:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cb6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60293120->60297215] (req 60293120->60297215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.716571:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760765.716578:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cb6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60293120->60297215] (req 60293120->60297215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.716588:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cb6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60293120->60297215] (req 60293120->60297215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.716616:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cb6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60293120->60297215] (req 60293120->60297215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.716625:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cb6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60293120->60297215] (req 60293120->60297215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.716883:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554648320:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7797us (8172us total) trans 4294982029 rc 0/0 00000100:00100000:3.0:1789760765.716898:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14731 00000100:00100000:3.0:1789760765.716911:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708939837262 time 708954685025 state 0x2 ref 0 00000100:00100000:3.0:1789760765.778510:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554648832 00000100:00100000:3.0:1789760765.778519:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554648832 00000100:00100000:3.0:1789760765.778591:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 708990318332 time 709008211450 state 0x0 ref 1 00000100:00100000:3.0:1789760765.778607:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709008471866 time 709008211450 state 0x1 ref 1 00000100:00100000:3.0:1789760765.778630:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709008471866 time 709024248164 state 0x0 ref 1 00000100:00100000:3.0:1789760765.778641:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554648832:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760765.778690:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60309504, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x1cc38 00010000:00010000:3.0:1789760765.778700:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cd2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.778715:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760765.778720:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.778726:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60309504->60313599] (req 60309504->60313599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.778740:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60309504->60313599] (req 60309504->60313599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760765.791123:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60309504 -> 60313600 00002000:00010000:3.0:1789760765.791129:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 117816 -> 117824 00010000:00010000:3.0:1789760765.791142:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60309504->60313599] (req 60309504->60313599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.791155:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cd2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60309504->60313599] (req 60309504->60313599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.791165:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cd2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60309504->60313599] (req 60309504->60313599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.791173:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760765.791178:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cd2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60309504->60313599] (req 60309504->60313599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.791187:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cd2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60309504->60313599] (req 60309504->60313599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.791218:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cd2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60309504->60313599] (req 60309504->60313599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.791227:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cd2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60309504->60313599] (req 60309504->60313599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.792199:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554648832:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13552us (13844us total) trans 4294982033 rc 0/0 00000100:00100000:3.0:1789760765.792211:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14735 00000100:00100000:3.0:1789760765.792222:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709008471866 time 709024248164 state 0x2 ref 0 00000100:00100000:3.0:1789760765.853704:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554649344 00000100:00100000:3.0:1789760765.853715:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554649344 00000100:00100000:3.0:1789760765.853776:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709063704889 time 709077335163 state 0x0 ref 1 00000100:00100000:3.0:1789760765.853813:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709077595579 time 709077335163 state 0x1 ref 1 00000100:00100000:3.0:1789760765.853845:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709077595579 time 709099463603 state 0x0 ref 1 00000100:00100000:3.0:1789760765.853858:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554649344:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760765.853902:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60325888, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x1cc58 00010000:00010000:3.0:1789760765.853912:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.853928:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760765.853933:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.853939:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60325888->60329983] (req 60325888->60329983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.853952:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60325888->60329983] (req 60325888->60329983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760765.859234:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60325888 -> 60329984 00002000:00010000:3.0:1789760765.859240:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 117848 -> 117856 00010000:00010000:3.0:1789760765.859254:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60325888->60329983] (req 60325888->60329983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.859271:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60325888->60329983] (req 60325888->60329983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.859281:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60325888->60329983] (req 60325888->60329983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.859289:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760765.859293:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60325888->60329983] (req 60325888->60329983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.859302:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60325888->60329983] (req 60325888->60329983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.859333:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60325888->60329983] (req 60325888->60329983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.859341:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3cee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60325888->60329983] (req 60325888->60329983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.859461:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554649344:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5602us (6381us total) trans 4294982037 rc 0/0 00000100:00100000:3.0:1789760765.859472:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14739 00000100:00100000:3.0:1789760765.859481:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709077595579 time 709099463603 state 0x2 ref 0 00000100:00100000:3.0:1789760765.910823:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554649856 00000100:00100000:3.0:1789760765.910835:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554649856 00000100:00100000:3.0:1789760765.910888:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709118831711 time 709136370285 state 0x0 ref 1 00000100:00100000:3.0:1789760765.910904:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709136630701 time 709136370285 state 0x1 ref 1 00000100:00100000:3.0:1789760765.910937:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709136630701 time 709156555617 state 0x0 ref 1 00000100:00100000:3.0:1789760765.910949:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554649856:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760765.911010:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60342272, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x1cc78 00010000:00010000:3.0:1789760765.911022:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3d0a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.911038:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760765.911043:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.911049:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3d0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60342272->60346367] (req 60342272->60346367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.911064:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3d0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60342272->60346367] (req 60342272->60346367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760765.921321:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60342272 -> 60346368 00002000:00010000:3.0:1789760765.921327:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 117880 -> 117888 00010000:00010000:3.0:1789760765.921340:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3d0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60342272->60346367] (req 60342272->60346367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.921353:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3d0a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60342272->60346367] (req 60342272->60346367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.921367:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3d0a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60342272->60346367] (req 60342272->60346367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.921374:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760765.921409:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3d0a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60342272->60346367] (req 60342272->60346367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.921418:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3d0a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60342272->60346367] (req 60342272->60346367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.921438:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3d0a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60342272->60346367] (req 60342272->60346367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.921446:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3d0a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60342272->60346367] (req 60342272->60346367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.921538:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554649856:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10585us (10863us total) trans 4294982041 rc 0/0 00000100:00100000:3.0:1789760765.921549:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14743 00000100:00100000:3.0:1789760765.921569:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709136630701 time 709156555617 state 0x2 ref 0 00000100:00100000:3.0:1789760765.975287:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554650240 00000100:00100000:3.0:1789760765.975299:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554650240 00000100:00100000:3.0:1789760765.975352:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709184024956 time 709204170946 state 0x0 ref 1 00000100:00100000:3.0:1789760765.975367:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709204431362 time 709204170946 state 0x1 ref 1 00000100:00100000:3.0:1789760765.975420:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709204431362 time 709221037868 state 0x0 ref 1 00000100:00100000:3.0:1789760765.975432:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554650240:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760765.975482:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60354560, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x1cc90 00010000:00010000:3.0:1789760765.975493:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d1f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.975509:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760765.975514:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760765.975520:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60354560->60358655] (req 60354560->60358655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.975534:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60354560->60358655] (req 60354560->60358655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760765.987056:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60354560 -> 60358656 00002000:00010000:3.0:1789760765.987062:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 117904 -> 117912 00010000:00010000:3.0:1789760765.987078:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60354560->60358655] (req 60354560->60358655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.987092:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d1f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60354560->60358655] (req 60354560->60358655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.987102:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60354560->60358655] (req 60354560->60358655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.987115:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760765.987120:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60354560->60358655] (req 60354560->60358655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.987130:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d1f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60354560->60358655] (req 60354560->60358655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.987150:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d1f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60354560->60358655] (req 60354560->60358655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760765.987159:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d1f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60354560->60358655] (req 60354560->60358655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760765.987274:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554650240:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11838us (12609us total) trans 4294982044 rc 0/0 00000100:00100000:3.0:1789760765.987285:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14746 00000100:00100000:3.0:1789760765.987294:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709204431362 time 709221037868 state 0x2 ref 0 00000100:00100000:3.0:1789760766.005506:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760766.051210:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760766.066623:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554650880 00000100:00100000:3.0:1789760766.066635:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554650880 00000100:00100000:3.0:1789760766.066698:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709283490109 time 709297132435 state 0x0 ref 1 00000100:00100000:3.0:1789760766.066716:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709297392851 time 709297132435 state 0x1 ref 1 00000100:00100000:3.0:1789760766.066760:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709297392851 time 709312378021 state 0x0 ref 1 00000100:00100000:3.0:1789760766.066774:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554650880:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760766.066831:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60375040, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x1ccb8 00010000:00010000:3.0:1789760766.066841:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d42 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.066857:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760766.066862:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.066869:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60375040->60379135] (req 60375040->60379135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.066883:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60375040->60379135] (req 60375040->60379135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760766.084460:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60375040 -> 60379136 00002000:00010000:3.0:1789760766.084466:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 117944 -> 117952 00010000:00010000:3.0:1789760766.084480:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60375040->60379135] (req 60375040->60379135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.084495:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d42 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60375040->60379135] (req 60375040->60379135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.084504:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60375040->60379135] (req 60375040->60379135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.084518:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760766.084524:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60375040->60379135] (req 60375040->60379135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.084534:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d42 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60375040->60379135] (req 60375040->60379135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.084554:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d42 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60375040->60379135] (req 60375040->60379135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.084562:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d42 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60375040->60379135] (req 60375040->60379135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.084674:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554650880:12345-192.168.203.22@tcp:4:dd.0 Request processed in 17896us (18140us total) trans 4294982049 rc 0/0 00000100:00100000:3.0:1789760766.084686:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14751 00000100:00100000:3.0:1789760766.084698:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709297392851 time 709312378021 state 0x2 ref 0 00000100:00100000:3.0:1789760766.125776:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554651264 00000100:00100000:3.0:1789760766.125786:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554651264 00000100:00100000:3.0:1789760766.125852:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709336374312 time 709352847341 state 0x0 ref 1 00000100:00100000:3.0:1789760766.125879:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709353107757 time 709352847341 state 0x1 ref 1 00000100:00100000:3.0:1789760766.125914:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709353107757 time 709371531715 state 0x0 ref 1 00000100:00100000:3.0:1789760766.125932:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554651264:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760766.125980:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60387328, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x1ccd0 00010000:00010000:3.0:1789760766.125989:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d57 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.126007:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760766.126012:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.126019:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60387328->60391423] (req 60387328->60391423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.126034:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60387328->60391423] (req 60387328->60391423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760766.137217:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60387328 -> 60391424 00002000:00010000:3.0:1789760766.137224:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 117968 -> 117976 00010000:00010000:3.0:1789760766.137247:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60387328->60391423] (req 60387328->60391423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.137261:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d57 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60387328->60391423] (req 60387328->60391423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.137270:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60387328->60391423] (req 60387328->60391423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.137277:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760766.137282:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60387328->60391423] (req 60387328->60391423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.137295:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d57 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60387328->60391423] (req 60387328->60391423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.137315:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d57 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60387328->60391423] (req 60387328->60391423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.137323:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d57 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60387328->60391423] (req 60387328->60391423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.137474:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554651264:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11539us (12151us total) trans 4294982052 rc 0/0 00000100:00100000:3.0:1789760766.137486:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14754 00000100:00100000:3.0:1789760766.137496:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709353107757 time 709371531715 state 0x2 ref 0 00000100:00100000:3.0:1789760766.143833:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554651392 00000100:00100000:3.0:1789760766.143851:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554651392 00000100:00100000:3.0:1789760766.143917:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709353107757 time 709371531715 state 0x0 ref 1 00000100:00100000:3.0:1789760766.143933:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709371792131 time 709371531715 state 0x1 ref 1 00000100:00100000:3.0:1789760766.143984:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709371792131 time 709389599804 state 0x0 ref 1 00000100:00100000:3.0:1789760766.144005:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554651392:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760766.144078:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60391424, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x1ccd8 00010000:00010000:3.0:1789760766.144093:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d5e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.144113:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760766.144120:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.144133:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60391424->60395519] (req 60391424->60395519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.144156:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60391424->60395519] (req 60391424->60395519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760766.151524:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60391424 -> 60395520 00002000:00010000:3.0:1789760766.151529:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 117976 -> 117984 00010000:00010000:3.0:1789760766.151542:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60391424->60395519] (req 60391424->60395519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.151554:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d5e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60391424->60395519] (req 60391424->60395519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.151574:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60391424->60395519] (req 60391424->60395519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.151581:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760766.151586:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60391424->60395519] (req 60391424->60395519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.151595:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d5e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60391424->60395519] (req 60391424->60395519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.151615:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d5e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60391424->60395519] (req 60391424->60395519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.151623:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d5e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60391424->60395519] (req 60391424->60395519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.151716:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554651392:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7712us (8022us total) trans 4294982053 rc 0/0 00000100:00100000:3.0:1789760766.151727:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14755 00000100:00100000:3.0:1789760766.151737:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709371792131 time 709389599804 state 0x2 ref 0 00000100:00100000:3.0:1789760766.198082:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554651776 00000100:00100000:3.0:1789760766.198092:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554651776 00000100:00100000:3.0:1789760766.198146:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709406567698 time 709428695995 state 0x0 ref 1 00000100:00100000:3.0:1789760766.198162:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709428956411 time 709428695995 state 0x1 ref 1 00000100:00100000:3.0:1789760766.198201:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709428956411 time 709443818948 state 0x0 ref 1 00000100:00100000:3.0:1789760766.198213:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554651776:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760766.198254:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60403712, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x1ccf0 00010000:00010000:3.0:1789760766.198263:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d73 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.198290:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760766.198295:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.198301:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60403712->60407807] (req 60403712->60407807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.198315:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60403712->60407807] (req 60403712->60407807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760766.206763:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60403712 -> 60407808 00002000:00010000:3.0:1789760766.206768:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 118000 -> 118008 00010000:00010000:3.0:1789760766.206781:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60403712->60407807] (req 60403712->60407807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.206795:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d73 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60403712->60407807] (req 60403712->60407807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.206814:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d73 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60403712->60407807] (req 60403712->60407807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.206821:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760766.206826:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d73 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60403712->60407807] (req 60403712->60407807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.206834:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d73 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60403712->60407807] (req 60403712->60407807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.206861:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d73 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60403712->60407807] (req 60403712->60407807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.206874:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d73 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60403712->60407807] (req 60403712->60407807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.206969:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554651776:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8753us (9131us total) trans 4294982056 rc 0/0 00000100:00100000:3.0:1789760766.206980:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14758 00000100:00100000:3.0:1789760766.206990:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709428956411 time 709443818948 state 0x2 ref 0 00000100:00100000:3.0:1789760766.211574:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554651904 00000100:00100000:3.0:1789760766.211585:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554651904 00000100:00100000:3.0:1789760766.211640:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709428956411 time 709443818948 state 0x0 ref 1 00000100:00100000:3.0:1789760766.211655:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709444079364 time 709443818948 state 0x1 ref 1 00000100:00100000:3.0:1789760766.211687:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709444079364 time 709457305284 state 0x0 ref 1 00000100:00100000:3.0:1789760766.211699:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554651904:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760766.211745:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60407808, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x1ccf8 00010000:00010000:3.0:1789760766.211754:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d7a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.211769:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760766.211774:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.211780:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60407808->60411903] (req 60407808->60411903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.211813:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60407808->60411903] (req 60407808->60411903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760766.221906:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60407808 -> 60411904 00002000:00010000:3.0:1789760766.221911:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 118008 -> 118016 00010000:00010000:3.0:1789760766.221925:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60407808->60411903] (req 60407808->60411903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.221938:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d7a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60407808->60411903] (req 60407808->60411903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.221947:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d7a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60407808->60411903] (req 60407808->60411903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.221954:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760766.221959:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d7a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60407808->60411903] (req 60407808->60411903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.221967:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d7a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60407808->60411903] (req 60407808->60411903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.221989:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d7a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60407808->60411903] (req 60407808->60411903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.221996:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d7a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60407808->60411903] (req 60407808->60411903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.222095:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554651904:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10392us (10792us total) trans 4294982057 rc 0/0 00000100:00100000:3.0:1789760766.222106:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14759 00000100:00100000:3.0:1789760766.222115:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709444079364 time 709457305284 state 0x2 ref 0 00002000:00010000:3.0:1789760766.248467:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60416000 -> 60420096 00002000:00010000:3.0:1789760766.248473:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 118024 -> 118032 00010000:00010000:3.0:1789760766.248486:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c3d88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60416000->60420095] (req 60416000->60420095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.248499:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c3d88 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60416000->60420095] (req 60416000->60420095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.248508:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c3d88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60416000->60420095] (req 60416000->60420095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.248516:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760766.248520:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c3d88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60416000->60420095] (req 60416000->60420095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.248529:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c3d88 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60416000->60420095] (req 60416000->60420095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.248548:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c3d88 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60416000->60420095] (req 60416000->60420095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.248573:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c3d88 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60416000->60420095] (req 60416000->60420095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.248672:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554652160:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7667us (7936us total) trans 4294982059 rc 0/0 00000100:00100000:3.0:1789760766.248683:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14761 00000100:00100000:3.0:1789760766.248693:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709473457544 time 709486609096 state 0x2 ref 0 00000100:00100000:3.0:1789760766.255701:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554652288 00000100:00100000:3.0:1789760766.255711:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554652288 00000100:00100000:3.0:1789760766.255764:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709473457544 time 709486609096 state 0x0 ref 1 00000100:00100000:3.0:1789760766.255780:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709486869512 time 709486609096 state 0x1 ref 1 00000100:00100000:3.0:1789760766.255822:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709486869512 time 709501439939 state 0x0 ref 1 00000100:00100000:3.0:1789760766.255835:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554652288:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760766.255894:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60420096, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x1cd10 00010000:00010000:3.0:1789760766.255904:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d8f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.255920:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760766.255924:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.255931:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60420096->60424191] (req 60420096->60424191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.255952:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3d8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60420096->60424191] (req 60420096->60424191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.263922:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982060 is committed 00000100:00100000:3.0:1789760766.276320:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554652416 00000100:00100000:3.0:1789760766.276331:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554652416 00000100:00100000:3.0:1789760766.276424:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709486869512 time 709501439939 state 0x0 ref 1 00000100:00100000:3.0:1789760766.276440:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709501700355 time 709501439939 state 0x1 ref 1 00000100:00100000:3.0:1789760766.276473:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709501700355 time 709522092010 state 0x0 ref 1 00000100:00100000:3.0:1789760766.276486:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554652416:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760766.276530:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60424192, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x1cd18 00010000:00010000:3.0:1789760766.276539:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3d96 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.276554:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760766.276558:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.276565:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3d96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60424192->60428287] (req 60424192->60428287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.276579:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3d96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60424192->60428287] (req 60424192->60428287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760766.283132:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60424192 -> 60428288 00002000:00010000:3.0:1789760766.283137:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 118040 -> 118048 00010000:00010000:3.0:1789760766.283150:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3d96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60424192->60428287] (req 60424192->60428287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.283161:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3d96 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60424192->60428287] (req 60424192->60428287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.283170:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3d96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60424192->60428287] (req 60424192->60428287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.283187:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760766.283191:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3d96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60424192->60428287] (req 60424192->60428287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.283199:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3d96 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60424192->60428287] (req 60424192->60428287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.283217:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3d96 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60424192->60428287] (req 60424192->60428287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.283224:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3d96 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60424192->60428287] (req 60424192->60428287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.283318:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554652416:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6829us (7238us total) trans 4294982061 rc 0/0 00000100:00100000:3.0:1789760766.283328:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14763 00000100:00100000:3.0:1789760766.283337:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709501700355 time 709522092010 state 0x2 ref 0 00000100:00100000:3.0:1789760766.324873:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554652800 00000100:00100000:3.0:1789760766.324883:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554652800 00000100:00100000:3.0:1789760766.324935:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709535914113 time 709557539649 state 0x0 ref 1 00000100:00100000:3.0:1789760766.324950:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709557800065 time 709557539649 state 0x1 ref 1 00000100:00100000:3.0:1789760766.324980:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709557800065 time 709570598304 state 0x0 ref 1 00000100:00100000:3.0:1789760766.324992:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554652800:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760766.325038:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60436480, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x1cd30 00010000:00010000:3.0:1789760766.325048:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3dab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.325065:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760766.325070:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.325076:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3dab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60436480->60440575] (req 60436480->60440575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.325091:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3dab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60436480->60440575] (req 60436480->60440575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.331716:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982064 is committed 00000100:00100000:3.0:1789760766.338103:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554652928 00000100:00100000:3.0:1789760766.338113:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554652928 00000100:00100000:3.0:1789760766.338165:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709557800065 time 709570598304 state 0x0 ref 1 00000100:00100000:3.0:1789760766.338180:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709570858720 time 709570598304 state 0x1 ref 1 00000100:00100000:3.0:1789760766.338212:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709570858720 time 709583829844 state 0x0 ref 1 00000100:00100000:3.0:1789760766.338225:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554652928:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760766.338272:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60440576, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x1cd38 00010000:00010000:3.0:1789760766.338291:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3db2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.338308:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760766.338313:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.338319:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3db2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60440576->60444671] (req 60440576->60444671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.338332:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3db2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60440576->60444671] (req 60440576->60444671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760766.344315:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60440576 -> 60444672 00002000:00010000:3.0:1789760766.344321:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 118072 -> 118080 00010000:00010000:3.0:1789760766.344334:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3db2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60440576->60444671] (req 60440576->60444671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.344348:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3db2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60440576->60444671] (req 60440576->60444671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.344364:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3db2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60440576->60444671] (req 60440576->60444671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.344372:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760766.344399:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3db2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60440576->60444671] (req 60440576->60444671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.344410:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3db2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60440576->60444671] (req 60440576->60444671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.344431:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3db2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60440576->60444671] (req 60440576->60444671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.344439:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3db2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60440576->60444671] (req 60440576->60444671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.344539:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554652928:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6311us (6527us total) trans 4294982065 rc 0/0 00000100:00100000:3.0:1789760766.344550:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14767 00000100:00100000:3.0:1789760766.344570:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709570858720 time 709583829844 state 0x2 ref 0 00000001:00080000:3.0:1789760766.359601:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982066 is committed 00000001:00080000:3.0:1789760766.372294:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982067 is committed 00000100:00100000:3.0:1789760766.377126:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554653312 00000100:00100000:3.0:1789760766.377137:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554653312 00000100:00100000:3.0:1789760766.377195:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709597425016 time 709611048610 state 0x0 ref 1 00000100:00100000:3.0:1789760766.377212:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709611309026 time 709611048610 state 0x1 ref 1 00000100:00100000:3.0:1789760766.377248:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709611309026 time 709622866375 state 0x0 ref 1 00000100:00100000:3.0:1789760766.377261:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554653312:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760766.377306:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60452864, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x1cd50 00010000:00010000:3.0:1789760766.377316:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3dc7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.377332:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760766.377337:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.377343:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3dc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60452864->60456959] (req 60452864->60456959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.377357:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3dc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60452864->60456959] (req 60452864->60456959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.385143:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982068 is committed 00000001:00080000:3.0:1789760766.401944:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982069 is committed 00000001:00080000:3.0:1789760766.427151:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982071 is committed 00000100:00100000:3.0:1789760766.440209:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554653952 00000100:00100000:3.0:1789760766.440220:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554653952 00000100:00100000:3.0:1789760766.440271:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709668486180 time 709677197936 state 0x0 ref 1 00000100:00100000:3.0:1789760766.440287:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709677458352 time 709677197936 state 0x1 ref 1 00000100:00100000:3.0:1789760766.440350:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709677458352 time 709685968229 state 0x0 ref 1 00000100:00100000:3.0:1789760766.440363:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554653952:12345-192.168.203.22@tcp:4:dd.0 00000100:00100000:3.0:1789760766.463236:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554654208 00000100:00100000:3.0:1789760766.463247:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554654208 00000100:00100000:3.0:1789760766.463302:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709686228645 time 709696328671 state 0x0 ref 1 00000100:00100000:3.0:1789760766.463317:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709696589087 time 709696328671 state 0x1 ref 1 00000100:00100000:3.0:1789760766.463408:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709696589087 time 709708997101 state 0x0 ref 1 00000100:00100000:3.0:1789760766.463421:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04562680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554654208:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760766.463468:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60481536, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x1cd88 00010000:00010000:3.0:1789760766.463476:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3df8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.463490:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760766.463494:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.463500:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3df8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60481536->60485631] (req 60481536->60485631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.463512:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3df8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60481536->60485631] (req 60481536->60485631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.480051:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554654336 00000100:00100000:3.0:1789760766.480060:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554654336 00000100:00100000:3.0:1789760766.480111:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709696589087 time 709708997101 state 0x0 ref 1 00000100:00100000:3.0:1789760766.480129:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709709257517 time 709708997101 state 0x1 ref 1 00000100:00100000:3.0:1789760766.480158:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709709257517 time 709725776445 state 0x0 ref 1 00000100:00100000:3.0:1789760766.480171:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554654336:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760766.480542:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60485632, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x1cd90 00010000:00010000:3.0:1789760766.480551:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3dff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.480767:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760766.480772:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.480778:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3dff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60485632->60489727] (req 60485632->60489727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.480793:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3dff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60485632->60489727] (req 60485632->60489727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760766.484845:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60485632 -> 60489728 00002000:00010000:3.0:1789760766.484913:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 118160 -> 118168 00010000:00010000:3.0:1789760766.484929:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3dff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60485632->60489727] (req 60485632->60489727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.484941:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3dff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60485632->60489727] (req 60485632->60489727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.484950:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3dff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60485632->60489727] (req 60485632->60489727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.484958:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760766.484962:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3dff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60485632->60489727] (req 60485632->60489727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.484971:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3dff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60485632->60489727] (req 60485632->60489727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.484989:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3dff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60485632->60489727] (req 60485632->60489727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.484996:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3dff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60485632->60489727] (req 60485632->60489727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.485141:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554654336:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4966us (5243us total) trans 4294982076 rc 0/0 00000100:00100000:3.0:1789760766.485150:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14778 00000100:00100000:3.0:1789760766.485159:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709709257517 time 709725776445 state 0x2 ref 0 00000001:00080000:3.0:1789760766.496312:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982077 is committed 00000001:00080000:3.0:1789760766.522720:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982079 is committed 00000100:00100000:3.0:1789760766.526890:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554654848 00000100:00100000:3.0:1789760766.526900:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554654848 00000100:00100000:3.0:1789760766.526959:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709745838757 time 709762307541 state 0x0 ref 1 00000100:00100000:3.0:1789760766.526977:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709762567957 time 709762307541 state 0x1 ref 1 00000100:00100000:3.0:1789760766.527012:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709762567957 time 709772630167 state 0x0 ref 1 00000100:00100000:3.0:1789760766.527024:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04560a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554654848:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760766.527074:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60502016, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x1cdb0 00010000:00010000:3.0:1789760766.527084:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3e1b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.527100:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760766.527104:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.527111:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3e1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60502016->60506111] (req 60502016->60506111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.527124:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c3e1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60502016->60506111] (req 60502016->60506111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760766.571423:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982083 is committed 00000001:00080000:3.0:1789760766.585341:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982084 is committed 00000001:00080000:3.0:1789760766.598699:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982085 is committed 00000001:00080000:3.0:1789760766.607681:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982086 is committed 00000001:00080000:3.0:1789760766.623430:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982087 is committed 00000100:00100000:3.0:1789760766.629008:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554655872 00000100:00100000:3.0:1789760766.629017:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554655872 00000100:00100000:3.0:1789760766.629065:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709848969782 time 709864898496 state 0x0 ref 1 00000100:00100000:3.0:1789760766.629080:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709865158912 time 709864898496 state 0x1 ref 1 00000100:00100000:3.0:1789760766.629109:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709865158912 time 709874728000 state 0x0 ref 1 00000100:00100000:3.0:1789760766.629121:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554655872:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760766.629161:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60534784, mtime: 0x6aad94fd, atime: 0x0, ctime: 0x6aad94fd, blocks: 0x1cdf0 00010000:00010000:3.0:1789760766.629170:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e53 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.629195:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760766.629199:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.629205:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60534784->60538879] (req 60534784->60538879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.629219:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60534784->60538879] (req 60534784->60538879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760766.633904:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60534784 -> 60538880 00002000:00010000:3.0:1789760766.633909:0:15101:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760765 -> 1789760766 00002000:00010000:3.0:1789760766.633912:0:15101:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760765 -> 1789760766 00002000:00010000:3.0:1789760766.633914:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 118256 -> 118264 00010000:00010000:3.0:1789760766.633928:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60534784->60538879] (req 60534784->60538879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.633942:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e53 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60534784->60538879] (req 60534784->60538879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.633951:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e53 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60534784->60538879] (req 60534784->60538879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.633964:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760766.633969:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e53 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60534784->60538879] (req 60534784->60538879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.633978:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e53 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60534784->60538879] (req 60534784->60538879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.633997:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e53 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60534784->60538879] (req 60534784->60538879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.634005:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e53 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60534784->60538879] (req 60534784->60538879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.634197:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554655872:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5072us (5393us total) trans 4294982088 rc 0/0 00000100:00100000:3.0:1789760766.634209:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14790 00000100:00100000:3.0:1789760766.634219:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709865158912 time 709874728000 state 0x2 ref 0 00000100:00100000:3.0:1789760766.680560:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554656384 00000100:00100000:3.0:1789760766.680570:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554656384 00000100:00100000:3.0:1789760766.680625:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709901735024 time 709915873777 state 0x0 ref 1 00000100:00100000:3.0:1789760766.680643:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709916134193 time 709915873777 state 0x1 ref 1 00000100:00100000:3.0:1789760766.680708:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709916134193 time 709926326427 state 0x0 ref 1 00000100:00100000:3.0:1789760766.680725:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554656384:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760766.680779:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60551168, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x1ce10 00010000:00010000:3.0:1789760766.680789:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e6f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.680813:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760766.680817:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.680823:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60551168->60555263] (req 60551168->60555263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.680837:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60551168->60555263] (req 60551168->60555263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760766.684741:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60551168 -> 60555264 00002000:00010000:3.0:1789760766.684747:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 118288 -> 118296 00010000:00010000:3.0:1789760766.684761:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60551168->60555263] (req 60551168->60555263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.684774:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e6f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60551168->60555263] (req 60551168->60555263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.684783:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60551168->60555263] (req 60551168->60555263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.684791:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760766.684806:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60551168->60555263] (req 60551168->60555263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.684815:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e6f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60551168->60555263] (req 60551168->60555263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.684835:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e6f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60551168->60555263] (req 60551168->60555263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.684843:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e6f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60551168->60555263] (req 60551168->60555263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.684934:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554656384:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4207us (5099us total) trans 4294982092 rc 0/0 00000100:00100000:3.0:1789760766.684945:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14794 00000100:00100000:3.0:1789760766.684954:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709916134193 time 709926326427 state 0x2 ref 0 00000100:00100000:3.0:1789760766.728997:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554656896 00000100:00100000:3.0:1789760766.729008:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554656896 00000100:00100000:3.0:1789760766.729060:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709945995143 time 709956405749 state 0x0 ref 1 00000100:00100000:3.0:1789760766.729075:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709956666165 time 709956405749 state 0x1 ref 1 00000100:00100000:3.0:1789760766.729105:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709956666165 time 709974723574 state 0x0 ref 1 00000100:00100000:3.0:1789760766.729116:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554656896:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760766.729163:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60567552, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x1ce30 00010000:00010000:3.0:1789760766.729182:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e8b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.729199:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=709s v=(0 0) 00010000:00010000:3.0:1789760766.729203:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.729209:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60567552->60571647] (req 60567552->60571647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.729222:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60567552->60571647] (req 60567552->60571647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760766.735746:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60567552 -> 60571648 00002000:00010000:3.0:1789760766.735751:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 118320 -> 118328 00010000:00010000:3.0:1789760766.735764:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60567552->60571647] (req 60567552->60571647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.735778:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e8b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60567552->60571647] (req 60567552->60571647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.736240:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60567552->60571647] (req 60567552->60571647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.736251:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760766.736255:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60567552->60571647] (req 60567552->60571647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.736265:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e8b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60567552->60571647] (req 60567552->60571647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.736286:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e8b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60567552->60571647] (req 60567552->60571647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.736295:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3e8b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60567552->60571647] (req 60567552->60571647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.736487:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554656896:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7365us (7710us total) trans 4294982096 rc 0/0 00000100:00100000:3.0:1789760766.736498:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14798 00000100:00100000:3.0:1789760766.736507:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 709956666165 time 709974723574 state 0x2 ref 0 00000100:00100000:3.0:1789760766.780665:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554657408 00000100:00100000:3.0:1789760766.780676:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554657408 00000100:00100000:3.0:1789760766.780732:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710002567426 time 710013949273 state 0x0 ref 1 00000100:00100000:3.0:1789760766.780747:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710014209689 time 710013949273 state 0x1 ref 1 00000100:00100000:3.0:1789760766.780778:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710014209689 time 710026396578 state 0x0 ref 1 00000100:00100000:3.0:1789760766.780928:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554657408:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760766.780974:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60583936, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x1ce50 00010000:00010000:3.0:1789760766.780984:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3ea7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.781002:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760766.781007:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.781014:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3ea7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60583936->60588031] (req 60583936->60588031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.781029:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3ea7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60583936->60588031] (req 60583936->60588031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760766.786088:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60583936 -> 60588032 00002000:00010000:3.0:1789760766.786094:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 118352 -> 118360 00010000:00010000:3.0:1789760766.786108:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3ea7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60583936->60588031] (req 60583936->60588031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.786122:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3ea7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60583936->60588031] (req 60583936->60588031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.786130:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3ea7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60583936->60588031] (req 60583936->60588031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.786137:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760766.786141:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3ea7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60583936->60588031] (req 60583936->60588031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.786150:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3ea7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60583936->60588031] (req 60583936->60588031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.786170:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3ea7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60583936->60588031] (req 60583936->60588031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.786249:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3ea7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60583936->60588031] (req 60583936->60588031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.786404:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554657408:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5456us (6034us total) trans 4294982100 rc 0/0 00000100:00100000:3.0:1789760766.786415:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14802 00000100:00100000:3.0:1789760766.786425:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710014209689 time 710026396578 state 0x2 ref 0 00000100:00100000:3.0:1789760766.819062:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554657792 00000100:00100000:3.0:1789760766.819072:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554657792 00000100:00100000:3.0:1789760766.819128:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710038647780 time 710051292229 state 0x0 ref 1 00000100:00100000:3.0:1789760766.819222:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710051552645 time 710051292229 state 0x1 ref 1 00000100:00100000:3.0:1789760766.819257:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710051552645 time 710064875127 state 0x0 ref 1 00000100:00100000:3.0:1789760766.819272:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554657792:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760766.819321:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60596224, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x1ce68 00010000:00010000:3.0:1789760766.819331:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3ebc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.819348:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760766.819353:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.819359:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3ebc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60596224->60600319] (req 60596224->60600319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.819400:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3ebc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60596224->60600319] (req 60596224->60600319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760766.828111:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60596224 -> 60600320 00002000:00010000:3.0:1789760766.828117:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 118376 -> 118384 00010000:00010000:3.0:1789760766.828130:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3ebc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60596224->60600319] (req 60596224->60600319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.828142:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3ebc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60596224->60600319] (req 60596224->60600319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.828151:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3ebc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60596224->60600319] (req 60596224->60600319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.828157:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760766.828162:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3ebc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60596224->60600319] (req 60596224->60600319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.828171:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3ebc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60596224->60600319] (req 60596224->60600319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.828192:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3ebc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60596224->60600319] (req 60596224->60600319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.828200:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3ebc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60596224->60600319] (req 60596224->60600319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.828300:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554657792:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9027us (9398us total) trans 4294982103 rc 0/0 00000100:00100000:3.0:1789760766.828310:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14805 00000100:00100000:3.0:1789760766.828322:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710051552645 time 710064875127 state 0x2 ref 0 00000100:00100000:3.0:1789760766.839230:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554657920 00000100:00100000:3.0:1789760766.839240:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554657920 00000100:00100000:3.0:1789760766.839294:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710051552645 time 710064875127 state 0x0 ref 1 00000100:00100000:3.0:1789760766.839309:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710065135543 time 710064875127 state 0x1 ref 1 00000100:00100000:3.0:1789760766.839423:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710065135543 time 710085040523 state 0x0 ref 1 00000100:00100000:3.0:1789760766.839436:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554657920:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760766.839484:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60600320, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x1ce70 00010000:00010000:3.0:1789760766.839493:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3ec3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.839510:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760766.839514:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760766.839520:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3ec3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60600320->60604415] (req 60600320->60604415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760766.839545:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c3ec3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60600320->60604415] (req 60600320->60604415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760766.866851:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cc07bcd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5711db52-8702-4d6b-b42c-8543e9bf1cb2:3646:1876699573910144:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.866904:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760766.866953:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cc07bcc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573910656:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.866979:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760766.866999:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cc07bcdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573911168:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.867020:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760766.867037:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb044b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573911680:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.867057:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760766.867079:0:3646:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:3.0:1789760766.867084:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cc07bcce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3646:1876699573910272:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.867104:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760766.867121:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cc07bcf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3646:1876699573910784:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.867140:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760766.867157:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cc07bcc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3646:1876699573911296:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.867226:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760766.867255:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cc07bcca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3648:1876699573910400:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.867277:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760766.867294:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cc07bced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3648:1876699573910912:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.867313:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760766.867329:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cc07bcdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3648:1876699573911424:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.867349:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760766.867365:0:3648:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:3.0:1789760766.867371:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cc07bcf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3648:1876699573910528:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.867420:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760766.867432:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cc07bce300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3648:1876699573911040:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.867451:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760766.867469:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cc07bcfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3648:1876699573911552:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.867488:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760766.867520:0:6514:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573910144 00000100:00100000:3.0:1789760766.867527:0:6514:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573910144 00000100:00100000:3.0:1789760766.870069:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 465 00000100:00100000:3.0:1789760766.870080:0:6514:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc0c827050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5711db52-8702-4d6b-b42c-8543e9bf1cb2+24:3646:x1876699573910144:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.870820:0:6514:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc0c827050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5711db52-8702-4d6b-b42c-8543e9bf1cb2+24:3646:x1876699573910144:12345-0@lo:400:kworker.0 Request processed in 741us (3914us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760766.870828:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 465 00000100:00100000:3.0:1789760766.870866:0:7470:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 850 00000100:00100000:3.0:1789760766.870873:0:7470:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3646:x1876699573910784:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.870904:0:7470:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3646:x1876699573910784:12345-0@lo:400:kworker.0 Request processed in 33us (3763us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760766.870911:0:7470:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 850 00000100:00100000:3.0:1789760766.870926:0:7470:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 851 00000100:00100000:3.0:1789760766.870932:0:7470:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3646:x1876699573911296:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.870960:0:7470:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3646:x1876699573911296:12345-0@lo:400:kworker.0 Request processed in 28us (3734us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760766.870966:0:7470:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 851 00000100:00100000:3.0:1789760766.870978:0:7470:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 852 00000100:00100000:3.0:1789760766.870984:0:7470:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3648:x1876699573910912:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.871017:0:7470:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3648:x1876699573910912:12345-0@lo:400:kworker.0 Request processed in 33us (3702us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760766.871024:0:7470:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 852 00000100:00100000:3.0:1789760766.871035:0:7470:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 853 00000100:00100000:3.0:1789760766.871041:0:7470:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3648:x1876699573911424:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.871067:0:7470:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3648:x1876699573911424:12345-0@lo:400:kworker.0 Request processed in 26us (3717us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760766.871072:0:7470:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 853 00000100:00100000:3.0:1789760766.871082:0:7470:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 854 00000100:00100000:3.0:1789760766.871088:0:7470:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3648:x1876699573910528:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.871113:0:7470:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3648:x1876699573910528:12345-0@lo:400:kworker.0 Request processed in 25us (3692us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760766.871118:0:7470:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 854 00000100:00100000:3.0:1789760766.871140:0:8438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573911168 00000100:00100000:3.0:1789760766.871144:0:8438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573911168 00000100:00100000:3.0:1789760766.871194:0:8438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573911680 00000100:00100000:3.0:1789760766.871197:0:8438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573911680 00000100:00100000:3.0:1789760766.871212:0:8438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573911040 00000100:00100000:3.0:1789760766.871215:0:8438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573911040 00000100:00100000:3.0:1789760766.871228:0:8438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573911552 00000100:00100000:3.0:1789760766.871231:0:8438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573911552 00000100:00100000:3.0:1789760766.871244:0:8438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 288 00000100:00100000:3.0:1789760766.871251:0:8438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573911168:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.871284:0:8438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573911168:12345-0@lo:400:kworker.0 Request processed in 33us (4262us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760766.871290:0:8438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 288 00000100:00100000:3.0:1789760766.871301:0:8438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 289 00000100:00100000:3.0:1789760766.871308:0:8438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573911680:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.871333:0:8438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573911680:12345-0@lo:400:kworker.0 Request processed in 26us (4276us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760766.871338:0:8438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 289 00000100:00100000:3.0:1789760766.871349:0:8438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 290 00000100:00100000:3.0:1789760766.871354:0:8438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3648:x1876699573911040:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.871396:0:8438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3648:x1876699573911040:12345-0@lo:400:kworker.0 Request processed in 24us (3926us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760766.871402:0:8438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 290 00000100:00100000:3.0:1789760766.871412:0:8438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 291 00000100:00100000:3.0:1789760766.871418:0:8438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3648:x1876699573911552:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.871448:0:8438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3648:x1876699573911552:12345-0@lo:400:kworker.0 Request processed in 29us (3958us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760766.871454:0:8438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 291 00000100:00100000:3.0:1789760766.871486:0:3647:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb049d5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3647:1876699573911808:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760766.871517:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760766.871564:0:8439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573911808 00000100:00100000:3.0:1789760766.871568:0:8439:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573911808 00000100:00100000:3.0:1789760766.871590:0:8439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 351 00000100:00100000:3.0:1789760766.871596:0:8439:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3647:x1876699573911808:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1789760766.871609:0:8439:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:3.0:1789760766.871644:0:8439:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3647:x1876699573911808:12345-0@lo:13:osp-pre-1-1.0 Request processed in 48us (126us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760766.871650:0:8439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 351 00000100:00100000:3.0:1789760766.871685:0:3647:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb049d5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3647:1876699573911808:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760766.871721:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cc07bcf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3646:1876699573910784:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.871732:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cc07bcc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3646:1876699573911296:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.871741:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cc07bcd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5711db52-8702-4d6b-b42c-8543e9bf1cb2:3646:1876699573910144:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.871750:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cc07bcc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573910656:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.871760:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cc07bcdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573911168:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.871770:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb044b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573911680:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.872563:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cc07bcce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3646:1876699573910272:0@lo:400:kworker.0 00000100:00100000:3.0:1789760766.897670:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554658304 00000100:00100000:3.0:1789760766.897686:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554658304 00000100:00100000:3.0:1789760767.048972:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554659584 00000100:00100000:3.0:1789760767.048983:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554659584 00000100:00100000:3.0:1789760767.049043:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710256154550 time 710276994946 state 0x0 ref 1 00000100:00100000:3.0:1789760767.049058:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710277255362 time 710276994946 state 0x1 ref 1 00000100:00100000:3.0:1789760767.049137:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710277255362 time 710294754133 state 0x0 ref 1 00000100:00100000:3.0:1789760767.049151:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554659584:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760767.049210:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60653568, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x1ced8 00010000:00010000:3.0:1789760767.049220:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f1e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.049237:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760767.049242:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.049248:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60653568->60657663] (req 60653568->60657663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.049262:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60653568->60657663] (req 60653568->60657663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760767.057571:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60653568 -> 60657664 00002000:00010000:3.0:1789760767.057576:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 118488 -> 118496 00010000:00010000:3.0:1789760767.057589:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60653568->60657663] (req 60653568->60657663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.057602:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f1e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60653568->60657663] (req 60653568->60657663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.057611:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60653568->60657663] (req 60653568->60657663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.057618:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760767.057632:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60653568->60657663] (req 60653568->60657663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.057641:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f1e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60653568->60657663] (req 60653568->60657663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.057662:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f1e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60653568->60657663] (req 60653568->60657663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.057677:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f1e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60653568->60657663] (req 60653568->60657663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.057955:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554659584:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8801us (9137us total) trans 4294982117 rc 0/0 00000100:00100000:3.0:1789760767.057965:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14819 00000100:00100000:3.0:1789760767.057975:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710277255362 time 710294754133 state 0x2 ref 0 00000001:00080000:3.0:1789760767.097542:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982119 is committed 00000001:00080000:3.0:1789760767.118306:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982120 is committed 00000100:00100000:3.0:1789760767.131716:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554660096 00000100:00100000:3.0:1789760767.131729:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554660096 00000100:00100000:3.0:1789760767.131792:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710336499336 time 710356762736 state 0x0 ref 1 00000100:00100000:3.0:1789760767.131881:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710357023152 time 710356762736 state 0x1 ref 1 00000100:00100000:3.0:1789760767.131916:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710357023152 time 710377535487 state 0x0 ref 1 00000100:00100000:3.0:1789760767.131931:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554660096:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760767.131979:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60669952, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x1cef8 00010000:00010000:3.0:1789760767.131989:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f3a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.132005:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760767.132010:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.132017:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60669952->60674047] (req 60669952->60674047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.132033:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60669952->60674047] (req 60669952->60674047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760767.138003:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60669952 -> 60674048 00002000:00010000:3.0:1789760767.138009:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 118520 -> 118528 00010000:00010000:3.0:1789760767.138026:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60669952->60674047] (req 60669952->60674047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.138038:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f3a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60669952->60674047] (req 60669952->60674047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.138048:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60669952->60674047] (req 60669952->60674047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.138055:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760767.138061:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60669952->60674047] (req 60669952->60674047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.138070:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f3a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60669952->60674047] (req 60669952->60674047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.138091:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f3a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60669952->60674047] (req 60669952->60674047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.138104:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f3a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60669952->60674047] (req 60669952->60674047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.138215:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554660096:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6283us (14800us total) trans 4294982121 rc 0/0 00000100:00100000:3.0:1789760767.138227:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14823 00000100:00100000:3.0:1789760767.138237:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710357023152 time 710377535487 state 0x2 ref 0 00000100:00100000:3.0:1789760767.145253:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554660224 00000100:00100000:3.0:1789760767.145264:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554660224 00000100:00100000:3.0:1789760767.145320:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710357023152 time 710377535487 state 0x0 ref 1 00000100:00100000:3.0:1789760767.145338:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710377795903 time 710377535487 state 0x1 ref 1 00000100:00100000:3.0:1789760767.145370:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710377795903 time 710390988333 state 0x0 ref 1 00000100:00100000:3.0:1789760767.145402:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554660224:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760767.145450:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60674048, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x1cf00 00010000:00010000:3.0:1789760767.145460:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3f41 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.145477:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760767.145481:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.145488:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3f41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60674048->60678143] (req 60674048->60678143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.145508:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3f41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60674048->60678143] (req 60674048->60678143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760767.156093:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60674048 -> 60678144 00002000:00010000:3.0:1789760767.156099:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 118528 -> 118536 00010000:00010000:3.0:1789760767.156114:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3f41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60674048->60678143] (req 60674048->60678143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.156126:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3f41 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60674048->60678143] (req 60674048->60678143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.156135:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3f41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60674048->60678143] (req 60674048->60678143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.156142:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760767.156148:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3f41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60674048->60678143] (req 60674048->60678143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.156156:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3f41 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60674048->60678143] (req 60674048->60678143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.156185:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3f41 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60674048->60678143] (req 60674048->60678143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.156193:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3f41 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60674048->60678143] (req 60674048->60678143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.156345:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554660224:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10941us (11336us total) trans 4294982122 rc 0/0 00000100:00100000:3.0:1789760767.156356:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14824 00000100:00100000:3.0:1789760767.156365:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710377795903 time 710390988333 state 0x2 ref 0 00000100:00100000:3.0:1789760767.166531:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554660352 00000100:00100000:3.0:1789760767.166541:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554660352 00000100:00100000:3.0:1789760767.166604:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710377795903 time 710390988333 state 0x0 ref 1 00000100:00100000:3.0:1789760767.166620:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710391248749 time 710390988333 state 0x1 ref 1 00000100:00100000:3.0:1789760767.166653:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710391248749 time 710412272081 state 0x0 ref 1 00000100:00100000:3.0:1789760767.166665:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554660352:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760767.166710:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60678144, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x1cf08 00010000:00010000:3.0:1789760767.166719:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3f48 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.166735:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760767.166740:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.166746:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3f48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60678144->60682239] (req 60678144->60682239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.166758:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c3f48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60678144->60682239] (req 60678144->60682239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760767.204129:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60686336 -> 60690432 00002000:00010000:3.0:1789760767.204135:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 118552 -> 118560 00010000:00010000:3.0:1789760767.204148:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c3f56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60686336->60690431] (req 60686336->60690431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.204161:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c3f56 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60686336->60690431] (req 60686336->60690431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.204180:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c3f56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60686336->60690431] (req 60686336->60690431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.204187:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760767.204192:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c3f56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60686336->60690431] (req 60686336->60690431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.204202:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c3f56 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60686336->60690431] (req 60686336->60690431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.204221:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c3f56 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60686336->60690431] (req 60686336->60690431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.204229:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c3f56 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60686336->60690431] (req 60686336->60690431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.204980:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554660608:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6500us (7251us total) trans 4294982125 rc 0/0 00000100:00100000:3.0:1789760767.204991:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14827 00000100:00100000:3.0:1789760767.205000:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710427225232 time 710444081099 state 0x2 ref 0 00000100:00100000:3.0:1789760767.211272:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554660736 00000100:00100000:3.0:1789760767.211282:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554660736 00000100:00100000:3.0:1789760767.211335:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710427225232 time 710444081099 state 0x0 ref 1 00000100:00100000:3.0:1789760767.211350:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710444341515 time 710444081099 state 0x1 ref 1 00000100:00100000:3.0:1789760767.211442:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710444341515 time 710457059491 state 0x0 ref 1 00000100:00100000:3.0:1789760767.211456:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554660736:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760767.211504:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60690432, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x1cf20 00010000:00010000:3.0:1789760767.211514:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f5d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.211529:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760767.211534:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.211541:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60690432->60694527] (req 60690432->60694527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.211565:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60690432->60694527] (req 60690432->60694527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760767.218954:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60690432 -> 60694528 00002000:00010000:3.0:1789760767.218960:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 118560 -> 118568 00010000:00010000:3.0:1789760767.218976:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60690432->60694527] (req 60690432->60694527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.218989:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f5d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60690432->60694527] (req 60690432->60694527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.218997:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60690432->60694527] (req 60690432->60694527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.219004:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760767.219009:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60690432->60694527] (req 60690432->60694527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.219017:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f5d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60690432->60694527] (req 60690432->60694527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.219035:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f5d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60690432->60694527] (req 60690432->60694527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.219043:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f5d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60690432->60694527] (req 60690432->60694527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.219144:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554660736:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7686us (8021us total) trans 4294982126 rc 0/0 00000100:00100000:3.0:1789760767.219154:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14828 00000100:00100000:3.0:1789760767.219165:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710444341515 time 710457059491 state 0x2 ref 0 00000001:00080000:3.0:1789760767.244106:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982128 is committed 00000100:00100000:3.0:1789760767.252450:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554661120 00000100:00100000:3.0:1789760767.252461:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554661120 00000001:00080000:3.0:1789760767.258678:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982129 is committed 00000100:00100000:3.0:1789760767.267644:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554661248 00000100:00100000:3.0:1789760767.267654:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554661248 00000100:00100000:3.0:1789760767.267709:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710482040206 time 710498357600 state 0x0 ref 1 00000100:00100000:3.0:1789760767.267724:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710498618016 time 710498357600 state 0x1 ref 1 00000100:00100000:3.0:1789760767.267756:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710498618016 time 710513375307 state 0x0 ref 1 00000100:00100000:3.0:1789760767.267769:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554661248:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760767.267831:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60706816, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x1cf40 00010000:00010000:3.0:1789760767.267842:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f79 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.267860:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760767.267865:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.267872:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60706816->60710911] (req 60706816->60710911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.267885:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c3f79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60706816->60710911] (req 60706816->60710911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760767.318199:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982133 is committed 00002000:00010000:3.0:1789760767.406509:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60739584 -> 60743680 00002000:00010000:3.0:1789760767.406513:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 118656 -> 118664 00010000:00010000:3.0:1789760767.406524:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c3fb1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60739584->60743679] (req 60739584->60743679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.406537:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c3fb1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60739584->60743679] (req 60739584->60743679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.407059:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c3fb1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60739584->60743679] (req 60739584->60743679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.407072:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760767.407077:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c3fb1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60739584->60743679] (req 60739584->60743679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.407088:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c3fb1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60739584->60743679] (req 60739584->60743679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.407108:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c3fb1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60739584->60743679] (req 60739584->60743679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.407116:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c3fb1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60739584->60743679] (req 60739584->60743679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.407225:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554662272:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9115us (9392us total) trans 4294982138 rc 0/0 00000100:00100000:3.0:1789760767.407236:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14840 00000100:00100000:3.0:1789760767.407245:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710630760789 time 710643712668 state 0x2 ref 0 00000001:00080000:3.0:1789760767.434092:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982140 is committed 00000001:00080000:3.0:1789760767.448262:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982141 is committed 00000100:00100000:3.0:1789760767.452264:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554662784 00000100:00100000:3.0:1789760767.452273:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554662784 00000100:00100000:3.0:1789760767.452320:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710671500693 time 710687974349 state 0x0 ref 1 00000100:00100000:3.0:1789760767.452334:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710688234765 time 710687974349 state 0x1 ref 1 00000100:00100000:3.0:1789760767.452364:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710688234765 time 710697982750 state 0x0 ref 1 00000100:00100000:3.0:1789760767.452375:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554662784:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760767.452438:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60755968, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x1cfa0 00010000:00010000:3.0:1789760767.452446:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3fcd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.452462:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760767.452467:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.452473:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3fcd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60755968->60760063] (req 60755968->60760063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.452486:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3fcd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60755968->60760063] (req 60755968->60760063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760767.457317:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60755968 -> 60760064 00002000:00010000:3.0:1789760767.457322:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 118688 -> 118696 00010000:00010000:3.0:1789760767.457334:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3fcd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60755968->60760063] (req 60755968->60760063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.457349:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3fcd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60755968->60760063] (req 60755968->60760063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.457359:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3fcd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60755968->60760063] (req 60755968->60760063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.457366:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760767.457371:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3fcd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60755968->60760063] (req 60755968->60760063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.457449:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3fcd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60755968->60760063] (req 60755968->60760063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.457469:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3fcd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60755968->60760063] (req 60755968->60760063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.457477:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c3fcd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60755968->60760063] (req 60755968->60760063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.457703:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554662784:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5324us (5598us total) trans 4294982142 rc 0/0 00000100:00100000:3.0:1789760767.457712:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14844 00000100:00100000:3.0:1789760767.457722:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710688234765 time 710697982750 state 0x2 ref 0 00000100:00100000:3.0:1789760767.463774:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554662912 00000100:00100000:3.0:1789760767.463786:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554662912 00000100:00100000:3.0:1789760767.463850:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710688234765 time 710697982750 state 0x0 ref 1 00000100:00100000:3.0:1789760767.463935:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710698243166 time 710697982750 state 0x1 ref 1 00000100:00100000:3.0:1789760767.463970:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710698243166 time 710709589108 state 0x0 ref 1 00000100:00100000:3.0:1789760767.463984:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554662912:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760767.464043:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60760064, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x1cfa8 00010000:00010000:3.0:1789760767.464054:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3fd4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.464071:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760767.464077:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.464084:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3fd4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60760064->60764159] (req 60760064->60764159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.464335:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3fd4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60760064->60764159] (req 60760064->60764159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760767.469906:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60760064 -> 60764160 00002000:00010000:3.0:1789760767.469912:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 118696 -> 118704 00010000:00010000:3.0:1789760767.469927:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3fd4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60760064->60764159] (req 60760064->60764159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.469939:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3fd4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60760064->60764159] (req 60760064->60764159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.469955:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3fd4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60760064->60764159] (req 60760064->60764159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.469964:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760767.469969:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3fd4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60760064->60764159] (req 60760064->60764159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.469979:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3fd4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60760064->60764159] (req 60760064->60764159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.470000:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3fd4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60760064->60764159] (req 60760064->60764159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.470008:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3fd4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60760064->60764159] (req 60760064->60764159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.470155:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554662912:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6168us (6882us total) trans 4294982143 rc 0/0 00000100:00100000:3.0:1789760767.470176:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14845 00000100:00100000:3.0:1789760767.470187:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710698243166 time 710709589108 state 0x2 ref 0 00000001:00080000:3.0:1789760767.487280:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982144 is committed 00000001:00080000:3.0:1789760767.501282:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982145 is committed 00000100:00100000:3.0:1789760767.517324:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554663296 00000100:00100000:3.0:1789760767.517335:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554663296 00000100:00100000:3.0:1789760767.517414:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710721537874 time 710741315471 state 0x0 ref 1 00000100:00100000:3.0:1789760767.517430:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710741575887 time 710741315471 state 0x1 ref 1 00000100:00100000:3.0:1789760767.517461:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710741575887 time 710763080086 state 0x0 ref 1 00000100:00100000:3.0:1789760767.517473:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05b8d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554663296:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760767.517518:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60772352, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x1cfc0 00010000:00010000:3.0:1789760767.517528:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3fe9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.517544:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760767.517549:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.517555:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3fe9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60772352->60776447] (req 60772352->60776447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.517568:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3fe9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60772352->60776447] (req 60772352->60776447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760767.523878:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60772352 -> 60776448 00002000:00010000:3.0:1789760767.523883:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 118720 -> 118728 00010000:00010000:3.0:1789760767.523897:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3fe9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60772352->60776447] (req 60772352->60776447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.523911:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3fe9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60772352->60776447] (req 60772352->60776447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.523921:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3fe9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60772352->60776447] (req 60772352->60776447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.523929:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760767.523934:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3fe9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60772352->60776447] (req 60772352->60776447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.523943:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3fe9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60772352->60776447] (req 60772352->60776447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.523964:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3fe9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60772352->60776447] (req 60772352->60776447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.523973:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c3fe9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60772352->60776447] (req 60772352->60776447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.524076:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05b8d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554663296:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6599us (6878us total) trans 4294982146 rc 0/0 00000100:00100000:3.0:1789760767.524087:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14848 00000100:00100000:3.0:1789760767.524096:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710741575887 time 710763080086 state 0x2 ref 0 00000100:00100000:3.0:1789760767.529321:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554663424 00000100:00100000:3.0:1789760767.529331:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554663424 00000100:00100000:3.0:1789760767.529615:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710741575887 time 710763080086 state 0x0 ref 1 00000100:00100000:3.0:1789760767.529632:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710763340502 time 710763080086 state 0x1 ref 1 00002000:00010000:3.0:1789760767.582038:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60784640 -> 60788736 00002000:00010000:3.0:1789760767.582045:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 118744 -> 118752 00010000:00010000:3.0:1789760767.582059:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c3ffe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60784640->60788735] (req 60784640->60788735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.582072:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c3ffe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60784640->60788735] (req 60784640->60788735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.582082:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c3ffe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60784640->60788735] (req 60784640->60788735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.582090:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760767.582095:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c3ffe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60784640->60788735] (req 60784640->60788735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.582106:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c3ffe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60784640->60788735] (req 60784640->60788735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.582125:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c3ffe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60784640->60788735] (req 60784640->60788735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.582134:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c3ffe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60784640->60788735] (req 60784640->60788735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.582321:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05b8fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554663680:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5586us (6147us total) trans 4294982149 rc 0/0 00000100:00100000:3.0:1789760767.582332:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14851 00000100:00100000:3.0:1789760767.582343:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710798977468 time 710822336727 state 0x2 ref 0 00000100:00100000:3.0:1789760767.591037:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554663808 00000100:00100000:3.0:1789760767.591048:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554663808 00000100:00100000:3.0:1789760767.591111:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710798977468 time 710822336727 state 0x0 ref 1 00000100:00100000:3.0:1789760767.591128:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710822597143 time 710822336727 state 0x1 ref 1 00000100:00100000:3.0:1789760767.591210:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710822597143 time 710836827359 state 0x0 ref 1 00000100:00100000:3.0:1789760767.591222:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554663808:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760767.591270:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60788736, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x1cfe0 00010000:00010000:3.0:1789760767.591280:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4005 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.591296:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760767.591301:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.591307:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4005 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60788736->60792831] (req 60788736->60792831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.591321:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4005 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60788736->60792831] (req 60788736->60792831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760767.600928:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60788736 -> 60792832 00002000:00010000:3.0:1789760767.600938:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 118752 -> 118760 00010000:00010000:3.0:1789760767.600952:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4005 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60788736->60792831] (req 60788736->60792831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.600973:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4005 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60788736->60792831] (req 60788736->60792831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.600983:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4005 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60788736->60792831] (req 60788736->60792831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.600992:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760767.600997:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4005 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60788736->60792831] (req 60788736->60792831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.601006:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4005 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60788736->60792831] (req 60788736->60792831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.601025:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4005 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60788736->60792831] (req 60788736->60792831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.601034:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4005 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60788736->60792831] (req 60788736->60792831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.601138:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554663808:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9912us (10207us total) trans 4294982150 rc 0/0 00000100:00100000:3.0:1789760767.601149:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14852 00000100:00100000:3.0:1789760767.601159:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710822597143 time 710836827359 state 0x2 ref 0 00000100:00100000:3.0:1789760767.606109:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554663936 00000100:00100000:3.0:1789760767.606120:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554663936 00000100:00100000:3.0:1789760767.606187:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710822597143 time 710836827359 state 0x0 ref 1 00000100:00100000:3.0:1789760767.606204:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710837087775 time 710836827359 state 0x1 ref 1 00000100:00100000:3.0:1789760767.606240:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710837087775 time 710851857908 state 0x0 ref 1 00000100:00100000:3.0:1789760767.606253:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554663936:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760767.606301:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60792832, mtime: 0x6aad94fe, atime: 0x0, ctime: 0x6aad94fe, blocks: 0x1cfe8 00010000:00010000:3.0:1789760767.606311:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c400c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.606329:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760767.606334:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.606341:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c400c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60792832->60796927] (req 60792832->60796927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.606357:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c400c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60792832->60796927] (req 60792832->60796927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760767.612509:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60792832 -> 60796928 00002000:00010000:3.0:1789760767.612515:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 118760 -> 118768 00010000:00010000:3.0:1789760767.612584:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c400c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60792832->60796927] (req 60792832->60796927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.612598:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c400c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60792832->60796927] (req 60792832->60796927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.612614:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c400c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60792832->60796927] (req 60792832->60796927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.612622:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760767.612627:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c400c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60792832->60796927] (req 60792832->60796927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.612637:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c400c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60792832->60796927] (req 60792832->60796927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.612657:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c400c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60792832->60796927] (req 60792832->60796927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.612665:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c400c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60792832->60796927] (req 60792832->60796927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.612756:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554663936:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6502us (6870us total) trans 4294982151 rc 0/0 00000100:00100000:3.0:1789760767.612767:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14853 00000100:00100000:3.0:1789760767.612777:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710837087775 time 710851857908 state 0x2 ref 0 00000100:00100000:3.0:1789760767.653935:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554664320 00000100:00100000:3.0:1789760767.653952:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554664320 00000001:00080000:3.0:1789760767.663244:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982154 is committed 00000100:00100000:3.0:1789760767.676270:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554664448 00000100:00100000:3.0:1789760767.676286:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554664448 00000100:00100000:3.0:1789760767.676352:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710879339725 time 710903063389 state 0x0 ref 1 00000100:00100000:3.0:1789760767.676647:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710903323805 time 710903063389 state 0x1 ref 1 00000100:00100000:3.0:1789760767.676682:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 710903323805 time 710922300342 state 0x0 ref 1 00000100:00100000:3.0:1789760767.676697:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554664448:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760767.676758:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60809216, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x1d008 00010000:00010000:3.0:1789760767.676768:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4028 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.676786:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=710s v=(0 0) 00010000:00010000:3.0:1789760767.676801:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.676808:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4028 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60809216->60813311] (req 60809216->60813311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.676822:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4028 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60809216->60813311] (req 60809216->60813311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.846009:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711073039830 time 711091621317 state 0x0 ref 1 00000100:00100000:3.0:1789760767.846036:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554665856:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760767.846100:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60854272, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x1d060 00010000:00010000:3.0:1789760767.846120:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4075 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.846144:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760767.846149:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.846156:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4075 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60854272->60858367] (req 60854272->60858367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.846171:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4075 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60854272->60858367] (req 60854272->60858367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760767.859918:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60854272 -> 60858368 00002000:00010000:3.0:1789760767.859925:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 118880 -> 118888 00010000:00010000:3.0:1789760767.859938:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4075 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60854272->60858367] (req 60854272->60858367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.859952:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4075 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60854272->60858367] (req 60854272->60858367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.859961:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4075 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60854272->60858367] (req 60854272->60858367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.859970:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760767.859975:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4075 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60854272->60858367] (req 60854272->60858367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.859985:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4075 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60854272->60858367] (req 60854272->60858367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.860011:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4075 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60854272->60858367] (req 60854272->60858367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.860019:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4075 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60854272->60858367] (req 60854272->60858367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.860124:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554665856:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14092us (17027us total) trans 4294982166 rc 0/0 00000100:00100000:3.0:1789760767.860135:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14868 00000100:00100000:3.0:1789760767.860143:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711073039830 time 711091621317 state 0x2 ref 0 00000001:00080000:3.0:1789760767.885744:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982167 is committed 00000001:00080000:3.0:1789760767.906567:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982168 is committed 00000001:00080000:3.0:1789760767.922195:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982169 is committed 00000100:00100000:3.0:1789760767.927896:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554666368 00000100:00100000:3.0:1789760767.927907:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554666368 00000100:00100000:3.0:1789760767.927964:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711142101944 time 711157674619 state 0x0 ref 1 00000100:00100000:3.0:1789760767.927979:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711157935035 time 711157674619 state 0x1 ref 1 00000100:00100000:3.0:1789760767.928050:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711157935035 time 711173667643 state 0x0 ref 1 00000100:00100000:3.0:1789760767.928063:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554666368:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760767.928224:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60870656, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x1d080 00010000:00010000:3.0:1789760767.928234:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4091 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.928253:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760767.928258:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760767.928264:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4091 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60870656->60874751] (req 60870656->60874751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.928279:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4091 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60870656->60874751] (req 60870656->60874751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760767.941252:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60870656 -> 60874752 00002000:00010000:3.0:1789760767.941257:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 118912 -> 118920 00010000:00010000:3.0:1789760767.941270:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4091 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60870656->60874751] (req 60870656->60874751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.941282:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4091 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60870656->60874751] (req 60870656->60874751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.941291:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4091 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60870656->60874751] (req 60870656->60874751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.941299:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760767.941304:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4091 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60870656->60874751] (req 60870656->60874751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.941314:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4091 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60870656->60874751] (req 60870656->60874751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.941339:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4091 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60870656->60874751] (req 60870656->60874751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760767.941348:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4091 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60870656->60874751] (req 60870656->60874751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760767.941496:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554666368:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13430us (14017us total) trans 4294982170 rc 0/0 00000100:00100000:3.0:1789760767.941506:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14872 00000100:00100000:3.0:1789760767.941516:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711157935035 time 711173667643 state 0x2 ref 0 00000100:00100000:3.0:1789760768.001472:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554666880 00000100:00100000:3.0:1789760768.001482:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554666880 00000100:00100000:3.0:1789760768.001534:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711214975675 time 711231840720 state 0x0 ref 1 00000100:00100000:3.0:1789760768.001560:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711232101136 time 711231840720 state 0x1 ref 1 00000100:00100000:3.0:1789760768.001585:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711232101136 time 711247203409 state 0x0 ref 1 00000100:00100000:3.0:1789760768.001597:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554666880:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760768.001642:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60887040, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x1d0a0 00010000:00010000:3.0:1789760768.001651:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40ad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.001668:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760768.001673:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.001680:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60887040->60891135] (req 60887040->60891135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.001695:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60887040->60891135] (req 60887040->60891135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760768.018297:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60887040 -> 60891136 00002000:00010000:3.0:1789760768.018302:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 118944 -> 118952 00010000:00010000:3.0:1789760768.018315:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60887040->60891135] (req 60887040->60891135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.018327:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40ad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60887040->60891135] (req 60887040->60891135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.018335:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60887040->60891135] (req 60887040->60891135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.020333:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760768.020341:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60887040->60891135] (req 60887040->60891135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.020484:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60887040->60891135] (req 60887040->60891135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.020510:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40ad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60887040->60891135] (req 60887040->60891135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.020525:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40ad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60887040->60891135] (req 60887040->60891135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.020700:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554666880:12345-192.168.203.22@tcp:4:dd.0 Request processed in 19095us (19412us total) trans 4294982174 rc 0/0 00000100:00100000:3.0:1789760768.020712:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14876 00000100:00100000:3.0:1789760768.020724:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711232101136 time 711247203409 state 0x2 ref 0 00000001:00080000:3.0:1789760768.048114:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982175 is committed 00000001:00080000:3.0:1789760768.071197:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982176 is committed 00000100:00100000:3.0:1789760768.096545:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554667392 00000100:00100000:3.0:1789760768.096554:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554667392 00000100:00100000:3.0:1789760768.096666:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711306149123 time 711326709407 state 0x0 ref 1 00000100:00100000:3.0:1789760768.096684:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711326969823 time 711326709407 state 0x1 ref 1 00000100:00100000:3.0:1789760768.096761:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711326969823 time 711342331904 state 0x0 ref 1 00000100:00100000:3.0:1789760768.096778:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554667392:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760768.096835:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60903424, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x1d0c0 00010000:00010000:3.0:1789760768.096845:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40c9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.096860:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760768.096864:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.096871:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60903424->60907519] (req 60903424->60907519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.097085:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60903424->60907519] (req 60903424->60907519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760768.104809:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60903424 -> 60907520 00002000:00010000:3.0:1789760768.104814:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 118976 -> 118984 00010000:00010000:3.0:1789760768.105003:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60903424->60907519] (req 60903424->60907519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.105018:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40c9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60903424->60907519] (req 60903424->60907519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.105031:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60903424->60907519] (req 60903424->60907519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.105038:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760768.105043:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60903424->60907519] (req 60903424->60907519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.105051:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40c9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60903424->60907519] (req 60903424->60907519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.105249:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40c9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60903424->60907519] (req 60903424->60907519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.105265:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40c9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60903424->60907519] (req 60903424->60907519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.105479:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554667392:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8699us (9037us total) trans 4294982178 rc 0/0 00000100:00100000:3.0:1789760768.105490:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14880 00000100:00100000:3.0:1789760768.105500:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711326969823 time 711342331904 state 0x2 ref 0 00000100:00100000:3.0:1789760768.181975:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554667904 00000100:00100000:3.0:1789760768.181986:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554667904 00000100:00100000:3.0:1789760768.182042:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711386990446 time 711411528922 state 0x0 ref 1 00000100:00100000:3.0:1789760768.182058:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711411789338 time 711411528922 state 0x1 ref 1 00000100:00100000:3.0:1789760768.182099:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711411789338 time 711427717312 state 0x0 ref 1 00000100:00100000:3.0:1789760768.182112:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554667904:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760768.182407:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60919808, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x1d0e0 00010000:00010000:3.0:1789760768.182417:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40e5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.182433:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760768.182438:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.182445:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60919808->60923903] (req 60919808->60923903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.182464:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60919808->60923903] (req 60919808->60923903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760768.191095:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60919808 -> 60923904 00002000:00010000:3.0:1789760768.191101:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119008 -> 119016 00010000:00010000:3.0:1789760768.191115:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60919808->60923903] (req 60919808->60923903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.191141:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40e5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60919808->60923903] (req 60919808->60923903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.191151:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60919808->60923903] (req 60919808->60923903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.191159:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760768.191163:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60919808->60923903] (req 60919808->60923903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.191173:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60919808->60923903] (req 60919808->60923903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.191193:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40e5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60919808->60923903] (req 60919808->60923903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.191201:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c40e5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60919808->60923903] (req 60919808->60923903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.191307:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554667904:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9191us (9432us total) trans 4294982182 rc 0/0 00000100:00100000:3.0:1789760768.191319:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14884 00000100:00100000:3.0:1789760768.191327:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711411789338 time 711427717312 state 0x2 ref 0 00000001:00080000:3.0:1789760768.230737:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982185 is committed 00000100:00100000:3.0:1789760768.239069:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554668416 00000100:00100000:3.0:1789760768.239079:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554668416 00000100:00100000:3.0:1789760768.239136:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711458795041 time 711469372925 state 0x0 ref 1 00000100:00100000:3.0:1789760768.239150:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711469633341 time 711469372925 state 0x1 ref 1 00000100:00100000:3.0:1789760768.239228:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711469633341 time 711484846588 state 0x0 ref 1 00000100:00100000:3.0:1789760768.239240:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554668416:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760768.239284:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60936192, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x1d100 00010000:00010000:3.0:1789760768.239294:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4101 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.239310:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760768.239315:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.239321:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4101 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60936192->60940287] (req 60936192->60940287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.239334:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4101 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60936192->60940287] (req 60936192->60940287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760768.246095:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60936192 -> 60940288 00002000:00010000:3.0:1789760768.246101:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119040 -> 119048 00010000:00010000:3.0:1789760768.246114:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4101 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60936192->60940287] (req 60936192->60940287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.246126:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4101 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60936192->60940287] (req 60936192->60940287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.246134:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4101 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60936192->60940287] (req 60936192->60940287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.246141:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760768.246146:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4101 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60936192->60940287] (req 60936192->60940287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.246155:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4101 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60936192->60940287] (req 60936192->60940287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.246183:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4101 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60936192->60940287] (req 60936192->60940287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.246192:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4101 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60936192->60940287] (req 60936192->60940287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.246334:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554668416:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7090us (7352us total) trans 4294982186 rc 0/0 00000100:00100000:3.0:1789760768.246346:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14888 00000100:00100000:3.0:1789760768.246354:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711469633341 time 711484846588 state 0x2 ref 0 00000100:00100000:3.0:1789760768.256254:0:7470:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554668800 00000100:00100000:3.0:1789760768.256264:0:7470:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554668800 00000100:00100000:3.0:1789760768.256318:0:7470:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554669056 00000100:00100000:3.0:1789760768.256321:0:7470:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554669056 00000100:00100000:3.0:1789760768.256336:0:7470:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554668928 00000100:00100000:3.0:1789760768.256339:0:7470:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554668928 00000100:00100000:3.0:1789760768.256356:0:7470:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 855 00000100:00100000:3.0:1789760768.256364:0:7470:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bce300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+8:2413:x1876699554668800:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:3.0:1789760768.256471:0:7470:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bce300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+8:2413:x1876699554668800:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 107us (431us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760768.256479:0:7470:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 855 00000100:00100000:3.0:1789760768.256496:0:7470:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 856 00000100:00100000:3.0:1789760768.256503:0:7470:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2413:x1876699554669056:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:3.0:1789760768.256528:0:7470:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2413:x1876699554669056:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 25us (440us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760768.256534:0:7470:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 856 00000100:00100000:3.0:1789760768.256654:0:7470:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 857 00000100:00100000:3.0:1789760768.256661:0:7470:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554668928:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:3.0:1789760768.256687:0:7470:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554668928:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 27us (535us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760768.256693:0:7470:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 857 00000100:00080000:3.0:1789760768.256712:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789760768.256717:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760768.258283:0:7470:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554669184 00000100:00100000:3.0:1789760768.258290:0:7470:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554669184 00000100:00100000:3.0:1789760768.258347:0:7470:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 858 00000100:00100000:3.0:1789760768.258355:0:7470:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2413:x1876699554669184:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:3.0:1789760768.258418:0:7470:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2413:x1876699554669184:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 63us (1185us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760768.258425:0:7470:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 858 00000100:00100000:3.0:1789760768.285325:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760768.291994:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982190 is committed 00000001:00080000:3.0:1789760768.309084:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982191 is committed 00000100:00100000:3.0:1789760768.315534:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554669824 00000100:00100000:3.0:1789760768.315546:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554669824 00000100:00100000:3.0:1789760768.315600:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711531505343 time 711547868668 state 0x0 ref 1 00000100:00100000:3.0:1789760768.315616:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711548129084 time 711547868668 state 0x1 ref 1 00000100:00100000:3.0:1789760768.315652:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711548129084 time 711561270832 state 0x0 ref 1 00000100:00100000:3.0:1789760768.315666:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554669824:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760768.315710:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60960768, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x1d130 00010000:00010000:3.0:1789760768.315719:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c412b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.315734:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760768.315739:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.315745:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c412b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60960768->60964863] (req 60960768->60964863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.315760:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c412b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60960768->60964863] (req 60960768->60964863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760768.327881:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982192 is committed 00000100:00100000:3.0:1789760768.333532:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760768.343446:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982193 is committed 00000100:00100000:3.0:1789760768.383015:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760768.413908:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760768.435206:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554670848 00000100:00100000:3.0:1789760768.435224:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554670848 00000100:00100000:3.0:1789760768.435291:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711648865407 time 711659774399 state 0x0 ref 1 00000100:00100000:3.0:1789760768.435307:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711660034815 time 711659774399 state 0x1 ref 1 00000100:00100000:3.0:1789760768.435334:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711660034815 time 711680952778 state 0x0 ref 1 00000100:00100000:3.0:1789760768.435347:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554670848:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760768.435475:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 60993536, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x1d170 00010000:00010000:3.0:1789760768.435484:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4163 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.435501:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760768.435506:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.435512:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4163 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60993536->60997631] (req 60993536->60997631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.435531:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4163 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60993536->60997631] (req 60993536->60997631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760768.441951:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 60993536 -> 60997632 00002000:00010000:3.0:1789760768.441959:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119152 -> 119160 00010000:00010000:3.0:1789760768.441972:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4163 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60993536->60997631] (req 60993536->60997631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.441985:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4163 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60993536->60997631] (req 60993536->60997631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.441995:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4163 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60993536->60997631] (req 60993536->60997631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.442003:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760768.442008:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4163 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60993536->60997631] (req 60993536->60997631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.442017:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4163 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60993536->60997631] (req 60993536->60997631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.442035:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4163 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60993536->60997631] (req 60993536->60997631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.442041:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4163 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [60993536->60997631] (req 60993536->60997631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.442135:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554670848:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6786us (10127us total) trans 4294982200 rc 0/0 00000100:00100000:3.0:1789760768.442146:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14902 00000100:00100000:3.0:1789760768.442155:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711660034815 time 711680952778 state 0x2 ref 0 00000001:00080000:3.0:1789760768.456228:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982201 is committed 00000001:00080000:3.0:1789760768.469498:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982202 is committed 00000001:00080000:3.0:1789760768.481015:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982203 is committed 00000001:00080000:3.0:1789760768.519315:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982205 is committed 00000001:00080000:3.0:1789760768.538014:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982206 is committed 00000001:00080000:3.0:1789760768.549960:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982207 is committed 00000001:00080000:3.0:1789760768.580829:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982209 is committed 00000100:00100000:3.0:1789760768.625038:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554672384 00000100:00100000:3.0:1789760768.625049:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554672384 00000100:00100000:3.0:1789760768.625101:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711844966945 time 711856859768 state 0x0 ref 1 00000100:00100000:3.0:1789760768.625116:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711857120184 time 711856859768 state 0x1 ref 1 00000100:00100000:3.0:1789760768.625191:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711857120184 time 711870807799 state 0x0 ref 1 00000100:00100000:3.0:1789760768.625205:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554672384:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760768.625250:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61042688, mtime: 0x6aad94ff, atime: 0x0, ctime: 0x6aad94ff, blocks: 0x1d1d0 00010000:00010000:3.0:1789760768.625259:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c41b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.625274:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=711s v=(0 0) 00010000:00010000:3.0:1789760768.625279:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760768.625286:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c41b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61042688->61046783] (req 61042688->61046783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.625300:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c41b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61042688->61046783] (req 61042688->61046783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760768.631062:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61042688 -> 61046784 00002000:00010000:3.0:1789760768.631067:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119248 -> 119256 00010000:00010000:3.0:1789760768.631080:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c41b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61042688->61046783] (req 61042688->61046783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.631094:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c41b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61042688->61046783] (req 61042688->61046783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.631104:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c41b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61042688->61046783] (req 61042688->61046783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.631112:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760768.631117:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c41b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61042688->61046783] (req 61042688->61046783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.631127:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c41b7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61042688->61046783] (req 61042688->61046783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.631148:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c41b7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61042688->61046783] (req 61042688->61046783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760768.631163:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c41b7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61042688->61046783] (req 61042688->61046783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760768.631307:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554672384:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6099us (6439us total) trans 4294982212 rc 0/0 00000100:00100000:3.0:1789760768.631318:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14914 00000100:00100000:3.0:1789760768.631328:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711857120184 time 711870807799 state 0x2 ref 0 00000001:00080000:3.0:1789760768.642890:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982213 is committed 00000100:00100000:3.0:1789760768.700211:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554672896 00000100:00100000:3.0:1789760768.700223:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554672896 00000100:00100000:3.0:1789760768.700280:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711896646114 time 711919749217 state 0x0 ref 1 00000100:00100000:3.0:1789760768.700297:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711920009633 time 711919749217 state 0x1 ref 1 00000100:00100000:3.0:1789760768.700330:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711920009633 time 711945948322 state 0x0 ref 1 00000100:00100000:3.0:1789760768.700343:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554672896:12345-192.168.203.22@tcp:4:dd.0 00000001:00080000:3.0:1789760768.735836:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982217 is committed 00000001:00080000:3.0:1789760768.751712:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982218 is committed 00000001:00080000:3.0:1789760768.771115:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982219 is committed 00010000:00010000:0.0:1789760768.771373:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c41e8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61071360->61075455] (req 61071360->61075455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.771416:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c41e8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61071360->61075455] (req 61071360->61075455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.771423:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c41e8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61071360->61075455] (req 61071360->61075455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.771513:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04563b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554673280:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7333us (7575us total) trans 4294982219 rc 0/0 00000100:00100000:0.0:1789760768.771525:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14921 00000100:00100000:0.0:1789760768.771534:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711989634157 time 712009776104 state 0x2 ref 0 00000100:00100000:0.0:1789760768.779829:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554673408 00000100:00100000:0.0:1789760768.779843:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554673408 00000100:00100000:0.0:1789760768.779901:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 711989634157 time 712009776104 state 0x0 ref 1 00000100:00100000:0.0:1789760768.779917:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712010036520 time 712009776104 state 0x1 ref 1 00000100:00100000:0.0:1789760768.779949:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712010036520 time 712025567366 state 0x0 ref 1 00000100:00100000:0.0:1789760768.779964:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554673408:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760768.780011:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61075456, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d210 00010000:00010000:0.0:1789760768.780020:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c41ef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.780036:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760768.780041:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.780048:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c41ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61075456->61079551] (req 61075456->61079551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.780062:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c41ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61075456->61079551] (req 61075456->61079551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760768.789858:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61075456 -> 61079552 00002000:00010000:0.0:1789760768.789865:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119312 -> 119320 00010000:00010000:0.0:1789760768.789880:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c41ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61075456->61079551] (req 61075456->61079551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.789894:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c41ef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61075456->61079551] (req 61075456->61079551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.789903:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c41ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61075456->61079551] (req 61075456->61079551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.789910:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760768.789915:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c41ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61075456->61079551] (req 61075456->61079551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.789924:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c41ef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61075456->61079551] (req 61075456->61079551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.789943:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c41ef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61075456->61079551] (req 61075456->61079551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.789955:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c41ef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61075456->61079551] (req 61075456->61079551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.790294:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554673408:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10328us (14074us total) trans 4294982220 rc 0/0 00000100:00100000:0.0:1789760768.790305:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14922 00000100:00100000:0.0:1789760768.790314:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712010036520 time 712025567366 state 0x2 ref 0 00000100:00100000:0.0:1789760768.808650:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554673664 00000100:00100000:0.0:1789760768.808658:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554673664 00000100:00100000:0.0:1789760768.808707:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712025827782 time 712040325282 state 0x0 ref 1 00000100:00100000:0.0:1789760768.808731:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712040585698 time 712040325282 state 0x1 ref 1 00000100:00100000:0.0:1789760768.808792:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712040585698 time 712054400160 state 0x0 ref 1 00000100:00100000:0.0:1789760768.808804:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04560e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554673664:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760768.808847:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61083648, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d220 00010000:00010000:0.0:1789760768.808857:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c41fd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.808872:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760768.808877:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.808883:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c41fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61083648->61087743] (req 61083648->61087743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.808902:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c41fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61083648->61087743] (req 61083648->61087743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760768.820440:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982222 is committed 00000100:00100000:0.0:1789760768.825024:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554673792 00000100:00100000:0.0:1789760768.825032:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554673792 00000100:00100000:0.0:1789760768.825081:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712040585698 time 712054400160 state 0x0 ref 1 00000100:00100000:0.0:1789760768.825092:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712054660576 time 712054400160 state 0x1 ref 1 00000100:00100000:0.0:1789760768.825114:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712054660576 time 712070734177 state 0x0 ref 1 00000100:00100000:0.0:1789760768.825123:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04562a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554673792:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760768.825265:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61087744, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d228 00010000:00010000:0.0:1789760768.825274:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4204 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.825287:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760768.825290:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.825295:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4204 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61087744->61091839] (req 61087744->61091839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.825305:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4204 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61087744->61091839] (req 61087744->61091839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760768.836253:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61087744 -> 61091840 00002000:00010000:0.0:1789760768.836259:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119336 -> 119344 00010000:00010000:0.0:1789760768.836272:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4204 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61087744->61091839] (req 61087744->61091839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.836283:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4204 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61087744->61091839] (req 61087744->61091839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.836293:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4204 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61087744->61091839] (req 61087744->61091839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.836300:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760768.836305:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4204 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61087744->61091839] (req 61087744->61091839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.836314:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4204 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61087744->61091839] (req 61087744->61091839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.836333:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4204 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61087744->61091839] (req 61087744->61091839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.836342:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4204 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61087744->61091839] (req 61087744->61091839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.836613:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04562a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554673792:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11484us (11760us total) trans 4294982223 rc 0/0 00000100:00100000:0.0:1789760768.836624:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14925 00000100:00100000:0.0:1789760768.836633:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712054660576 time 712070734177 state 0x2 ref 0 00000100:00100000:0.0:1789760768.840905:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760768.841007:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554673920 00000100:00100000:2.0:1789760768.841062:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712054660576 time 712070734177 state 0x0 ref 1 00000100:00100000:2.0:1789760768.841078:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712070994593 time 712070734177 state 0x1 ref 1 00000100:00100000:2.0:1789760768.841148:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712070994593 time 712086765107 state 0x0 ref 1 00000100:00100000:2.0:1789760768.841172:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554673920:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760768.841223:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61091840, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d230 00010000:00010000:2.0:1789760768.841233:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c420b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.841250:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760768.841256:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.841263:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c420b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61091840->61095935] (req 61091840->61095935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.841279:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c420b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61091840->61095935] (req 61091840->61095935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760768.852033:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982224 is committed 00002000:00010000:2.0:1789760768.852167:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61091840 -> 61095936 00002000:00010000:2.0:1789760768.852172:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119344 -> 119352 00010000:00010000:2.0:1789760768.852185:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c420b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61091840->61095935] (req 61091840->61095935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.852199:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c420b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61091840->61095935] (req 61091840->61095935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.852213:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c420b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61091840->61095935] (req 61091840->61095935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.852221:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760768.852225:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c420b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61091840->61095935] (req 61091840->61095935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.852234:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c420b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61091840->61095935] (req 61091840->61095935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.852254:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c420b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61091840->61095935] (req 61091840->61095935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.852272:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c420b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61091840->61095935] (req 61091840->61095935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.852372:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554673920:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11208us (11473us total) trans 4294982224 rc 0/0 00000100:00100000:2.0:1789760768.852420:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14926 00000100:00100000:2.0:1789760768.852430:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712070994593 time 712086765107 state 0x2 ref 0 00000100:00100000:2.0:1789760768.856152:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760768.865633:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982225 is committed 00000100:00100000:0.0:1789760768.874250:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760768.874757:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554674176 00000100:00100000:2.0:1789760768.874767:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554674176 00000100:00100000:2.0:1789760768.874831:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712087025523 time 712102462028 state 0x0 ref 1 00000100:00100000:2.0:1789760768.874846:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712102722444 time 712102462028 state 0x1 ref 1 00000100:00100000:2.0:1789760768.874879:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712102722444 time 712120497155 state 0x0 ref 1 00000100:00100000:2.0:1789760768.874892:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554674176:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760768.874939:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61100032, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d240 00010000:00010000:2.0:1789760768.874948:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4219 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.874965:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760768.874970:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.874976:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4219 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61100032->61104127] (req 61100032->61104127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.874992:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4219 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61100032->61104127] (req 61100032->61104127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760768.887615:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982226 is committed 00002000:00010000:2.0:1789760768.892977:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61100032 -> 61104128 00002000:00010000:2.0:1789760768.892987:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119360 -> 119368 00010000:00010000:2.0:1789760768.893002:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4219 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61100032->61104127] (req 61100032->61104127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.893016:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4219 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61100032->61104127] (req 61100032->61104127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.893031:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4219 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61100032->61104127] (req 61100032->61104127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.893039:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760768.893044:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4219 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61100032->61104127] (req 61100032->61104127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.893059:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4219 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61100032->61104127] (req 61100032->61104127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.893079:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4219 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61100032->61104127] (req 61100032->61104127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.893092:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4219 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61100032->61104127] (req 61100032->61104127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.893425:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554674176:12345-192.168.203.22@tcp:4:dd.0 Request processed in 18530us (19179us total) trans 4294982226 rc 0/0 00000100:00100000:2.0:1789760768.893438:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14928 00000100:00100000:2.0:1789760768.893447:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712102722444 time 712120497155 state 0x2 ref 0 00000100:00100000:0.0:1789760768.898875:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760768.898965:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554674304 00000100:00100000:2.0:1789760768.898975:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554674304 00000100:00100000:2.0:1789760768.899027:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712102722444 time 712120497155 state 0x0 ref 1 00000100:00100000:2.0:1789760768.899047:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712120757571 time 712120497155 state 0x1 ref 1 00000100:00100000:2.0:1789760768.899083:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712120757571 time 712144701286 state 0x0 ref 1 00000100:00100000:2.0:1789760768.899095:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554674304:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760768.899141:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61104128, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d248 00010000:00010000:2.0:1789760768.899151:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4220 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.899179:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760768.899184:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.899191:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4220 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61104128->61108223] (req 61104128->61108223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.899204:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4220 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61104128->61108223] (req 61104128->61108223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760768.905813:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982227 is committed 00002000:00010000:0.0:1789760768.905922:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61104128 -> 61108224 00002000:00010000:0.0:1789760768.905927:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119368 -> 119376 00010000:00010000:0.0:1789760768.905940:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4220 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61104128->61108223] (req 61104128->61108223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.905953:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4220 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61104128->61108223] (req 61104128->61108223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.905963:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4220 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61104128->61108223] (req 61104128->61108223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.905971:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760768.905976:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4220 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61104128->61108223] (req 61104128->61108223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.905986:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4220 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61104128->61108223] (req 61104128->61108223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.906007:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4220 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61104128->61108223] (req 61104128->61108223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.906015:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4220 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61104128->61108223] (req 61104128->61108223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760768.906124:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554674304:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7026us (7254us total) trans 4294982227 rc 0/0 00000100:00100000:0.0:1789760768.906136:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14929 00000100:00100000:0.0:1789760768.906145:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712120757571 time 712144701286 state 0x2 ref 0 00000100:00100000:3.0:1789760768.911365:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554674432 00000100:00100000:3.0:1789760768.911481:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554674432 00000100:00100000:2.0:1789760768.911677:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712120757571 time 712144701286 state 0x0 ref 1 00000100:00100000:2.0:1789760768.911692:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712144961702 time 712144701286 state 0x1 ref 1 00000100:00100000:2.0:1789760768.911746:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712144961702 time 712157363938 state 0x0 ref 1 00000100:00100000:2.0:1789760768.911759:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554674432:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760768.911818:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61108224, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d250 00010000:00010000:2.0:1789760768.911827:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4227 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.911843:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760768.911847:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.911854:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4227 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61108224->61112319] (req 61108224->61112319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.911868:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4227 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61108224->61112319] (req 61108224->61112319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.920832:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982228 is committed 00002000:00010000:2.0:1789760768.921649:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61108224 -> 61112320 00002000:00010000:2.0:1789760768.921655:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119376 -> 119384 00010000:00010000:2.0:1789760768.921668:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4227 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61108224->61112319] (req 61108224->61112319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.921691:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4227 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61108224->61112319] (req 61108224->61112319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.921700:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4227 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61108224->61112319] (req 61108224->61112319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.921707:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760768.921712:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4227 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61108224->61112319] (req 61108224->61112319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.921721:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4227 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61108224->61112319] (req 61108224->61112319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.921746:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4227 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61108224->61112319] (req 61108224->61112319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.921754:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4227 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61108224->61112319] (req 61108224->61112319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.921857:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554674432:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10094us (11096us total) trans 4294982228 rc 0/0 00000100:00100000:2.0:1789760768.921868:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14930 00000100:00100000:2.0:1789760768.921878:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712144961702 time 712157363938 state 0x2 ref 0 00000100:00100000:2.0:1789760768.932485:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760768.938800:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982229 is committed 00000100:00100000:0.0:1789760768.943750:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760768.943858:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554674688 00000100:00100000:2.0:1789760768.943868:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554674688 00000100:00100000:2.0:1789760768.943919:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712157624354 time 712178298639 state 0x0 ref 1 00000100:00100000:2.0:1789760768.943934:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712178559055 time 712178298639 state 0x1 ref 1 00000100:00100000:2.0:1789760768.943968:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712178559055 time 712189586816 state 0x0 ref 1 00000100:00100000:2.0:1789760768.943986:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554674688:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760768.944031:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61116416, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d260 00010000:00010000:2.0:1789760768.944040:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4235 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.944062:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760768.944067:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.944074:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4235 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61116416->61120511] (req 61116416->61120511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.944088:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4235 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61116416->61120511] (req 61116416->61120511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.949503:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982230 is committed 00002000:00010000:2.0:1789760768.949813:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61116416 -> 61120512 00002000:00010000:2.0:1789760768.949818:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119392 -> 119400 00010000:00010000:2.0:1789760768.949831:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4235 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61116416->61120511] (req 61116416->61120511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.949844:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4235 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61116416->61120511] (req 61116416->61120511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.949853:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4235 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61116416->61120511] (req 61116416->61120511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.949861:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760768.949866:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4235 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61116416->61120511] (req 61116416->61120511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.949875:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4235 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61116416->61120511] (req 61116416->61120511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.949899:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4235 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61116416->61120511] (req 61116416->61120511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.949907:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4235 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61116416->61120511] (req 61116416->61120511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.950002:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554674688:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6018us (6258us total) trans 4294982230 rc 0/0 00000100:00100000:2.0:1789760768.950012:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14932 00000100:00100000:2.0:1789760768.950022:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712178559055 time 712189586816 state 0x2 ref 0 00000100:00100000:2.0:1789760768.953253:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760768.953345:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554674816 00000100:00100000:0.0:1789760768.953354:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554674816 00000100:00100000:0.0:1789760768.953448:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712178559055 time 712189586816 state 0x0 ref 1 00000100:00100000:0.0:1789760768.953463:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712189847232 time 712189586816 state 0x1 ref 1 00000100:00100000:0.0:1789760768.953485:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712189847232 time 712199103720 state 0x0 ref 1 00000100:00100000:0.0:1789760768.953497:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554674816:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760768.953550:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61120512, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d268 00010000:00010000:0.0:1789760768.953560:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c423c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.953576:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760768.953581:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760768.953588:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c423c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61120512->61124607] (req 61120512->61124607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760768.953607:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c423c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61120512->61124607] (req 61120512->61124607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.958748:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982231 is committed 00002000:00010000:2.0:1789760768.958864:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61120512 -> 61124608 00002000:00010000:2.0:1789760768.958869:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119400 -> 119408 00010000:00010000:2.0:1789760768.958882:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c423c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61120512->61124607] (req 61120512->61124607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.958893:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c423c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61120512->61124607] (req 61120512->61124607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.958901:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c423c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61120512->61124607] (req 61120512->61124607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.958907:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760768.958912:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c423c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61120512->61124607] (req 61120512->61124607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.958921:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c423c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61120512->61124607] (req 61120512->61124607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.958938:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c423c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61120512->61124607] (req 61120512->61124607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.958945:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c423c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61120512->61124607] (req 61120512->61124607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.959035:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554674816:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5533us (5786us total) trans 4294982231 rc 0/0 00000100:00100000:2.0:1789760768.959046:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14933 00000100:00100000:2.0:1789760768.959055:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712189847232 time 712199103720 state 0x2 ref 0 00000100:00100000:0.0:1789760768.972292:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760768.980141:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554674944 00000100:00100000:2.0:1789760768.980165:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554674944 00000100:00100000:2.0:1789760768.980229:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712189847232 time 712199103720 state 0x0 ref 1 00000100:00100000:2.0:1789760768.980247:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712199364136 time 712199103720 state 0x1 ref 1 00000100:00100000:2.0:1789760768.980285:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712199364136 time 712225903705 state 0x0 ref 1 00000100:00100000:2.0:1789760768.980298:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554674944:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760768.980348:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61124608, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d270 00010000:00010000:2.0:1789760768.980364:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4243 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.980397:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760768.980403:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.980410:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4243 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61124608->61128703] (req 61124608->61128703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.980424:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4243 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61124608->61128703] (req 61124608->61128703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760768.985074:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982232 is committed 00002000:00010000:2.0:1789760768.985328:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61124608 -> 61128704 00002000:00010000:2.0:1789760768.985334:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119408 -> 119416 00010000:00010000:2.0:1789760768.985347:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4243 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61124608->61128703] (req 61124608->61128703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.985359:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4243 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61124608->61128703] (req 61124608->61128703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.985369:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4243 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61124608->61128703] (req 61124608->61128703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.985611:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760768.985617:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4243 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61124608->61128703] (req 61124608->61128703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.985628:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4243 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61124608->61128703] (req 61124608->61128703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.985653:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4243 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61124608->61128703] (req 61124608->61128703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.985660:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4243 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61124608->61128703] (req 61124608->61128703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760768.985798:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554674944:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5496us (13510us total) trans 4294982232 rc 0/0 00000100:00100000:2.0:1789760768.985809:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14934 00000100:00100000:2.0:1789760768.985819:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712199364136 time 712225903705 state 0x2 ref 0 00000100:00100000:0.0:1789760768.989889:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760768.989968:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554675072 00000100:00100000:2.0:1789760768.989978:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554675072 00000100:00100000:2.0:1789760768.990028:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712199364136 time 712225903705 state 0x0 ref 1 00000100:00100000:2.0:1789760768.990044:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712226164121 time 712225903705 state 0x1 ref 1 00000100:00100000:2.0:1789760768.990075:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712226164121 time 712235693656 state 0x0 ref 1 00000100:00100000:2.0:1789760768.990088:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554675072:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760768.990146:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61128704, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d278 00010000:00010000:2.0:1789760768.990164:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c424a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.990180:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760768.990186:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760768.990192:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c424a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61128704->61132799] (req 61128704->61132799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760768.990205:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c424a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61128704->61132799] (req 61128704->61132799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760768.994351:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982233 is committed 00000100:00100000:3.0:1789760769.000492:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554675200 00000100:00100000:3.0:1789760769.000502:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554675200 00000100:00100000:3.0:1789760769.000608:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712226164121 time 712235693656 state 0x0 ref 1 00000100:00100000:3.0:1789760769.000622:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712235954072 time 712235693656 state 0x1 ref 1 00000100:00100000:3.0:1789760769.000649:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712235954072 time 712246268403 state 0x0 ref 1 00000100:00100000:3.0:1789760769.000661:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554675200:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760769.000703:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61132800, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d280 00010000:00010000:3.0:1789760769.000711:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4251 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.000728:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760769.000733:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.000741:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4251 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61132800->61136895] (req 61132800->61136895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.000757:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4251 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61132800->61136895] (req 61132800->61136895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760769.001642:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760769.005395:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982234 is committed 00002000:00010000:3.0:1789760769.005507:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61132800 -> 61136896 00002000:00010000:3.0:1789760769.005513:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119424 -> 119432 00010000:00010000:3.0:1789760769.005526:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4251 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61132800->61136895] (req 61132800->61136895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.005549:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4251 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61132800->61136895] (req 61132800->61136895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.005560:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4251 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61132800->61136895] (req 61132800->61136895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.005569:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760769.005574:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4251 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61132800->61136895] (req 61132800->61136895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.005583:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4251 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61132800->61136895] (req 61132800->61136895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.005603:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4251 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61132800->61136895] (req 61132800->61136895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.005612:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4251 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61132800->61136895] (req 61132800->61136895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.005706:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554675200:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5041us (5301us total) trans 4294982234 rc 0/0 00000100:00100000:3.0:1789760769.005718:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14936 00000100:00100000:3.0:1789760769.005727:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712235954072 time 712246268403 state 0x2 ref 0 00000100:00100000:2.0:1789760769.010035:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760769.010862:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554675328 00000100:00100000:3.0:1789760769.010874:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554675328 00000100:00100000:3.0:1789760769.010930:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712235954072 time 712246268403 state 0x0 ref 1 00000100:00100000:3.0:1789760769.010945:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712246528819 time 712246268403 state 0x1 ref 1 00000100:00100000:3.0:1789760769.010962:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712246528819 time 712256581443 state 0x0 ref 1 00000100:00100000:3.0:1789760769.010974:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554675328:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760769.011019:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61136896, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d288 00010000:00010000:3.0:1789760769.011028:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4258 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.011045:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760769.011050:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.011056:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4258 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61136896->61140991] (req 61136896->61140991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.011069:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4258 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61136896->61140991] (req 61136896->61140991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760769.018637:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982235 is committed 00002000:00010000:3.0:1789760769.018758:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61136896 -> 61140992 00002000:00010000:3.0:1789760769.018764:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119432 -> 119440 00010000:00010000:3.0:1789760769.018777:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4258 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61136896->61140991] (req 61136896->61140991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.018799:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4258 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61136896->61140991] (req 61136896->61140991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.018808:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4258 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61136896->61140991] (req 61136896->61140991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.018821:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760769.018826:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4258 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61136896->61140991] (req 61136896->61140991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.018835:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4258 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61136896->61140991] (req 61136896->61140991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.018855:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4258 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61136896->61140991] (req 61136896->61140991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.018862:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4258 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61136896->61140991] (req 61136896->61140991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.018974:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554675328:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7997us (8943us total) trans 4294982235 rc 0/0 00000100:00100000:3.0:1789760769.018985:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14937 00000100:00100000:3.0:1789760769.018995:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712246528819 time 712256581443 state 0x2 ref 0 00000100:00100000:0.0:1789760769.022767:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760769.022845:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554675456 00000100:00100000:2.0:1789760769.022855:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554675456 00000100:00100000:2.0:1789760769.022911:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712246528819 time 712256581443 state 0x0 ref 1 00000100:00100000:2.0:1789760769.022926:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712256841859 time 712256581443 state 0x1 ref 1 00000100:00100000:2.0:1789760769.022989:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712256841859 time 712268606972 state 0x0 ref 1 00000100:00100000:2.0:1789760769.023002:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554675456:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760769.023049:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61140992, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d290 00010000:00010000:2.0:1789760769.023059:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c425f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.023075:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760769.023080:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.023086:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c425f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61140992->61145087] (req 61140992->61145087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.023117:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c425f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61140992->61145087] (req 61140992->61145087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760769.030725:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61140992 -> 61145088 00002000:00010000:2.0:1789760769.030731:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119440 -> 119448 00010000:00010000:2.0:1789760769.030744:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c425f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61140992->61145087] (req 61140992->61145087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.030758:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c425f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61140992->61145087] (req 61140992->61145087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.030768:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c425f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61140992->61145087] (req 61140992->61145087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.030775:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760769.030792:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c425f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61140992->61145087] (req 61140992->61145087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.030802:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c425f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61140992->61145087] (req 61140992->61145087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.030830:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c425f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61140992->61145087] (req 61140992->61145087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.030839:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c425f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61140992->61145087] (req 61140992->61145087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.030935:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554675456:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7930us (8174us total) trans 4294982236 rc 0/0 00000100:00100000:2.0:1789760769.030947:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14938 00000100:00100000:2.0:1789760769.030956:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712256841859 time 712268606972 state 0x2 ref 0 00000100:00100000:2.0:1789760769.034794:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:2.0:1789760769.043269:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61145088 -> 61149184 00002000:00010000:2.0:1789760769.043275:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119448 -> 119456 00010000:00010000:2.0:1789760769.043289:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4266 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61145088->61149183] (req 61145088->61149183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.043302:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4266 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61145088->61149183] (req 61145088->61149183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.043311:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4266 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61145088->61149183] (req 61145088->61149183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.043318:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760769.043322:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4266 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61145088->61149183] (req 61145088->61149183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.043330:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4266 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61145088->61149183] (req 61145088->61149183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.043348:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4266 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61145088->61149183] (req 61145088->61149183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.043356:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4266 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61145088->61149183] (req 61145088->61149183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.043525:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554675584:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8481us (8743us total) trans 4294982237 rc 0/0 00000100:00100000:2.0:1789760769.043547:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14939 00000100:00100000:2.0:1789760769.043557:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712268867388 time 712280646230 state 0x2 ref 0 00000100:00100000:0.0:1789760769.051065:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760769.051620:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554675712 00000100:00100000:3.0:1789760769.051630:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554675712 00000100:00100000:3.0:1789760769.051685:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712268867388 time 712280646230 state 0x0 ref 1 00000100:00100000:3.0:1789760769.051700:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712280906646 time 712280646230 state 0x1 ref 1 00000100:00100000:3.0:1789760769.051724:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712280906646 time 712297342791 state 0x0 ref 1 00000100:00100000:3.0:1789760769.051741:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554675712:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760769.051804:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61149184, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d2a0 00010000:00010000:3.0:1789760769.051814:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c426d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.051830:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760769.051835:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.051841:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c426d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61149184->61153279] (req 61149184->61153279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.051855:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c426d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61149184->61153279] (req 61149184->61153279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760769.056996:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982238 is committed 00002000:00010000:3.0:1789760769.057234:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61149184 -> 61153280 00002000:00010000:3.0:1789760769.057239:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119456 -> 119464 00010000:00010000:3.0:1789760769.057252:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c426d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61149184->61153279] (req 61149184->61153279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.057273:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c426d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61149184->61153279] (req 61149184->61153279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.057283:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c426d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61149184->61153279] (req 61149184->61153279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.057291:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760769.057298:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c426d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61149184->61153279] (req 61149184->61153279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.057307:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c426d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61149184->61153279] (req 61149184->61153279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.057325:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c426d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61149184->61153279] (req 61149184->61153279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.057332:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c426d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61149184->61153279] (req 61149184->61153279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.057714:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554675712:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5970us (6652us total) trans 4294982238 rc 0/0 00000100:00100000:2.0:1789760769.057725:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14940 00000100:00100000:2.0:1789760769.057735:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712280906646 time 712297342791 state 0x2 ref 0 00000100:00100000:0.0:1789760769.062179:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760769.062364:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554675840 00000100:00100000:3.0:1789760769.062373:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554675840 00000100:00100000:3.0:1789760769.062465:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712280906646 time 712297342791 state 0x0 ref 1 00000100:00100000:3.0:1789760769.062480:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712297603207 time 712297342791 state 0x1 ref 1 00000100:00100000:3.0:1789760769.062561:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712297603207 time 712308178631 state 0x0 ref 1 00000100:00100000:3.0:1789760769.062574:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554675840:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760769.062619:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61153280, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d2a8 00010000:00010000:3.0:1789760769.062632:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4274 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.062648:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760769.062653:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.062660:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4274 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61153280->61157375] (req 61153280->61157375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.062674:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4274 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61153280->61157375] (req 61153280->61157375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760769.075325:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982239 is committed 00002000:00010000:3.0:1789760769.075525:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61153280 -> 61157376 00002000:00010000:3.0:1789760769.075531:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119464 -> 119472 00010000:00010000:3.0:1789760769.075544:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4274 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61153280->61157375] (req 61153280->61157375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.075558:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4274 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61153280->61157375] (req 61153280->61157375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.075567:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4274 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61153280->61157375] (req 61153280->61157375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.075574:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760769.075579:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4274 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61153280->61157375] (req 61153280->61157375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.075589:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4274 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61153280->61157375] (req 61153280->61157375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.075608:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4274 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61153280->61157375] (req 61153280->61157375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.075622:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4274 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61153280->61157375] (req 61153280->61157375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.075718:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554675840:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13141us (13543us total) trans 4294982239 rc 0/0 00000100:00100000:3.0:1789760769.075730:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14941 00000100:00100000:3.0:1789760769.075739:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712297603207 time 712308178631 state 0x2 ref 0 00000100:00100000:3.0:1789760769.084205:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760769.084314:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554675968 00000100:00100000:2.0:1789760769.084325:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554675968 00000100:00100000:2.0:1789760769.084417:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712297603207 time 712308178631 state 0x0 ref 1 00000100:00100000:2.0:1789760769.084433:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712308439047 time 712308178631 state 0x1 ref 1 00000100:00100000:2.0:1789760769.084458:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712308439047 time 712330076218 state 0x0 ref 1 00000100:00100000:2.0:1789760769.084471:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554675968:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760769.084522:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61157376, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d2b0 00010000:00010000:2.0:1789760769.084532:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c427b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.084553:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760769.084558:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.084564:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c427b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61157376->61161471] (req 61157376->61161471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.084581:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c427b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61157376->61161471] (req 61157376->61161471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760769.090643:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982240 is committed 00002000:00010000:0.0:1789760769.090756:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61157376 -> 61161472 00002000:00010000:0.0:1789760769.090762:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119472 -> 119480 00010000:00010000:0.0:1789760769.090775:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c427b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61157376->61161471] (req 61157376->61161471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.090797:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c427b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61157376->61161471] (req 61157376->61161471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.090805:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c427b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61157376->61161471] (req 61157376->61161471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.090812:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760769.090817:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c427b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61157376->61161471] (req 61157376->61161471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.090827:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c427b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61157376->61161471] (req 61157376->61161471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.090847:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c427b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61157376->61161471] (req 61157376->61161471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.090855:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c427b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61157376->61161471] (req 61157376->61161471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.090950:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554675968:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6477us (6749us total) trans 4294982240 rc 0/0 00000100:00100000:0.0:1789760769.090961:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14942 00000100:00100000:0.0:1789760769.090971:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712308439047 time 712330076218 state 0x2 ref 0 00000100:00100000:0.0:1789760769.100101:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760769.100355:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554676096 00000100:00100000:2.0:1789760769.100364:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554676096 00000100:00100000:2.0:1789760769.100595:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712308439047 time 712330076218 state 0x0 ref 1 00000100:00100000:2.0:1789760769.100612:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712330336634 time 712330076218 state 0x1 ref 1 00000100:00100000:2.0:1789760769.100649:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712330336634 time 712346266310 state 0x0 ref 1 00000100:00100000:2.0:1789760769.100661:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe1f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554676096:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760769.100711:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61161472, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d2b8 00010000:00010000:2.0:1789760769.100721:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4282 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.100821:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760769.100826:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.100833:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4282 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61161472->61165567] (req 61161472->61165567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.100983:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4282 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61161472->61165567] (req 61161472->61165567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760769.110979:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61161472 -> 61165568 00002000:00010000:2.0:1789760769.110984:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119480 -> 119488 00010000:00010000:2.0:1789760769.110998:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4282 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61161472->61165567] (req 61161472->61165567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.111011:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4282 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61161472->61165567] (req 61161472->61165567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.111020:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4282 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61161472->61165567] (req 61161472->61165567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.111027:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760769.111032:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4282 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61161472->61165567] (req 61161472->61165567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.111041:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4282 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61161472->61165567] (req 61161472->61165567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.111059:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4282 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61161472->61165567] (req 61161472->61165567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.111067:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4282 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61161472->61165567] (req 61161472->61165567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.111176:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe1f050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554676096:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10512us (11081us total) trans 4294982241 rc 0/0 00000100:00100000:2.0:1789760769.111188:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14943 00000100:00100000:2.0:1789760769.111197:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712330336634 time 712346266310 state 0x2 ref 0 00000100:00100000:2.0:1789760769.114787:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760769.116059:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554676224 00000100:00100000:0.0:1789760769.116071:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554676224 00000100:00100000:0.0:1789760769.116142:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712330336634 time 712346266310 state 0x0 ref 1 00000100:00100000:0.0:1789760769.116159:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712346526726 time 712346266310 state 0x1 ref 1 00000100:00100000:0.0:1789760769.116193:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712346526726 time 712361811153 state 0x0 ref 1 00000100:00100000:0.0:1789760769.116207:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554676224:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760769.116255:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61165568, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d2c0 00010000:00010000:0.0:1789760769.116264:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4289 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.116281:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760769.116285:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.116291:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4289 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61165568->61169663] (req 61165568->61169663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.116303:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4289 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61165568->61169663] (req 61165568->61169663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760769.122181:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61165568 -> 61169664 00002000:00010000:0.0:1789760769.122186:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119488 -> 119496 00010000:00010000:0.0:1789760769.122199:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4289 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61165568->61169663] (req 61165568->61169663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.122212:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4289 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61165568->61169663] (req 61165568->61169663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.122221:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4289 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61165568->61169663] (req 61165568->61169663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.122229:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760769.122234:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4289 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61165568->61169663] (req 61165568->61169663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.122244:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4289 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61165568->61169663] (req 61165568->61169663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.122266:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4289 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61165568->61169663] (req 61165568->61169663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.122274:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4289 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61165568->61169663] (req 61165568->61169663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.122544:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554676224:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6334us (7771us total) trans 4294982242 rc 0/0 00000100:00100000:0.0:1789760769.122555:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14944 00000100:00100000:0.0:1789760769.122563:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712346526726 time 712361811153 state 0x2 ref 0 00000100:00100000:3.0:1789760769.128346:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760769.130726:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554676352 00000100:00100000:0.0:1789760769.130740:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554676352 00000001:00080000:2.0:1789760769.137263:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982243 is committed 00000100:00100000:2.0:1789760769.152335:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760769.152470:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554676480 00000100:00100000:0.0:1789760769.152481:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554676480 00000100:00100000:0.0:1789760769.152533:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712362071569 time 712377529312 state 0x0 ref 1 00000100:00100000:0.0:1789760769.152549:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712377789728 time 712377529312 state 0x1 ref 1 00000100:00100000:0.0:1789760769.152580:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712377789728 time 712398198781 state 0x0 ref 1 00000100:00100000:0.0:1789760769.152592:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554676480:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760769.152637:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61173760, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d2d0 00010000:00010000:0.0:1789760769.152646:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4297 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.152668:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760769.152672:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.152679:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4297 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61173760->61177855] (req 61173760->61177855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.152692:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4297 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61173760->61177855] (req 61173760->61177855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760769.157769:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61173760 -> 61177856 00002000:00010000:0.0:1789760769.157774:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119504 -> 119512 00010000:00010000:0.0:1789760769.157796:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4297 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61173760->61177855] (req 61173760->61177855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.157809:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4297 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61173760->61177855] (req 61173760->61177855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.157818:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4297 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61173760->61177855] (req 61173760->61177855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.157825:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760769.157830:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4297 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61173760->61177855] (req 61173760->61177855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.157838:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4297 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61173760->61177855] (req 61173760->61177855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.157864:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4297 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61173760->61177855] (req 61173760->61177855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.157872:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4297 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61173760->61177855] (req 61173760->61177855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.157968:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554676480:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5372us (5640us total) trans 4294982244 rc 0/0 00000100:00100000:0.0:1789760769.157980:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14946 00000100:00100000:0.0:1789760769.157989:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712377789728 time 712398198781 state 0x2 ref 0 00000100:00100000:2.0:1789760769.164976:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:2.0:1789760769.170466:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61177856 -> 61181952 00002000:00010000:2.0:1789760769.170471:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119512 -> 119520 00010000:00010000:2.0:1789760769.170484:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c429e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61177856->61181951] (req 61177856->61181951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.170496:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c429e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61177856->61181951] (req 61177856->61181951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.170504:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c429e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61177856->61181951] (req 61177856->61181951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.170511:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760769.170516:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c429e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61177856->61181951] (req 61177856->61181951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.170524:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c429e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61177856->61181951] (req 61177856->61181951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.170552:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c429e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61177856->61181951] (req 61177856->61181951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.170560:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c429e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61177856->61181951] (req 61177856->61181951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.170684:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554676608:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5473us (5712us total) trans 4294982245 rc 0/0 00000100:00100000:2.0:1789760769.170694:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14947 00000100:00100000:2.0:1789760769.170712:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712398459197 time 712410814414 state 0x2 ref 0 00000100:00100000:3.0:1789760769.175532:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760769.182217:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982246 is committed 00000100:00100000:0.0:1789760769.189532:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760769.189869:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554676864 00000100:00100000:3.0:1789760769.189879:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554676864 00000100:00100000:3.0:1789760769.189932:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712411074830 time 712421441286 state 0x0 ref 1 00000100:00100000:3.0:1789760769.189949:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712421701702 time 712421441286 state 0x1 ref 1 00000100:00100000:3.0:1789760769.189984:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712421701702 time 712435601644 state 0x0 ref 1 00000100:00100000:3.0:1789760769.189997:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554676864:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760769.190041:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61186048, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d2e8 00010000:00010000:3.0:1789760769.190051:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c42ac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.190073:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760769.190078:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.190085:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c42ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61186048->61190143] (req 61186048->61190143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.190100:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c42ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61186048->61190143] (req 61186048->61190143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760769.195131:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982247 is committed 00002000:00010000:3.0:1789760769.195334:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61186048 -> 61190144 00002000:00010000:3.0:1789760769.195341:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119528 -> 119536 00010000:00010000:3.0:1789760769.195355:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c42ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61186048->61190143] (req 61186048->61190143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.195370:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c42ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61186048->61190143] (req 61186048->61190143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.195477:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c42ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61186048->61190143] (req 61186048->61190143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.195487:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760769.195492:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c42ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61186048->61190143] (req 61186048->61190143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.195503:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c42ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61186048->61190143] (req 61186048->61190143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.195530:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c42ac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61186048->61190143] (req 61186048->61190143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.195611:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c42ac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61186048->61190143] (req 61186048->61190143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.195752:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554676864:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5751us (6233us total) trans 4294982247 rc 0/0 00000100:00100000:3.0:1789760769.195763:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14949 00000100:00100000:3.0:1789760769.195772:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712421701702 time 712435601644 state 0x2 ref 0 00000100:00100000:2.0:1789760769.204006:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760769.204133:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554676992 00000100:00100000:0.0:1789760769.204143:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554676992 00000100:00100000:0.0:1789760769.204349:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712421701702 time 712435601644 state 0x0 ref 1 00000100:00100000:0.0:1789760769.204365:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712435862060 time 712435601644 state 0x1 ref 1 00000001:00080000:3.0:1789760769.210354:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982248 is committed 00002000:00010000:0.0:1789760769.210497:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61190144 -> 61194240 00002000:00010000:0.0:1789760769.210502:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119536 -> 119544 00010000:00010000:0.0:1789760769.210515:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c42b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61190144->61194239] (req 61190144->61194239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.210528:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c42b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61190144->61194239] (req 61190144->61194239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.210551:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c42b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61190144->61194239] (req 61190144->61194239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.210559:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760769.210563:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c42b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61190144->61194239] (req 61190144->61194239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.210572:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c42b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61190144->61194239] (req 61190144->61194239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.210591:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c42b3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61190144->61194239] (req 61190144->61194239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.210598:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c42b3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61190144->61194239] (req 61190144->61194239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.210687:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554676992:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4991us (6687us total) trans 4294982248 rc 0/0 00000100:00100000:0.0:1789760769.210698:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14950 00000100:00100000:0.0:1789760769.210708:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712435862060 time 712451285150 state 0x2 ref 0 00000100:00100000:2.0:1789760769.215582:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760769.227349:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982249 is committed 00000100:00100000:3.0:1789760769.232718:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760769.232898:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554677248 00000100:00100000:0.0:1789760769.232909:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554677248 00000100:00100000:0.0:1789760769.232976:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712451545566 time 712461878008 state 0x0 ref 1 00000100:00100000:0.0:1789760769.232993:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712462138424 time 712461878008 state 0x1 ref 1 00000100:00100000:0.0:1789760769.233027:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712462138424 time 712478645577 state 0x0 ref 1 00000100:00100000:0.0:1789760769.233039:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554677248:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760769.233098:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61198336, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d300 00010000:00010000:0.0:1789760769.233107:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c42c1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.233124:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760769.233129:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.233135:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c42c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61198336->61202431] (req 61198336->61202431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.233150:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c42c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61198336->61202431] (req 61198336->61202431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.242842:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982250 is committed 00002000:00010000:2.0:1789760769.242964:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61198336 -> 61202432 00002000:00010000:2.0:1789760769.242970:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119552 -> 119560 00010000:00010000:2.0:1789760769.242983:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c42c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61198336->61202431] (req 61198336->61202431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.242996:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c42c1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61198336->61202431] (req 61198336->61202431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.243006:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c42c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61198336->61202431] (req 61198336->61202431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.243014:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760769.243019:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c42c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61198336->61202431] (req 61198336->61202431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.243029:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c42c1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61198336->61202431] (req 61198336->61202431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.243048:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c42c1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61198336->61202431] (req 61198336->61202431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.243056:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c42c1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61198336->61202431] (req 61198336->61202431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.243204:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554677248:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10160us (10491us total) trans 4294982250 rc 0/0 00000100:00100000:2.0:1789760769.243215:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14952 00000100:00100000:2.0:1789760769.243223:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712462138424 time 712478645577 state 0x2 ref 0 00000100:00100000:2.0:1789760769.246871:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760769.246989:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554677376 00000100:00100000:3.0:1789760769.247000:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554677376 00000100:00100000:3.0:1789760769.247057:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712462138424 time 712478645577 state 0x0 ref 1 00000100:00100000:3.0:1789760769.247073:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712478905993 time 712478645577 state 0x1 ref 1 00000100:00100000:3.0:1789760769.247108:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712478905993 time 712492726851 state 0x0 ref 1 00000100:00100000:3.0:1789760769.247120:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554677376:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760769.247280:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61202432, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d308 00010000:00010000:3.0:1789760769.247291:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c42c8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.247307:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760769.247311:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.247318:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c42c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61202432->61206527] (req 61202432->61206527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.247332:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c42c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61202432->61206527] (req 61202432->61206527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.253123:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982251 is committed 00002000:00010000:2.0:1789760769.253231:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61202432 -> 61206528 00002000:00010000:2.0:1789760769.253236:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119560 -> 119568 00010000:00010000:2.0:1789760769.253250:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c42c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61202432->61206527] (req 61202432->61206527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.253264:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c42c8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61202432->61206527] (req 61202432->61206527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.253274:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c42c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61202432->61206527] (req 61202432->61206527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.253282:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760769.253287:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c42c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61202432->61206527] (req 61202432->61206527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.253297:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c42c8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61202432->61206527] (req 61202432->61206527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.253317:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c42c8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61202432->61206527] (req 61202432->61206527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.253325:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c42c8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61202432->61206527] (req 61202432->61206527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.253449:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554677376:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6326us (6582us total) trans 4294982251 rc 0/0 00000100:00100000:2.0:1789760769.253459:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14953 00000100:00100000:2.0:1789760769.253468:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712478905993 time 712492726851 state 0x2 ref 0 00000100:00100000:3.0:1789760769.259879:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760769.264870:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982252 is committed 00000100:00100000:3.0:1789760769.268246:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760769.273939:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982253 is committed 00000100:00100000:2.0:1789760769.279148:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760769.279268:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554677760 00000100:00100000:0.0:1789760769.279278:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554677760 00000100:00100000:0.0:1789760769.279339:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712505996464 time 712514092829 state 0x0 ref 1 00000100:00100000:0.0:1789760769.279355:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712514353245 time 712514092829 state 0x1 ref 1 00000100:00100000:0.0:1789760769.282207:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712514353245 time 712527812470 state 0x0 ref 1 00000100:00100000:0.0:1789760769.282237:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554677760:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760769.282293:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61214720, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d320 00010000:00010000:0.0:1789760769.282303:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c42dd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.282319:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760769.282323:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.282330:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c42dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61214720->61218815] (req 61214720->61218815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.282344:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c42dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61214720->61218815] (req 61214720->61218815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760769.290336:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982254 is committed 00002000:00010000:0.0:1789760769.290467:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61214720 -> 61218816 00002000:00010000:0.0:1789760769.290473:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119584 -> 119592 00010000:00010000:0.0:1789760769.290488:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c42dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61214720->61218815] (req 61214720->61218815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.290502:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c42dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61214720->61218815] (req 61214720->61218815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.290512:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c42dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61214720->61218815] (req 61214720->61218815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.290519:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760769.290524:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c42dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61214720->61218815] (req 61214720->61218815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.290533:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c42dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61214720->61218815] (req 61214720->61218815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.290554:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c42dd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61214720->61218815] (req 61214720->61218815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.290562:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c42dd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61214720->61218815] (req 61214720->61218815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.290669:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554677760:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8440us (11526us total) trans 4294982254 rc 0/0 00000100:00100000:0.0:1789760769.290679:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14956 00000100:00100000:0.0:1789760769.290688:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712514353245 time 712527812470 state 0x2 ref 0 00000100:00100000:3.0:1789760769.295669:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760769.298234:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554677888 00000100:00100000:2.0:1789760769.298252:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554677888 00000100:00100000:2.0:1789760769.298317:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712514353245 time 712527812470 state 0x0 ref 1 00000100:00100000:2.0:1789760769.298445:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712528072886 time 712527812470 state 0x1 ref 1 00000100:00100000:2.0:1789760769.298540:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712528072886 time 712544156892 state 0x0 ref 1 00000100:00100000:2.0:1789760769.298556:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554677888:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760769.298605:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61218816, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d328 00010000:00010000:2.0:1789760769.298617:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c42e4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.298633:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760769.298637:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.298645:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c42e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61218816->61222911] (req 61218816->61222911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.298660:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c42e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61218816->61222911] (req 61218816->61222911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760769.304815:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982255 is committed 00002000:00010000:2.0:1789760769.304946:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61218816 -> 61222912 00002000:00010000:2.0:1789760769.304951:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119592 -> 119600 00010000:00010000:2.0:1789760769.304965:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c42e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61218816->61222911] (req 61218816->61222911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.304977:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c42e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61218816->61222911] (req 61218816->61222911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.304986:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c42e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61218816->61222911] (req 61218816->61222911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.304999:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760769.305005:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c42e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61218816->61222911] (req 61218816->61222911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.305014:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c42e4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61218816->61222911] (req 61218816->61222911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.305033:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c42e4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61218816->61222911] (req 61218816->61222911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.305041:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c42e4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61218816->61222911] (req 61218816->61222911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.305183:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554677888:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6626us (9518us total) trans 4294982255 rc 0/0 00000100:00100000:2.0:1789760769.305194:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14957 00000100:00100000:2.0:1789760769.305203:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712528072886 time 712544156892 state 0x2 ref 0 00000100:00100000:3.0:1789760769.311673:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:0.0:1789760769.320816:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982256 is committed 00000100:00100000:0.0:1789760769.337546:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760769.343664:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982257 is committed 00000100:00100000:3.0:1789760769.349205:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760769.349295:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554678272 00000100:00100000:0.0:1789760769.349310:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554678272 00000100:00100000:0.0:1789760769.349512:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712558154049 time 712583427089 state 0x0 ref 1 00000100:00100000:0.0:1789760769.349536:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712583687505 time 712583427089 state 0x1 ref 1 00000100:00100000:0.0:1789760769.349596:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712583687505 time 712595214151 state 0x0 ref 1 00000100:00100000:0.0:1789760769.349611:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e54e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554678272:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760769.349659:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61231104, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d340 00010000:00010000:0.0:1789760769.349668:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c42f9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.349683:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760769.349688:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.349694:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c42f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61231104->61235199] (req 61231104->61235199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.349710:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c42f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61231104->61235199] (req 61231104->61235199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760769.357852:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982258 is committed 00002000:00010000:2.0:1789760769.358616:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61231104 -> 61235200 00002000:00010000:2.0:1789760769.358621:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119616 -> 119624 00010000:00010000:2.0:1789760769.358632:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c42f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61231104->61235199] (req 61231104->61235199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.358642:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c42f9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61231104->61235199] (req 61231104->61235199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.358649:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c42f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61231104->61235199] (req 61231104->61235199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.358654:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760769.358657:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c42f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61231104->61235199] (req 61231104->61235199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.358663:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c42f9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61231104->61235199] (req 61231104->61235199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.358677:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c42f9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61231104->61235199] (req 61231104->61235199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.358682:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c42f9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61231104->61235199] (req 61231104->61235199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.358964:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554678272:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9352us (9769us total) trans 4294982258 rc 0/0 00000100:00100000:2.0:1789760769.358976:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14960 00000100:00100000:2.0:1789760769.358985:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712583687505 time 712595214151 state 0x2 ref 0 00000100:00100000:3.0:1789760769.373608:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760769.375083:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554678400 00000100:00100000:2.0:1789760769.375095:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554678400 00000100:00100000:2.0:1789760769.375154:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712583687505 time 712595214151 state 0x0 ref 1 00000100:00100000:2.0:1789760769.375178:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712595474567 time 712595214151 state 0x1 ref 1 00000100:00100000:2.0:1789760769.375213:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712595474567 time 712620831309 state 0x0 ref 1 00000100:00100000:2.0:1789760769.375226:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554678400:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760769.375280:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61235200, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d348 00010000:00010000:2.0:1789760769.375290:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4300 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.375307:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760769.375312:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.375319:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4300 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61235200->61239295] (req 61235200->61239295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.375333:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4300 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61235200->61239295] (req 61235200->61239295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760769.387892:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61235200 -> 61239296 00002000:00010000:2.0:1789760769.387896:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119624 -> 119632 00010000:00010000:2.0:1789760769.387910:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4300 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61235200->61239295] (req 61235200->61239295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.387922:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4300 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61235200->61239295] (req 61235200->61239295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.387931:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4300 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61235200->61239295] (req 61235200->61239295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.387943:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760769.387948:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4300 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61235200->61239295] (req 61235200->61239295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.387957:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4300 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61235200->61239295] (req 61235200->61239295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.387978:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4300 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61235200->61239295] (req 61235200->61239295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.387985:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4300 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61235200->61239295] (req 61235200->61239295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.388085:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554678400:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12858us (14484us total) trans 4294982259 rc 0/0 00000100:00100000:2.0:1789760769.388097:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14961 00000100:00100000:2.0:1789760769.388106:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712595474567 time 712620831309 state 0x2 ref 0 00000100:00100000:0.0:1789760769.392884:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760769.402226:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982260 is committed 00000100:00100000:3.0:1789760769.406261:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:0.0:1789760769.413801:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982261 is committed 00000100:00100000:3.0:1789760769.418586:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760769.419130:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554678784 00000100:00100000:2.0:1789760769.419144:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554678784 00000100:00100000:3.0:1789760769.419365:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712638969213 time 712653032745 state 0x0 ref 1 00000100:00100000:3.0:1789760769.419491:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712653293161 time 712653032745 state 0x1 ref 1 00000100:00100000:3.0:1789760769.419566:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712653293161 time 712665183173 state 0x0 ref 1 00000100:00100000:3.0:1789760769.419585:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e54000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554678784:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760769.419651:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61247488, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d360 00010000:00010000:3.0:1789760769.419662:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4315 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.419682:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760769.419688:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.419695:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4315 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61247488->61251583] (req 61247488->61251583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.419849:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4315 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61247488->61251583] (req 61247488->61251583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.426960:0:3647:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03ebf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3647:1876699573912320:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789760769.427195:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760769.427347:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760769.430670:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573912320 00000100:00100000:2.0:1789760769.430686:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573912320 00000100:00100000:2.0:1789760769.430801:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573912192 00000100:00100000:2.0:1789760769.430805:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573912192 00000100:00100000:2.0:1789760769.430829:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 352 00000100:00100000:2.0:1789760769.430838:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3647:x1876699573912320:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1789760769.430869:0:8440:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484945600 pending 28672 free 3862683648 avail 3628265472 00000100:00100000:2.0:1789760769.430936:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3647:x1876699573912320:12345-0@lo:13:osp-pre-0-1.0 Request processed in 98us (3740us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760769.430944:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 352 00000100:00100000:2.0:1789760769.430960:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 353 00000100:00100000:2.0:1789760769.430966:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573912192:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789760769.430974:0:8440:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484945600 pending 28672 free 3862683648 avail 3628265472 00000100:00100000:2.0:1789760769.431000:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573912192:12345-0@lo:13:osp-pre-0-0.0 Request processed in 35us (3652us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760769.431005:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 353 00000100:00080000:0.0:1789760769.431994:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760769.432076:0:3647:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03ebf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3647:1876699573912320:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789760769.432134:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03ebd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573912192:0@lo:13:osp-pre-0-0.0 00000001:00080000:3.0:1789760769.438356:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982262 is committed 00002000:00010000:2.0:1789760769.438517:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61247488 -> 61251584 00002000:00010000:2.0:1789760769.438526:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119648 -> 119656 00010000:00010000:2.0:1789760769.438544:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4315 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61247488->61251583] (req 61247488->61251583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.438559:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4315 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61247488->61251583] (req 61247488->61251583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.438588:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4315 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61247488->61251583] (req 61247488->61251583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.438597:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760769.438603:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4315 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61247488->61251583] (req 61247488->61251583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.438614:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4315 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61247488->61251583] (req 61247488->61251583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.438642:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4315 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61247488->61251583] (req 61247488->61251583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.438651:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4315 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61247488->61251583] (req 61247488->61251583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.439540:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554678784:12345-192.168.203.22@tcp:4:dd.0 Request processed in 19947us (20958us total) trans 4294982262 rc 0/0 00000100:00100000:2.0:1789760769.439556:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14964 00000100:00100000:2.0:1789760769.439585:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712653293161 time 712665183173 state 0x2 ref 0 00000100:00100000:0.0:1789760769.442749:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760769.452498:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982263 is committed 00000100:00100000:3.0:1789760769.459052:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760769.468788:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982264 is committed 00002000:00010000:0.0:1789760769.468899:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61255680 -> 61259776 00002000:00010000:0.0:1789760769.468905:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119664 -> 119672 00010000:00010000:0.0:1789760769.468917:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c4323 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61255680->61259775] (req 61255680->61259775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.468930:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c4323 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61255680->61259775] (req 61255680->61259775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.468945:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c4323 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61255680->61259775] (req 61255680->61259775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.468954:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760769.468959:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c4323 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61255680->61259775] (req 61255680->61259775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.468968:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c4323 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61255680->61259775] (req 61255680->61259775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.468988:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c4323 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61255680->61259775] (req 61255680->61259775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.468996:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c4323 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61255680->61259775] (req 61255680->61259775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.469103:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e56300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554679040:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9822us (10056us total) trans 4294982264 rc 0/0 00000100:00100000:0.0:1789760769.469115:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14966 00000100:00100000:0.0:1789760769.469125:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712688846054 time 712704883447 state 0x2 ref 0 00000100:00100000:3.0:1789760769.475935:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760769.483737:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982265 is committed 00000100:00100000:2.0:1789760769.497498:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760769.497649:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554679296 00000100:00100000:0.0:1789760769.497660:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554679296 00000100:00100000:0.0:1789760769.497715:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712705143863 time 712722177638 state 0x0 ref 1 00000100:00100000:0.0:1789760769.497731:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712722438054 time 712722177638 state 0x1 ref 1 00000100:00100000:0.0:1789760769.497767:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712722438054 time 712743385813 state 0x0 ref 1 00000100:00100000:0.0:1789760769.497791:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554679296:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760769.497843:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61263872, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d380 00010000:00010000:0.0:1789760769.497853:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4331 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.497871:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760769.497876:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.497883:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4331 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61263872->61267967] (req 61263872->61267967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.497898:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4331 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61263872->61267967] (req 61263872->61267967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.507229:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982266 is committed 00002000:00010000:0.0:1789760769.507360:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61263872 -> 61267968 00002000:00010000:0.0:1789760769.507365:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119680 -> 119688 00010000:00010000:0.0:1789760769.508001:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4331 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61263872->61267967] (req 61263872->61267967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.508017:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4331 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61263872->61267967] (req 61263872->61267967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.508027:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4331 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61263872->61267967] (req 61263872->61267967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.508041:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760769.508046:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4331 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61263872->61267967] (req 61263872->61267967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.508056:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4331 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61263872->61267967] (req 61263872->61267967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.508080:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4331 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61263872->61267967] (req 61263872->61267967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.508088:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4331 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61263872->61267967] (req 61263872->61267967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.508200:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554679296:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10416us (10705us total) trans 4294982266 rc 0/0 00000100:00100000:0.0:1789760769.508211:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14968 00000100:00100000:0.0:1789760769.508220:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712722438054 time 712743385813 state 0x2 ref 0 00000100:00100000:3.0:1789760769.514617:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760769.515076:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554679424 00000100:00100000:2.0:1789760769.515087:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554679424 00000100:00100000:2.0:1789760769.515142:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712722438054 time 712743385813 state 0x0 ref 1 00000100:00100000:2.0:1789760769.515169:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712743646229 time 712743385813 state 0x1 ref 1 00000100:00100000:2.0:1789760769.515245:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712743646229 time 712760862667 state 0x0 ref 1 00000100:00100000:2.0:1789760769.515259:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e56a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554679424:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760769.515307:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61267968, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d388 00010000:00010000:2.0:1789760769.515315:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4338 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.515337:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760769.515341:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.515347:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4338 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61267968->61272063] (req 61267968->61272063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.515361:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4338 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61267968->61272063] (req 61267968->61272063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.520142:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982267 is committed 00002000:00010000:2.0:1789760769.520294:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61267968 -> 61272064 00002000:00010000:2.0:1789760769.520300:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119688 -> 119696 00010000:00010000:2.0:1789760769.520313:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4338 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61267968->61272063] (req 61267968->61272063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.520326:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4338 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61267968->61272063] (req 61267968->61272063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.520335:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4338 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61267968->61272063] (req 61267968->61272063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.520342:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760769.520347:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4338 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61267968->61272063] (req 61267968->61272063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.520356:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4338 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61267968->61272063] (req 61267968->61272063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.521226:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4338 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61267968->61272063] (req 61267968->61272063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.521241:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4338 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61267968->61272063] (req 61267968->61272063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.521353:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e56a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554679424:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6092us (6739us total) trans 4294982267 rc 0/0 00000100:00100000:2.0:1789760769.521365:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14969 00000100:00100000:2.0:1789760769.521374:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712743646229 time 712760862667 state 0x2 ref 0 00000100:00100000:3.0:1789760769.527533:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760769.527631:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554679552 00000100:00100000:2.0:1789760769.527640:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554679552 00000100:00100000:2.0:1789760769.527696:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712743646229 time 712760862667 state 0x0 ref 1 00000100:00100000:2.0:1789760769.527711:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712761123083 time 712760862667 state 0x1 ref 1 00000100:00100000:2.0:1789760769.527744:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712761123083 time 712773362532 state 0x0 ref 1 00000100:00100000:2.0:1789760769.527756:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e57800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554679552:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760769.527811:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61272064, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d390 00010000:00010000:2.0:1789760769.527820:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c433f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.527836:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760769.527841:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.527847:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c433f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61272064->61276159] (req 61272064->61276159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.527859:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c433f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61272064->61276159] (req 61272064->61276159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760769.533060:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982268 is committed 00002000:00010000:0.0:1789760769.533283:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61272064 -> 61276160 00002000:00010000:0.0:1789760769.533288:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119696 -> 119704 00010000:00010000:0.0:1789760769.533301:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c433f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61272064->61276159] (req 61272064->61276159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.533313:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c433f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61272064->61276159] (req 61272064->61276159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.533327:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c433f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61272064->61276159] (req 61272064->61276159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.533334:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760769.533338:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c433f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61272064->61276159] (req 61272064->61276159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.533347:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c433f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61272064->61276159] (req 61272064->61276159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.533367:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c433f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61272064->61276159] (req 61272064->61276159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.533374:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c433f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61272064->61276159] (req 61272064->61276159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.553520:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760769.561368:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982269 is committed 00000100:00100000:3.0:1789760769.565404:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760769.565504:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554679808 00000100:00100000:0.0:1789760769.565515:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554679808 00000100:00100000:0.0:1789760769.565580:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712773622948 time 712799467599 state 0x0 ref 1 00000100:00100000:0.0:1789760769.565595:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712799728015 time 712799467599 state 0x1 ref 1 00000100:00100000:0.0:1789760769.565619:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712799728015 time 712811238334 state 0x0 ref 1 00000100:00100000:0.0:1789760769.565631:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554679808:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760769.565677:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61280256, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d3a0 00010000:00010000:0.0:1789760769.565687:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c434d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.565701:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760769.565705:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.565711:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c434d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61280256->61284351] (req 61280256->61284351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.565723:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c434d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61280256->61284351] (req 61280256->61284351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760769.570251:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982270 is committed 00002000:00010000:0.0:1789760769.570447:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61280256 -> 61284352 00002000:00010000:0.0:1789760769.570452:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119712 -> 119720 00010000:00010000:0.0:1789760769.570465:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c434d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61280256->61284351] (req 61280256->61284351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.570477:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c434d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61280256->61284351] (req 61280256->61284351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.570485:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c434d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61280256->61284351] (req 61280256->61284351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.570491:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760769.570496:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c434d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61280256->61284351] (req 61280256->61284351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.570508:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c434d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61280256->61284351] (req 61280256->61284351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.570528:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c434d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61280256->61284351] (req 61280256->61284351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.570536:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c434d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61280256->61284351] (req 61280256->61284351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.570636:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554679808:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4999us (5257us total) trans 4294982270 rc 0/0 00000100:00100000:0.0:1789760769.570647:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14972 00000100:00100000:0.0:1789760769.570657:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712799728015 time 712811238334 state 0x2 ref 0 00000100:00100000:3.0:1789760769.577507:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760769.577566:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554679936 00000100:00100000:2.0:1789760769.577577:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554679936 00000100:00100000:2.0:1789760769.577631:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712799728015 time 712811238334 state 0x0 ref 1 00000100:00100000:2.0:1789760769.577647:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712811498750 time 712811238334 state 0x1 ref 1 00000100:00100000:2.0:1789760769.577683:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712811498750 time 712823301320 state 0x0 ref 1 00000100:00100000:2.0:1789760769.577696:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e54a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554679936:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760769.577743:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61284352, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d3a8 00010000:00010000:2.0:1789760769.577752:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4354 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.577767:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760769.577771:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.577788:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4354 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61284352->61288447] (req 61284352->61288447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.577808:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4354 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61284352->61288447] (req 61284352->61288447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760769.583197:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61284352 -> 61288448 00002000:00010000:2.0:1789760769.583203:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119720 -> 119728 00010000:00010000:2.0:1789760769.583216:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4354 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61284352->61288447] (req 61284352->61288447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.583228:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4354 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61284352->61288447] (req 61284352->61288447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.583238:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4354 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61284352->61288447] (req 61284352->61288447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.583245:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760769.583250:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4354 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61284352->61288447] (req 61284352->61288447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.583259:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4354 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61284352->61288447] (req 61284352->61288447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.583280:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4354 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61284352->61288447] (req 61284352->61288447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.583289:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4354 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61284352->61288447] (req 61284352->61288447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.583432:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554679936:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5732us (5929us total) trans 4294982271 rc 0/0 00000100:00100000:2.0:1789760769.583443:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14973 00000100:00100000:2.0:1789760769.583453:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712811498750 time 712823301320 state 0x2 ref 0 00000100:00100000:2.0:1789760769.594498:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:2.0:1789760769.607089:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61288448 -> 61292544 00002000:00010000:2.0:1789760769.607100:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119728 -> 119736 00010000:00010000:2.0:1789760769.607114:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c435b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61288448->61292543] (req 61288448->61292543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.607128:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c435b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61288448->61292543] (req 61288448->61292543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.607138:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c435b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61288448->61292543] (req 61288448->61292543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.607145:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760769.607221:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c435b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61288448->61292543] (req 61288448->61292543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.607233:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c435b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61288448->61292543] (req 61288448->61292543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.607253:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c435b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61288448->61292543] (req 61288448->61292543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.607262:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c435b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61288448->61292543] (req 61288448->61292543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.607841:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554680064:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12496us (13347us total) trans 4294982272 rc 0/0 00000100:00100000:2.0:1789760769.607854:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14974 00000100:00100000:2.0:1789760769.607864:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712823561736 time 712840945063 state 0x2 ref 0 00010000:00010000:0.0:1789760769.619522:0:6509:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760769.619785:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760769.631499:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982273 is committed 00000100:00100000:3.0:1789760769.636060:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760769.636169:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554680320 00000100:00100000:0.0:1789760769.636180:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554680320 00000100:00100000:0.0:1789760769.636235:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712841205479 time 712865601905 state 0x0 ref 1 00000100:00100000:0.0:1789760769.636250:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712865862321 time 712865601905 state 0x1 ref 1 00000100:00100000:0.0:1789760769.636286:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712865862321 time 712881904493 state 0x0 ref 1 00000100:00100000:0.0:1789760769.636299:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e56680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554680320:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760769.636343:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61296640, mtime: 0x6aad9500, atime: 0x0, ctime: 0x6aad9500, blocks: 0x1d3c0 00010000:00010000:0.0:1789760769.636351:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4369 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.636367:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760769.636371:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.636414:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4369 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61296640->61300735] (req 61296640->61300735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.636429:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4369 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61296640->61300735] (req 61296640->61300735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760769.641851:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982274 is committed 00002000:00010000:0.0:1789760769.642106:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61296640 -> 61300736 00002000:00010000:0.0:1789760769.642111:0:15101:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760768 -> 1789760769 00002000:00010000:0.0:1789760769.642114:0:15101:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760768 -> 1789760769 00002000:00010000:0.0:1789760769.642116:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119744 -> 119752 00010000:00010000:0.0:1789760769.642128:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4369 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61296640->61300735] (req 61296640->61300735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.642141:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4369 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61296640->61300735] (req 61296640->61300735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.642234:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4369 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61296640->61300735] (req 61296640->61300735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.642243:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760769.642248:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4369 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61296640->61300735] (req 61296640->61300735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.642322:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4369 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61296640->61300735] (req 61296640->61300735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.642342:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4369 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61296640->61300735] (req 61296640->61300735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.642350:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4369 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61296640->61300735] (req 61296640->61300735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.642809:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e56680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554680320:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6506us (6751us total) trans 4294982274 rc 0/0 00000100:00100000:0.0:1789760769.642821:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14976 00000100:00100000:0.0:1789760769.642831:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712865862321 time 712881904493 state 0x2 ref 0 00000100:00100000:2.0:1789760769.650302:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760769.658077:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982275 is committed 00000100:00100000:3.0:1789760769.662672:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760769.662826:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554680576 00000100:00100000:2.0:1789760769.662836:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554680576 00000100:00100000:2.0:1789760769.662889:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712882164909 time 712896274326 state 0x0 ref 1 00000100:00100000:2.0:1789760769.662904:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712896534742 time 712896274326 state 0x1 ref 1 00000100:00100000:2.0:1789760769.662939:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712896534742 time 712908557646 state 0x0 ref 1 00000100:00100000:2.0:1789760769.662952:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554680576:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760769.662998:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61304832, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d3d0 00010000:00010000:2.0:1789760769.663008:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4377 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.663029:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:2.0:1789760769.663033:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760769.663040:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4377 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61304832->61308927] (req 61304832->61308927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.663054:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4377 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61304832->61308927] (req 61304832->61308927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760769.673803:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61304832 -> 61308928 00002000:00010000:3.0:1789760769.673811:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119760 -> 119768 00010000:00010000:3.0:1789760769.673828:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4377 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61304832->61308927] (req 61304832->61308927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.673841:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4377 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61304832->61308927] (req 61304832->61308927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.673851:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4377 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61304832->61308927] (req 61304832->61308927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.673859:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760769.673864:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4377 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61304832->61308927] (req 61304832->61308927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.673874:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4377 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61304832->61308927] (req 61304832->61308927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.673893:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4377 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61304832->61308927] (req 61304832->61308927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.673902:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4377 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61304832->61308927] (req 61304832->61308927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.673999:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554680576:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11044us (11333us total) trans 4294982276 rc 0/0 00000100:00100000:3.0:1789760769.674011:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14978 00000100:00100000:3.0:1789760769.674098:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712896534742 time 712908557646 state 0x2 ref 0 00000100:00100000:3.0:1789760769.678205:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:0.0:1789760769.685970:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61308928 -> 61313024 00002000:00010000:0.0:1789760769.685976:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119768 -> 119776 00010000:00010000:0.0:1789760769.685989:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c437e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61308928->61313023] (req 61308928->61313023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.686003:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c437e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61308928->61313023] (req 61308928->61313023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.686012:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c437e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61308928->61313023] (req 61308928->61313023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.686025:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760769.686029:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c437e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61308928->61313023] (req 61308928->61313023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.686038:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c437e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61308928->61313023] (req 61308928->61313023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.686059:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c437e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61308928->61313023] (req 61308928->61313023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.686067:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c437e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61308928->61313023] (req 61308928->61313023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.686174:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554680704:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7078us (7976us total) trans 4294982277 rc 0/0 00000100:00100000:0.0:1789760769.686184:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14979 00000100:00100000:0.0:1789760769.686194:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712908818062 time 712924700058 state 0x2 ref 0 00000100:00100000:2.0:1789760769.693937:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760769.694029:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554680832 00000100:00100000:0.0:1789760769.694040:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554680832 00000100:00100000:0.0:1789760769.694092:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712908818062 time 712924700058 state 0x0 ref 1 00000100:00100000:0.0:1789760769.694106:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712924960474 time 712924700058 state 0x1 ref 1 00000100:00100000:0.0:1789760769.694201:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712924960474 time 712939819575 state 0x0 ref 1 00000100:00100000:0.0:1789760769.694217:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554680832:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760769.695542:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61313024, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d3e0 00010000:00010000:0.0:1789760769.695556:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4385 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.695574:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:0.0:1789760769.695578:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.695585:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4385 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61313024->61317119] (req 61313024->61317119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.695599:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4385 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61313024->61317119] (req 61313024->61317119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760769.705065:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61313024 -> 61317120 00002000:00010000:0.0:1789760769.705072:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119776 -> 119784 00010000:00010000:0.0:1789760769.705085:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4385 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61313024->61317119] (req 61313024->61317119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.705096:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4385 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61313024->61317119] (req 61313024->61317119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.705106:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4385 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61313024->61317119] (req 61313024->61317119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.705113:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760769.705118:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4385 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61313024->61317119] (req 61313024->61317119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.705127:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4385 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61313024->61317119] (req 61313024->61317119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.705145:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4385 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61313024->61317119] (req 61313024->61317119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.705218:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4385 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61313024->61317119] (req 61313024->61317119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.705674:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554680832:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11453us (11740us total) trans 4294982278 rc 0/0 00000100:00100000:0.0:1789760769.705696:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14980 00000100:00100000:0.0:1789760769.705705:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712924960474 time 712939819575 state 0x2 ref 0 00000100:00100000:3.0:1789760769.712866:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760769.712998:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554680960 00000100:00100000:2.0:1789760769.713010:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554680960 00000100:00100000:3.0:1789760769.713137:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712924960474 time 712939819575 state 0x0 ref 1 00000100:00100000:3.0:1789760769.713212:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712940079991 time 712939819575 state 0x1 ref 1 00000100:00100000:3.0:1789760769.713245:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712940079991 time 712958863807 state 0x0 ref 1 00000100:00100000:3.0:1789760769.713259:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e57100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554680960:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760769.713305:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61317120, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d3e8 00010000:00010000:3.0:1789760769.713314:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c438c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.713331:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=712s v=(0 0) 00010000:00010000:3.0:1789760769.713335:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.713342:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c438c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61317120->61321215] (req 61317120->61321215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.713357:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c438c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61317120->61321215] (req 61317120->61321215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760769.724700:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61317120 -> 61321216 00002000:00010000:3.0:1789760769.724710:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119784 -> 119792 00010000:00010000:3.0:1789760769.724723:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c438c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61317120->61321215] (req 61317120->61321215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.724737:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c438c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61317120->61321215] (req 61317120->61321215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.724746:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c438c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61317120->61321215] (req 61317120->61321215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.724755:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760769.724760:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c438c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61317120->61321215] (req 61317120->61321215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.724770:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c438c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61317120->61321215] (req 61317120->61321215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.724932:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c438c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61317120->61321215] (req 61317120->61321215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.724942:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c438c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61317120->61321215] (req 61317120->61321215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.725099:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e57100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554680960:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11837us (12235us total) trans 4294982279 rc 0/0 00000100:00100000:3.0:1789760769.725112:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14981 00000100:00100000:3.0:1789760769.725134:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712940079991 time 712958863807 state 0x2 ref 0 00000100:00100000:3.0:1789760769.733227:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760769.750283:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982280 is committed 00002000:00010000:0.0:1789760769.750523:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61321216 -> 61325312 00002000:00010000:0.0:1789760769.750527:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119792 -> 119800 00010000:00010000:0.0:1789760769.750540:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4393 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61321216->61325311] (req 61321216->61325311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.750552:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4393 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61321216->61325311] (req 61321216->61325311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.750560:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4393 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61321216->61325311] (req 61321216->61325311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.750572:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760769.750577:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4393 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61321216->61325311] (req 61321216->61325311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.750587:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4393 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61321216->61325311] (req 61321216->61325311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.750608:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4393 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61321216->61325311] (req 61321216->61325311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.750615:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4393 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61321216->61325311] (req 61321216->61325311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.750727:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554681088:12345-192.168.203.22@tcp:4:dd.0 Request processed in 16481us (17503us total) trans 4294982280 rc 0/0 00000100:00100000:0.0:1789760769.750745:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14982 00000100:00100000:0.0:1789760769.750756:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712959124223 time 712979840968 state 0x2 ref 0 00000100:00100000:2.0:1789760769.757998:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760769.758095:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554681216 00000100:00100000:0.0:1789760769.758106:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554681216 00000100:00100000:0.0:1789760769.758219:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712959124223 time 712979840968 state 0x0 ref 1 00000100:00100000:0.0:1789760769.758236:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712980101384 time 712979840968 state 0x1 ref 1 00000100:00100000:0.0:1789760769.758303:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712980101384 time 713003920223 state 0x0 ref 1 00000100:00100000:0.0:1789760769.758318:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554681216:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760769.758364:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61325312, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d3f8 00010000:00010000:0.0:1789760769.758372:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c439a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.758433:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760769.758437:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.758443:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c439a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61325312->61329407] (req 61325312->61329407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.758459:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c439a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61325312->61329407] (req 61325312->61329407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760769.765856:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61325312 -> 61329408 00002000:00010000:0.0:1789760769.765861:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119800 -> 119808 00010000:00010000:0.0:1789760769.765875:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c439a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61325312->61329407] (req 61325312->61329407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.765888:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c439a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61325312->61329407] (req 61325312->61329407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.765897:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c439a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61325312->61329407] (req 61325312->61329407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.765904:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760769.765909:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c439a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61325312->61329407] (req 61325312->61329407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.765918:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c439a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61325312->61329407] (req 61325312->61329407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.765937:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c439a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61325312->61329407] (req 61325312->61329407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.765945:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c439a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61325312->61329407] (req 61325312->61329407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.766071:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554681216:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7750us (8078us total) trans 4294982281 rc 0/0 00000100:00100000:0.0:1789760769.766081:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14983 00000100:00100000:0.0:1789760769.766090:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712980101384 time 713003920223 state 0x2 ref 0 00000100:00100000:3.0:1789760769.770326:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760769.771007:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554681344 00000100:00100000:0.0:1789760769.771018:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554681344 00000100:00100000:0.0:1789760769.771077:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 712980101384 time 713003920223 state 0x0 ref 1 00000100:00100000:0.0:1789760769.771093:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713004180639 time 713003920223 state 0x1 ref 1 00000100:00100000:0.0:1789760769.771175:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713004180639 time 713016792606 state 0x0 ref 1 00000100:00100000:0.0:1789760769.771190:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554681344:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760769.771238:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61329408, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d400 00010000:00010000:0.0:1789760769.771249:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c43a1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.771265:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760769.771269:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.771275:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c43a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61329408->61333503] (req 61329408->61333503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.771289:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c43a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61329408->61333503] (req 61329408->61333503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760769.779362:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982282 is committed 00000100:00100000:0.0:1789760769.787050:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760769.787136:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554681472 00000100:00100000:3.0:1789760769.787145:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554681472 00000100:00100000:3.0:1789760769.787251:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713004180639 time 713016792606 state 0x0 ref 1 00000100:00100000:3.0:1789760769.787266:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713017053022 time 713016792606 state 0x1 ref 1 00000100:00100000:3.0:1789760769.787330:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713017053022 time 713032948546 state 0x0 ref 1 00000100:00100000:3.0:1789760769.787344:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554681472:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760769.787412:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61333504, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d408 00010000:00010000:3.0:1789760769.787422:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c43a8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.787437:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760769.787442:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.787448:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c43a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61333504->61337599] (req 61333504->61337599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.787462:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c43a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61333504->61337599] (req 61333504->61337599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.793646:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982283 is committed 00000100:00100000:2.0:1789760769.799499:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760769.806048:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982284 is committed 00000100:00100000:3.0:1789760769.810666:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760769.812085:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554681728 00000100:00100000:0.0:1789760769.812098:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554681728 00000100:00100000:0.0:1789760769.812167:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713033208962 time 713045332518 state 0x0 ref 1 00000100:00100000:0.0:1789760769.812183:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713045592934 time 713045332518 state 0x1 ref 1 00000100:00100000:0.0:1789760769.812215:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713045592934 time 713057833763 state 0x0 ref 1 00000100:00100000:0.0:1789760769.812227:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554681728:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760769.812443:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61341696, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d418 00010000:00010000:0.0:1789760769.812453:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c43b6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.812468:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760769.812473:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.812479:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c43b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61341696->61345791] (req 61341696->61345791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.812494:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c43b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61341696->61345791] (req 61341696->61345791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.818042:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982285 is committed 00002000:00010000:0.0:1789760769.818160:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61341696 -> 61345792 00002000:00010000:0.0:1789760769.818166:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119832 -> 119840 00010000:00010000:0.0:1789760769.818178:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c43b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61341696->61345791] (req 61341696->61345791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.818190:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c43b6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61341696->61345791] (req 61341696->61345791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.818199:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c43b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61341696->61345791] (req 61341696->61345791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.818207:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760769.818211:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c43b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61341696->61345791] (req 61341696->61345791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.818220:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c43b6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61341696->61345791] (req 61341696->61345791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.818239:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c43b6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61341696->61345791] (req 61341696->61345791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.818252:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c43b6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61341696->61345791] (req 61341696->61345791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.818504:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554681728:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6275us (7842us total) trans 4294982285 rc 0/0 00000100:00100000:0.0:1789760769.818516:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14987 00000100:00100000:0.0:1789760769.818525:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713045592934 time 713057833763 state 0x2 ref 0 00000100:00100000:0.0:1789760769.841686:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554681856 00000100:00100000:0.0:1789760769.841696:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554681856 00000100:00100000:0.0:1789760769.841861:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713045592934 time 713057833763 state 0x0 ref 1 00000100:00100000:0.0:1789760769.841877:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713058094179 time 713057833763 state 0x1 ref 1 00000100:00100000:0.0:1789760769.841961:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713058094179 time 713087578678 state 0x0 ref 1 00000100:00100000:0.0:1789760769.841975:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554681856:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760769.842020:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61345792, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d420 00010000:00010000:0.0:1789760769.842029:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c43bd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.842044:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760769.842048:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.842053:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c43bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61345792->61349887] (req 61345792->61349887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.842074:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c43bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61345792->61349887] (req 61345792->61349887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760769.846935:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982286 is committed 00002000:00010000:0.0:1789760769.847091:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61345792 -> 61349888 00002000:00010000:0.0:1789760769.847096:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119840 -> 119848 00010000:00010000:0.0:1789760769.847108:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c43bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61345792->61349887] (req 61345792->61349887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.847121:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c43bd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61345792->61349887] (req 61345792->61349887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.847130:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c43bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61345792->61349887] (req 61345792->61349887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.847137:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760769.847142:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c43bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61345792->61349887] (req 61345792->61349887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.847209:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c43bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61345792->61349887] (req 61345792->61349887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.847228:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c43bd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61345792->61349887] (req 61345792->61349887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.847235:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c43bd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61345792->61349887] (req 61345792->61349887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760769.847835:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554681856:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5857us (6411us total) trans 4294982286 rc 0/0 00000100:00100000:0.0:1789760769.847846:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14988 00000100:00100000:0.0:1789760769.847854:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713058094179 time 713087578678 state 0x2 ref 0 00000100:00100000:2.0:1789760769.858924:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760769.866214:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982287 is committed 00000100:00100000:3.0:1789760769.872955:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:0.0:1789760769.878188:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982288 is committed 00000100:00100000:3.0:1789760769.882977:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760769.883107:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554682240 00000100:00100000:0.0:1789760769.883116:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554682240 00000100:00100000:0.0:1789760769.883227:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713105366219 time 713119035067 state 0x0 ref 1 00000100:00100000:0.0:1789760769.883243:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713119295483 time 713119035067 state 0x1 ref 1 00000100:00100000:0.0:1789760769.883270:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713119295483 time 713128889248 state 0x0 ref 1 00000100:00100000:0.0:1789760769.883283:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554682240:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760769.883341:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61358080, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d438 00010000:00010000:0.0:1789760769.883350:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c43d2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.883366:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760769.883370:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.883406:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c43d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61358080->61362175] (req 61358080->61362175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.883420:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c43d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61358080->61362175] (req 61358080->61362175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760769.890438:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982289 is committed 00002000:00010000:3.0:1789760769.890683:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61358080 -> 61362176 00002000:00010000:3.0:1789760769.890689:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119864 -> 119872 00010000:00010000:3.0:1789760769.890703:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c43d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61358080->61362175] (req 61358080->61362175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.890716:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c43d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61358080->61362175] (req 61358080->61362175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.890726:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c43d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61358080->61362175] (req 61358080->61362175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.890734:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760769.890855:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c43d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61358080->61362175] (req 61358080->61362175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.890866:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c43d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61358080->61362175] (req 61358080->61362175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.890886:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c43d2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61358080->61362175] (req 61358080->61362175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.890894:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c43d2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61358080->61362175] (req 61358080->61362175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.890996:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554682240:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7711us (8032us total) trans 4294982289 rc 0/0 00000100:00100000:3.0:1789760769.891007:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14991 00000100:00100000:3.0:1789760769.891016:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713119295483 time 713128889248 state 0x2 ref 0 00000100:00100000:2.0:1789760769.895365:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760769.895556:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554682368 00000100:00100000:0.0:1789760769.895566:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554682368 00000100:00100000:0.0:1789760769.895786:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713119295483 time 713128889248 state 0x0 ref 1 00000100:00100000:0.0:1789760769.895802:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713129149664 time 713128889248 state 0x1 ref 1 00000100:00100000:0.0:1789760769.895834:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713129149664 time 713141452146 state 0x0 ref 1 00000100:00100000:0.0:1789760769.895846:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554682368:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760769.895897:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61362176, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d440 00010000:00010000:0.0:1789760769.895906:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c43d9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.895920:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760769.895923:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760769.895929:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c43d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61362176->61366271] (req 61362176->61366271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760769.895941:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c43d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61362176->61366271] (req 61362176->61366271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.903022:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982290 is committed 00002000:00010000:2.0:1789760769.908298:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61362176 -> 61366272 00002000:00010000:2.0:1789760769.908307:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119872 -> 119880 00010000:00010000:2.0:1789760769.908321:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c43d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61362176->61366271] (req 61362176->61366271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.908333:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c43d9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61362176->61366271] (req 61362176->61366271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.908341:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c43d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61362176->61366271] (req 61362176->61366271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.908353:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760769.908358:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c43d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61362176->61366271] (req 61362176->61366271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.908366:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c43d9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61362176->61366271] (req 61362176->61366271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.908494:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c43d9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61362176->61366271] (req 61362176->61366271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.908503:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c43d9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61362176->61366271] (req 61362176->61366271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.908601:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554682368:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12752us (13241us total) trans 4294982290 rc 0/0 00000100:00100000:2.0:1789760769.908612:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14992 00000100:00100000:2.0:1789760769.908621:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713129149664 time 713141452146 state 0x2 ref 0 00000100:00100000:2.0:1789760769.915530:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:0.0:1789760769.923107:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982291 is committed 00000100:00100000:3.0:1789760769.930569:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:0.0:1789760769.938359:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982292 is committed 00000100:00100000:2.0:1789760769.942538:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760769.942642:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554682752 00000100:00100000:3.0:1789760769.942653:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554682752 00000100:00100000:3.0:1789760769.942709:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713161618607 time 713176373437 state 0x0 ref 1 00000100:00100000:3.0:1789760769.942724:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713176633853 time 713176373437 state 0x1 ref 1 00000100:00100000:3.0:1789760769.942796:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713176633853 time 713188414512 state 0x0 ref 1 00000100:00100000:3.0:1789760769.942809:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554682752:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760769.942857:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61374464, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d458 00010000:00010000:3.0:1789760769.942866:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c43ee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.942882:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760769.942887:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.942893:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c43ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61374464->61378559] (req 61374464->61378559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.942914:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c43ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61374464->61378559] (req 61374464->61378559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760769.950269:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982293 is committed 00002000:00010000:3.0:1789760769.950425:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61374464 -> 61378560 00002000:00010000:3.0:1789760769.950430:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119896 -> 119904 00010000:00010000:3.0:1789760769.950445:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c43ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61374464->61378559] (req 61374464->61378559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.950456:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c43ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61374464->61378559] (req 61374464->61378559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.950464:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c43ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61374464->61378559] (req 61374464->61378559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.950471:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760769.950476:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c43ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61374464->61378559] (req 61374464->61378559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.950485:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c43ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61374464->61378559] (req 61374464->61378559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.950505:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c43ee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61374464->61378559] (req 61374464->61378559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.950512:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c43ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61374464->61378559] (req 61374464->61378559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760769.950657:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554682752:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7844us (8126us total) trans 4294982293 rc 0/0 00000100:00100000:3.0:1789760769.950668:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14995 00000100:00100000:3.0:1789760769.950678:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713176633853 time 713188414512 state 0x2 ref 0 00000100:00100000:2.0:1789760769.964714:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:0.0:1789760769.974325:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982294 is committed 00002000:00010000:2.0:1789760769.975190:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61378560 -> 61382656 00002000:00010000:2.0:1789760769.975197:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119904 -> 119912 00010000:00010000:2.0:1789760769.975211:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c43f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61378560->61382655] (req 61378560->61382655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.975227:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c43f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61378560->61382655] (req 61378560->61382655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.975236:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c43f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61378560->61382655] (req 61378560->61382655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.975243:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760769.975248:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c43f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61378560->61382655] (req 61378560->61382655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.975256:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c43f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61378560->61382655] (req 61378560->61382655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.975273:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c43f5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61378560->61382655] (req 61378560->61382655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760769.975280:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c43f5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61378560->61382655] (req 61378560->61382655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760769.975370:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554682880:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6650us (10663us total) trans 4294982294 rc 0/0 00000100:00100000:2.0:1789760769.978100:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14996 00000100:00100000:2.0:1789760769.978115:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713188674928 time 713214323362 state 0x2 ref 0 00000100:00100000:3.0:1789760769.987550:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554683008 00000100:00100000:3.0:1789760769.987565:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554683008 00000100:00100000:3.0:1789760769.987629:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713188674928 time 713214323362 state 0x0 ref 1 00000100:00100000:3.0:1789760769.987651:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713214583778 time 713214323362 state 0x1 ref 1 00000100:00100000:3.0:1789760769.987690:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713214583778 time 713233308719 state 0x0 ref 1 00000100:00100000:3.0:1789760769.987704:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554683008:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760769.987798:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61382656, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d468 00010000:00010000:3.0:1789760769.987808:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c43fc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.987825:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760769.987830:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760769.987838:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c43fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61382656->61386751] (req 61382656->61386751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760769.987852:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c43fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61382656->61386751] (req 61382656->61386751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760769.993319:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982295 is committed 00000100:00100000:0.0:1789760769.998333:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760770.008592:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982296 is committed 00000100:00100000:2.0:1789760770.017086:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760770.022899:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554683264 00000100:00100000:0.0:1789760770.022914:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554683264 00000100:00100000:0.0:1789760770.022978:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713233569135 time 713244163914 state 0x0 ref 1 00000100:00100000:0.0:1789760770.022993:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713244424330 time 713244163914 state 0x1 ref 1 00000100:00100000:0.0:1789760770.023113:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713244424330 time 713268729993 state 0x0 ref 1 00000100:00100000:0.0:1789760770.023127:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04562680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554683264:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760770.023184:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61390848, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d478 00010000:00010000:0.0:1789760770.023194:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c440a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.023208:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760770.023214:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.023219:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c440a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61390848->61394943] (req 61390848->61394943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.023232:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c440a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61390848->61394943] (req 61390848->61394943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.030880:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982297 is committed 00002000:00010000:0.0:1789760770.030998:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61390848 -> 61394944 00002000:00010000:0.0:1789760770.031003:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119928 -> 119936 00010000:00010000:0.0:1789760770.031030:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c440a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61390848->61394943] (req 61390848->61394943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.031047:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c440a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61390848->61394943] (req 61390848->61394943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.031056:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c440a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61390848->61394943] (req 61390848->61394943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.031063:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760770.031068:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c440a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61390848->61394943] (req 61390848->61394943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.031077:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c440a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61390848->61394943] (req 61390848->61394943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.031096:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c440a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61390848->61394943] (req 61390848->61394943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.031103:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c440a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61390848->61394943] (req 61390848->61394943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.031209:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04562680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554683264:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8080us (14127us total) trans 4294982297 rc 0/0 00000100:00100000:0.0:1789760770.031220:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 14999 00000100:00100000:0.0:1789760770.031228:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713244424330 time 713268729993 state 0x2 ref 0 00000100:00100000:3.0:1789760770.038548:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760770.038693:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554683392 00000100:00100000:2.0:1789760770.038704:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554683392 00000100:00100000:2.0:1789760770.038762:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713244424330 time 713268729993 state 0x0 ref 1 00000100:00100000:2.0:1789760770.038789:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713268990409 time 713268729993 state 0x1 ref 1 00000100:00100000:2.0:1789760770.038822:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713268990409 time 713284440806 state 0x0 ref 1 00000100:00100000:2.0:1789760770.038833:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe1e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554683392:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760770.038876:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61394944, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d480 00010000:00010000:2.0:1789760770.038886:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4411 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.038901:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760770.038906:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.038912:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4411 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61394944->61399039] (req 61394944->61399039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.038926:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4411 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61394944->61399039] (req 61394944->61399039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760770.053613:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982298 is committed 00002000:00010000:0.0:1789760770.053766:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61394944 -> 61399040 00002000:00010000:0.0:1789760770.053783:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119936 -> 119944 00010000:00010000:0.0:1789760770.053797:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4411 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61394944->61399039] (req 61394944->61399039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.053809:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4411 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61394944->61399039] (req 61394944->61399039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.053817:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4411 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61394944->61399039] (req 61394944->61399039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.053825:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760770.053830:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4411 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61394944->61399039] (req 61394944->61399039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.053840:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4411 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61394944->61399039] (req 61394944->61399039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.053859:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4411 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61394944->61399039] (req 61394944->61399039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.053865:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4411 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61394944->61399039] (req 61394944->61399039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.053968:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe1e850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554683392:12345-192.168.203.22@tcp:4:dd.0 Request processed in 15132us (15435us total) trans 4294982298 rc 0/0 00000100:00100000:0.0:1789760770.053980:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15000 00000100:00100000:0.0:1789760770.053990:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713268990409 time 713284440806 state 0x2 ref 0 00000100:00100000:2.0:1789760770.068520:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760770.079314:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982299 is committed 00000100:00100000:2.0:1789760770.087741:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760770.103195:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982300 is committed 00000100:00100000:0.0:1789760770.120705:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554683776 00000100:00100000:0.0:1789760770.120714:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554683776 00000100:00100000:0.0:1789760770.120765:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713314649473 time 713337132993 state 0x0 ref 1 00000100:00100000:0.0:1789760770.120793:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713337393409 time 713337132993 state 0x1 ref 1 00000100:00100000:0.0:1789760770.120854:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713337393409 time 713366472865 state 0x0 ref 1 00000100:00100000:0.0:1789760770.120867:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554683776:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760770.120926:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61407232, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d498 00010000:00010000:0.0:1789760770.120936:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4426 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.120953:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760770.120957:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.120963:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4426 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61407232->61411327] (req 61407232->61411327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.120977:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4426 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61407232->61411327] (req 61407232->61411327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.128025:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982301 is committed 00002000:00010000:0.0:1789760770.128154:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61407232 -> 61411328 00002000:00010000:0.0:1789760770.128160:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119960 -> 119968 00010000:00010000:0.0:1789760770.128178:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4426 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61407232->61411327] (req 61407232->61411327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.128196:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4426 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61407232->61411327] (req 61407232->61411327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.128204:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4426 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61407232->61411327] (req 61407232->61411327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.128276:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760770.128282:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4426 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61407232->61411327] (req 61407232->61411327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.128291:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4426 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61407232->61411327] (req 61407232->61411327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.128311:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4426 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61407232->61411327] (req 61407232->61411327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.128319:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4426 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61407232->61411327] (req 61407232->61411327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.128581:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554683776:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7710us (7974us total) trans 4294982301 rc 0/0 00000100:00100000:0.0:1789760770.128591:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15003 00000100:00100000:0.0:1789760770.128601:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713337393409 time 713366472865 state 0x2 ref 0 00000100:00100000:2.0:1789760770.133855:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760770.134422:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554683904 00000100:00100000:0.0:1789760770.134433:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554683904 00000100:00100000:0.0:1789760770.134493:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713337393409 time 713366472865 state 0x0 ref 1 00000100:00100000:0.0:1789760770.134508:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713366733281 time 713366472865 state 0x1 ref 1 00000100:00100000:0.0:1789760770.134530:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713366733281 time 713380149417 state 0x0 ref 1 00000100:00100000:0.0:1789760770.134542:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04560a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554683904:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760770.134603:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61411328, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d4a0 00010000:00010000:0.0:1789760770.134611:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c442d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.134628:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760770.134633:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.134639:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c442d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61411328->61415423] (req 61411328->61415423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.134653:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c442d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61411328->61415423] (req 61411328->61415423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.143079:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982302 is committed 00002000:00010000:0.0:1789760770.143217:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61411328 -> 61415424 00002000:00010000:0.0:1789760770.143223:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119968 -> 119976 00010000:00010000:0.0:1789760770.143236:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c442d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61411328->61415423] (req 61411328->61415423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.143248:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c442d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61411328->61415423] (req 61411328->61415423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.143261:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c442d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61411328->61415423] (req 61411328->61415423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.143268:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760770.143272:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c442d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61411328->61415423] (req 61411328->61415423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.143280:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c442d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61411328->61415423] (req 61411328->61415423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.143298:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c442d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61411328->61415423] (req 61411328->61415423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.143305:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c442d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61411328->61415423] (req 61411328->61415423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.143563:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04560a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554683904:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9017us (9712us total) trans 4294982302 rc 0/0 00000100:00100000:0.0:1789760770.143585:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15004 00000100:00100000:0.0:1789760770.143596:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713366733281 time 713380149417 state 0x2 ref 0 00000100:00100000:2.0:1789760770.159055:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760770.168958:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982303 is committed 00002000:00010000:0.0:1789760770.169073:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61415424 -> 61419520 00002000:00010000:0.0:1789760770.169078:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119976 -> 119984 00010000:00010000:0.0:1789760770.169090:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c4434 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61415424->61419519] (req 61415424->61419519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.169108:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c4434 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61415424->61419519] (req 61415424->61419519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.169117:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c4434 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61415424->61419519] (req 61415424->61419519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.169124:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760770.169128:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c4434 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61415424->61419519] (req 61415424->61419519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.169138:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c4434 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61415424->61419519] (req 61415424->61419519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.169219:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c4434 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61415424->61419519] (req 61415424->61419519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.169229:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c4434 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61415424->61419519] (req 61415424->61419519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.169321:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04563800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554684032:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9646us (10271us total) trans 4294982303 rc 0/0 00000100:00100000:0.0:1789760770.169331:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15005 00000100:00100000:0.0:1789760770.169341:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713380409833 time 713405279700 state 0x2 ref 0 00000100:00100000:0.0:1789760770.177990:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554684160 00000100:00100000:0.0:1789760770.178003:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554684160 00000100:00100000:0.0:1789760770.178064:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713380409833 time 713405279700 state 0x0 ref 1 00000100:00100000:0.0:1789760770.178082:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713405540116 time 713405279700 state 0x1 ref 1 00000100:00100000:0.0:1789760770.178119:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713405540116 time 713423736684 state 0x0 ref 1 00000100:00100000:0.0:1789760770.178132:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554684160:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760770.178180:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61419520, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d4b0 00010000:00010000:0.0:1789760770.178190:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c443b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.178205:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760770.178210:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.178217:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c443b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61419520->61423615] (req 61419520->61423615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.178230:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c443b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61419520->61423615] (req 61419520->61423615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.187142:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982304 is committed 00000100:00100000:2.0:1789760770.201933:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760770.202052:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554684288 00000100:00100000:0.0:1789760770.202062:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554684288 00000100:00100000:0.0:1789760770.202227:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713405540116 time 713423736684 state 0x0 ref 1 00000100:00100000:0.0:1789760770.202242:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713423997100 time 713423736684 state 0x1 ref 1 00000100:00100000:0.0:1789760770.202297:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713423997100 time 713447916146 state 0x0 ref 1 00000100:00100000:0.0:1789760770.202307:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04563480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554684288:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760770.202348:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61423616, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d4b8 00010000:00010000:0.0:1789760770.202356:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4442 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.202370:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760770.202373:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.202399:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4442 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61423616->61427711] (req 61423616->61427711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.202410:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4442 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61423616->61427711] (req 61423616->61427711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.208412:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982305 is committed 00002000:00010000:0.0:1789760770.208606:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61423616 -> 61427712 00002000:00010000:0.0:1789760770.208611:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 119992 -> 120000 00010000:00010000:0.0:1789760770.208625:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4442 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61423616->61427711] (req 61423616->61427711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.208638:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4442 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61423616->61427711] (req 61423616->61427711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.208647:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4442 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61423616->61427711] (req 61423616->61427711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.208655:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760770.208660:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4442 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61423616->61427711] (req 61423616->61427711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.208670:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4442 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61423616->61427711] (req 61423616->61427711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.208701:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4442 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61423616->61427711] (req 61423616->61427711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.208785:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4442 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61423616->61427711] (req 61423616->61427711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.208877:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04563480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554684288:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6566us (6952us total) trans 4294982305 rc 0/0 00000100:00100000:0.0:1789760770.208888:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15007 00000100:00100000:0.0:1789760770.208898:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713423997100 time 713447916146 state 0x2 ref 0 00000100:00100000:2.0:1789760770.216523:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760770.216717:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554684416 00000100:00100000:0.0:1789760770.216727:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554684416 00000100:00100000:0.0:1789760770.216797:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713423997100 time 713447916146 state 0x0 ref 1 00000100:00100000:0.0:1789760770.216813:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713448176562 time 713447916146 state 0x1 ref 1 00000100:00100000:0.0:1789760770.216936:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713448176562 time 713462552837 state 0x0 ref 1 00000100:00100000:0.0:1789760770.216951:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554684416:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760770.217064:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61427712, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d4c0 00010000:00010000:0.0:1789760770.217075:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4449 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.217091:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760770.217096:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.217102:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4449 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61427712->61431807] (req 61427712->61431807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.217117:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4449 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61427712->61431807] (req 61427712->61431807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.226871:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982306 is committed 00002000:00010000:2.0:1789760770.229881:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61427712 -> 61431808 00002000:00010000:2.0:1789760770.229891:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120000 -> 120008 00010000:00010000:2.0:1789760770.229914:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4449 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61427712->61431807] (req 61427712->61431807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.229928:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4449 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61427712->61431807] (req 61427712->61431807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.229938:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4449 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61427712->61431807] (req 61427712->61431807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.229945:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760770.229950:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4449 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61427712->61431807] (req 61427712->61431807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.229959:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4449 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61427712->61431807] (req 61427712->61431807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.229978:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4449 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61427712->61431807] (req 61427712->61431807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.229986:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4449 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61427712->61431807] (req 61427712->61431807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.230134:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554684416:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13181us (13617us total) trans 4294982306 rc 0/0 00000100:00100000:2.0:1789760770.230156:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15008 00000100:00100000:2.0:1789760770.230164:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713448176562 time 713462552837 state 0x2 ref 0 00000100:00100000:3.0:1789760770.240444:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554684544 00000100:00100000:3.0:1789760770.240455:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554684544 00000100:00100000:3.0:1789760770.240511:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713448176562 time 713462552837 state 0x0 ref 1 00000100:00100000:3.0:1789760770.240527:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713462813253 time 713462552837 state 0x1 ref 1 00000100:00100000:3.0:1789760770.240604:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713462813253 time 713486222430 state 0x0 ref 1 00000100:00100000:3.0:1789760770.240617:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554684544:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760770.240669:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61431808, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d4c8 00010000:00010000:3.0:1789760770.240678:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4450 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.240706:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760770.240911:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.240918:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4450 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61431808->61435903] (req 61431808->61435903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.240934:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4450 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61431808->61435903] (req 61431808->61435903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.247333:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982307 is committed 00002000:00010000:3.0:1789760770.247460:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61431808 -> 61435904 00002000:00010000:3.0:1789760770.247465:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120008 -> 120016 00010000:00010000:3.0:1789760770.247479:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4450 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61431808->61435903] (req 61431808->61435903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.247491:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4450 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61431808->61435903] (req 61431808->61435903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.247500:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4450 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61431808->61435903] (req 61431808->61435903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.247508:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760770.247513:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4450 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61431808->61435903] (req 61431808->61435903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.247522:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4450 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61431808->61435903] (req 61431808->61435903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.247543:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4450 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61431808->61435903] (req 61431808->61435903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.247550:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4450 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61431808->61435903] (req 61431808->61435903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.247931:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554684544:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7310us (7842us total) trans 4294982307 rc 0/0 00000100:00100000:3.0:1789760770.247944:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15009 00000100:00100000:3.0:1789760770.247954:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713462813253 time 713486222430 state 0x2 ref 0 00000100:00100000:0.0:1789760770.257854:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760770.268343:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982308 is committed 00000100:00100000:3.0:1789760770.279665:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760770.281078:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554684800 00000100:00100000:0.0:1789760770.281090:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554684800 00000100:00100000:0.0:1789760770.281158:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713486482846 time 713503881617 state 0x0 ref 1 00000100:00100000:0.0:1789760770.281175:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713504142033 time 713503881617 state 0x1 ref 1 00000100:00100000:0.0:1789760770.281208:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713504142033 time 713526827316 state 0x0 ref 1 00000100:00100000:0.0:1789760770.281221:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554684800:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760770.281269:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61440000, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d4d8 00010000:00010000:0.0:1789760770.281279:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c445e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.281300:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760770.281304:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.281310:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c445e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61440000->61444095] (req 61440000->61444095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.281328:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c445e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61440000->61444095] (req 61440000->61444095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.291335:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982309 is committed 00002000:00010000:3.0:1789760770.291476:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61440000 -> 61444096 00002000:00010000:3.0:1789760770.291482:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120024 -> 120032 00010000:00010000:3.0:1789760770.291497:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c445e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61440000->61444095] (req 61440000->61444095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.291510:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c445e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61440000->61444095] (req 61440000->61444095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.291520:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c445e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61440000->61444095] (req 61440000->61444095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.291527:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760770.291532:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c445e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61440000->61444095] (req 61440000->61444095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.291542:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c445e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61440000->61444095] (req 61440000->61444095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.291562:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c445e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61440000->61444095] (req 61440000->61444095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.291572:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c445e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61440000->61444095] (req 61440000->61444095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.291680:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554684800:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10447us (12014us total) trans 4294982309 rc 0/0 00000100:00100000:3.0:1789760770.291693:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15011 00000100:00100000:3.0:1789760770.291702:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713504142033 time 713526827316 state 0x2 ref 0 00000100:00100000:2.0:1789760770.299689:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554684928 00000100:00100000:2.0:1789760770.299699:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554684928 00000100:00100000:2.0:1789760770.299752:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713504142033 time 713526827316 state 0x0 ref 1 00000100:00100000:2.0:1789760770.299779:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713527087732 time 713526827316 state 0x1 ref 1 00000100:00100000:2.0:1789760770.299809:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713527087732 time 713545427241 state 0x0 ref 1 00000100:00100000:2.0:1789760770.299820:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554684928:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760770.299865:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61444096, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d4e0 00010000:00010000:2.0:1789760770.299875:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4465 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.299890:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760770.299895:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.299901:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4465 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61444096->61448191] (req 61444096->61448191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.299913:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4465 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61444096->61448191] (req 61444096->61448191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.306517:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982310 is committed 00002000:00010000:2.0:1789760770.306641:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61444096 -> 61448192 00002000:00010000:2.0:1789760770.306646:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120032 -> 120040 00010000:00010000:2.0:1789760770.306658:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4465 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61444096->61448191] (req 61444096->61448191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.306669:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4465 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61444096->61448191] (req 61444096->61448191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.306677:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4465 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61444096->61448191] (req 61444096->61448191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.306692:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760770.306697:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4465 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61444096->61448191] (req 61444096->61448191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.306713:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4465 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61444096->61448191] (req 61444096->61448191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.306731:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4465 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61444096->61448191] (req 61444096->61448191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.306739:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4465 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61444096->61448191] (req 61444096->61448191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.306845:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554684928:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7022us (7265us total) trans 4294982310 rc 0/0 00000100:00100000:2.0:1789760770.306855:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15012 00000100:00100000:2.0:1789760770.306864:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713527087732 time 713545427241 state 0x2 ref 0 00000100:00100000:2.0:1789760770.310752:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760770.311082:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554685056 00000100:00100000:3.0:1789760770.311091:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554685056 00000100:00100000:3.0:1789760770.311158:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713527087732 time 713545427241 state 0x0 ref 1 00000100:00100000:3.0:1789760770.311173:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713545687657 time 713545427241 state 0x1 ref 1 00000100:00100000:3.0:1789760770.311206:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713545687657 time 713556825324 state 0x0 ref 1 00000100:00100000:3.0:1789760770.311217:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04562d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554685056:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760770.311263:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61448192, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d4e8 00010000:00010000:3.0:1789760770.311272:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c446c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.311288:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760770.311292:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.311298:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c446c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61448192->61452287] (req 61448192->61452287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.311317:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c446c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61448192->61452287] (req 61448192->61452287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760770.317039:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982311 is committed 00002000:00010000:3.0:1789760770.317259:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61448192 -> 61452288 00002000:00010000:3.0:1789760770.317264:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120040 -> 120048 00010000:00010000:3.0:1789760770.317278:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c446c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61448192->61452287] (req 61448192->61452287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.317292:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c446c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61448192->61452287] (req 61448192->61452287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.317301:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c446c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61448192->61452287] (req 61448192->61452287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.317308:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760770.317313:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c446c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61448192->61452287] (req 61448192->61452287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.317324:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c446c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61448192->61452287] (req 61448192->61452287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.317488:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c446c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61448192->61452287] (req 61448192->61452287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.317498:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c446c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61448192->61452287] (req 61448192->61452287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.317606:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04562d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554685056:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6383us (6858us total) trans 4294982311 rc 0/0 00000100:00100000:3.0:1789760770.317619:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15013 00000100:00100000:3.0:1789760770.317629:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713545687657 time 713556825324 state 0x2 ref 0 00000100:00100000:2.0:1789760770.323025:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760770.323321:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554685184 00000100:00100000:3.0:1789760770.323335:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554685184 00000100:00100000:3.0:1789760770.323479:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713545687657 time 713556825324 state 0x0 ref 1 00000100:00100000:3.0:1789760770.323495:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713557085740 time 713556825324 state 0x1 ref 1 00000100:00100000:3.0:1789760770.323651:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713557085740 time 713569267787 state 0x0 ref 1 00000100:00100000:3.0:1789760770.323667:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04563b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554685184:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760770.323719:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61452288, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d4f0 00010000:00010000:3.0:1789760770.323729:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4473 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.323746:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760770.323751:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.323757:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4473 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61452288->61456383] (req 61452288->61456383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.323780:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4473 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61452288->61456383] (req 61452288->61456383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760770.328784:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982312 is committed 00002000:00010000:3.0:1789760770.328909:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61452288 -> 61456384 00002000:00010000:3.0:1789760770.328915:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120048 -> 120056 00010000:00010000:3.0:1789760770.328930:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4473 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61452288->61456383] (req 61452288->61456383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.328943:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4473 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61452288->61456383] (req 61452288->61456383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.328953:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4473 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61452288->61456383] (req 61452288->61456383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.328960:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760770.328966:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4473 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61452288->61456383] (req 61452288->61456383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.328976:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4473 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61452288->61456383] (req 61452288->61456383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.328998:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4473 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61452288->61456383] (req 61452288->61456383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.329007:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4473 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61452288->61456383] (req 61452288->61456383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.329141:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04563b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554685184:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5473us (6124us total) trans 4294982312 rc 0/0 00000100:00100000:3.0:1789760770.329161:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15014 00000100:00100000:3.0:1789760770.329170:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713557085740 time 713569267787 state 0x2 ref 0 00000100:00100000:2.0:1789760770.333291:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760770.339295:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982313 is committed 00000100:00100000:2.0:1789760770.344564:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760770.345874:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554685440 00000100:00100000:0.0:1789760770.345887:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554685440 00000100:00100000:0.0:1789760770.345953:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713569528203 time 713579171311 state 0x0 ref 1 00000100:00100000:0.0:1789760770.345975:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713579431727 time 713579171311 state 0x1 ref 1 00000100:00100000:0.0:1789760770.346018:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713579431727 time 713591635509 state 0x0 ref 1 00000100:00100000:0.0:1789760770.346034:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554685440:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760770.346095:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61460480, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d500 00010000:00010000:0.0:1789760770.346104:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4481 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.346134:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760770.346139:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.346145:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4481 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61460480->61464575] (req 61460480->61464575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.346162:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4481 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61460480->61464575] (req 61460480->61464575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760770.352983:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982314 is committed 00002000:00010000:0.0:1789760770.354625:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61460480 -> 61464576 00002000:00010000:0.0:1789760770.354638:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120064 -> 120072 00010000:00010000:0.0:1789760770.354655:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4481 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61460480->61464575] (req 61460480->61464575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.354672:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4481 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61460480->61464575] (req 61460480->61464575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.354682:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4481 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61460480->61464575] (req 61460480->61464575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.354692:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760770.354697:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4481 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61460480->61464575] (req 61460480->61464575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.354708:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4481 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61460480->61464575] (req 61460480->61464575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.354732:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4481 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61460480->61464575] (req 61460480->61464575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.354741:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4481 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61460480->61464575] (req 61460480->61464575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.354912:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554685440:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8877us (10353us total) trans 4294982314 rc 0/0 00000100:00100000:0.0:1789760770.354924:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15016 00000100:00100000:0.0:1789760770.354934:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713579431727 time 713591635509 state 0x2 ref 0 00000100:00100000:2.0:1789760770.362527:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760770.362853:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554685568 00000100:00100000:3.0:1789760770.362864:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554685568 00000100:00100000:3.0:1789760770.362925:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713579431727 time 713591635509 state 0x0 ref 1 00000100:00100000:3.0:1789760770.362945:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713591895925 time 713591635509 state 0x1 ref 1 00000100:00100000:3.0:1789760770.363032:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713591895925 time 713608649433 state 0x0 ref 1 00000100:00100000:3.0:1789760770.363046:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04560e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554685568:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760770.363098:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61464576, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d508 00010000:00010000:3.0:1789760770.363107:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4488 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.363137:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760770.363142:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.363157:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4488 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61464576->61468671] (req 61464576->61468671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.363172:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4488 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61464576->61468671] (req 61464576->61468671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760770.371734:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982315 is committed 00002000:00010000:3.0:1789760770.371881:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61464576 -> 61468672 00002000:00010000:3.0:1789760770.371887:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120072 -> 120080 00010000:00010000:3.0:1789760770.371902:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4488 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61464576->61468671] (req 61464576->61468671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.371916:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4488 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61464576->61468671] (req 61464576->61468671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.371925:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4488 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61464576->61468671] (req 61464576->61468671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.371934:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760770.371939:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4488 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61464576->61468671] (req 61464576->61468671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.371947:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4488 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61464576->61468671] (req 61464576->61468671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.371968:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4488 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61464576->61468671] (req 61464576->61468671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.371975:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4488 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61464576->61468671] (req 61464576->61468671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.373884:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04560e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554685568:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10829us (11357us total) trans 4294982315 rc 0/0 00000100:00100000:3.0:1789760770.373901:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15017 00000100:00100000:3.0:1789760770.373911:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713591895925 time 713608649433 state 0x2 ref 0 00000100:00100000:2.0:1789760770.380856:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760770.380959:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554685696 00000100:00100000:3.0:1789760770.380969:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554685696 00000100:00100000:3.0:1789760770.381020:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713591895925 time 713608649433 state 0x0 ref 1 00000100:00100000:3.0:1789760770.381036:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713608909849 time 713608649433 state 0x1 ref 1 00000100:00100000:3.0:1789760770.381069:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713608909849 time 713626686717 state 0x0 ref 1 00000100:00100000:3.0:1789760770.381082:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04562a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554685696:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760770.381127:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61468672, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d510 00010000:00010000:3.0:1789760770.381137:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c448f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.381165:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760770.381169:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.381175:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c448f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61468672->61472767] (req 61468672->61472767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.381188:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c448f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61468672->61472767] (req 61468672->61472767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760770.386311:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61468672 -> 61472768 00002000:00010000:3.0:1789760770.386317:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120080 -> 120088 00010000:00010000:3.0:1789760770.386331:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c448f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61468672->61472767] (req 61468672->61472767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.386351:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c448f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61468672->61472767] (req 61468672->61472767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.386360:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c448f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61468672->61472767] (req 61468672->61472767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.386369:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760770.386373:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c448f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61468672->61472767] (req 61468672->61472767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.386405:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c448f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61468672->61472767] (req 61468672->61472767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.386424:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c448f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61468672->61472767] (req 61468672->61472767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.386433:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c448f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61468672->61472767] (req 61468672->61472767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.386625:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04562a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554685696:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5535us (5768us total) trans 4294982316 rc 0/0 00000100:00100000:3.0:1789760770.386641:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15018 00000100:00100000:3.0:1789760770.386655:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713608909849 time 713626686717 state 0x2 ref 0 00000100:00100000:2.0:1789760770.391603:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:3.0:1789760770.403467:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61472768 -> 61476864 00002000:00010000:3.0:1789760770.403473:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120088 -> 120096 00010000:00010000:3.0:1789760770.403487:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4496 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61472768->61476863] (req 61472768->61476863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.403503:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4496 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61472768->61476863] (req 61472768->61476863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.403513:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4496 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61472768->61476863] (req 61472768->61476863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.403522:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760770.403608:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4496 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61472768->61476863] (req 61472768->61476863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.403620:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4496 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61472768->61476863] (req 61472768->61476863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.404062:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4496 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61472768->61476863] (req 61472768->61476863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.404073:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4496 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61472768->61476863] (req 61472768->61476863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.404363:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04560380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554685824:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12474us (12765us total) trans 4294982317 rc 0/0 00000100:00100000:2.0:1789760770.404393:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15019 00000100:00100000:2.0:1789760770.404406:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713626947133 time 713637488483 state 0x2 ref 0 00000100:00100000:2.0:1789760770.409516:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760770.417314:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982318 is committed 00000100:00100000:3.0:1789760770.422301:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760770.422544:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554686080 00000100:00100000:2.0:1789760770.422553:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554686080 00000100:00100000:2.0:1789760770.422610:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713637748899 time 713655651065 state 0x0 ref 1 00000100:00100000:2.0:1789760770.422626:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713655911481 time 713655651065 state 0x1 ref 1 00000100:00100000:2.0:1789760770.422661:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713655911481 time 713668278938 state 0x0 ref 1 00000100:00100000:2.0:1789760770.422675:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554686080:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760770.422721:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61480960, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d528 00010000:00010000:2.0:1789760770.422731:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c44a4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.422748:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760770.422752:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.422943:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c44a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61480960->61485055] (req 61480960->61485055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.422961:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c44a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61480960->61485055] (req 61480960->61485055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760770.435440:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61480960 -> 61485056 00002000:00010000:2.0:1789760770.435445:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120104 -> 120112 00010000:00010000:2.0:1789760770.435458:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c44a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61480960->61485055] (req 61480960->61485055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.435471:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c44a4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61480960->61485055] (req 61480960->61485055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.435480:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c44a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61480960->61485055] (req 61480960->61485055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.435487:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760770.435492:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c44a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61480960->61485055] (req 61480960->61485055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.435507:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c44a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61480960->61485055] (req 61480960->61485055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.435526:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c44a4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61480960->61485055] (req 61480960->61485055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.435533:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c44a4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61480960->61485055] (req 61480960->61485055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.435783:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554686080:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13098us (13480us total) trans 4294982319 rc 0/0 00000100:00100000:2.0:1789760770.435794:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15021 00000100:00100000:2.0:1789760770.435803:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713655911481 time 713668278938 state 0x2 ref 0 00000100:00100000:2.0:1789760770.449289:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760770.449475:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554686208 00000100:00100000:3.0:1789760770.449486:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554686208 00000100:00100000:3.0:1789760770.449553:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713655911481 time 713668278938 state 0x0 ref 1 00000100:00100000:3.0:1789760770.449572:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713668539354 time 713668278938 state 0x1 ref 1 00000100:00100000:3.0:1789760770.449598:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713668539354 time 713695216539 state 0x0 ref 1 00000100:00100000:3.0:1789760770.449610:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04562680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554686208:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760770.449657:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61485056, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d530 00010000:00010000:3.0:1789760770.449667:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44ab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.449683:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760770.449687:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.449694:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61485056->61489151] (req 61485056->61489151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.449708:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61485056->61489151] (req 61485056->61489151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.450929:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03e58e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3648:1876699573912448:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789760770.451029:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760770.451491:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573912448 00000100:00100000:2.0:1789760770.451498:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573912448 00000100:00100000:2.0:1789760770.451628:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 379 00000100:00100000:2.0:1789760770.451639:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3648:x1876699573912448:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789760770.451672:0:6525:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446383616 avail 1300144128 00000100:00100000:2.0:1789760770.451735:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3648:x1876699573912448:12345-0@lo:41:osp-pre-1-0.0 Request processed in 99us (706us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760770.451741:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 379 00000001:00080000:2.0:1789760770.458668:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982320 is committed 00002000:00010000:3.0:1789760770.458814:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61485056 -> 61489152 00002000:00010000:3.0:1789760770.458820:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120112 -> 120120 00010000:00010000:3.0:1789760770.458916:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61485056->61489151] (req 61485056->61489151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.458931:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44ab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61485056->61489151] (req 61485056->61489151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.458940:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61485056->61489151] (req 61485056->61489151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.458948:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760770.458954:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61485056->61489151] (req 61485056->61489151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.458963:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44ab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61485056->61489151] (req 61485056->61489151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.458982:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44ab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61485056->61489151] (req 61485056->61489151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.458989:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44ab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61485056->61489151] (req 61485056->61489151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.459087:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04562680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554686208:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9473us (9804us total) trans 4294982320 rc 0/0 00000100:00100000:3.0:1789760770.459098:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15022 00000100:00100000:3.0:1789760770.459108:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713668539354 time 713695216539 state 0x2 ref 0 00000100:00100000:2.0:1789760770.465474:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554686336 00000100:00100000:2.0:1789760770.465485:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554686336 00000100:00100000:2.0:1789760770.465549:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713668539354 time 713695216539 state 0x0 ref 1 00000100:00100000:2.0:1789760770.465566:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713695476955 time 713695216539 state 0x1 ref 1 00000100:00100000:2.0:1789760770.465596:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713695476955 time 713711215000 state 0x0 ref 1 00000100:00100000:2.0:1789760770.465611:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554686336:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760770.465671:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61489152, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d538 00010000:00010000:2.0:1789760770.465682:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c44b2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.465702:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760770.465706:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.465714:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c44b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61489152->61493247] (req 61489152->61493247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.465730:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c44b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61489152->61493247] (req 61489152->61493247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.473721:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982321 is committed 00002000:00010000:2.0:1789760770.473879:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61489152 -> 61493248 00002000:00010000:2.0:1789760770.473883:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120120 -> 120128 00010000:00010000:2.0:1789760770.473897:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c44b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61489152->61493247] (req 61489152->61493247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.473909:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c44b2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61489152->61493247] (req 61489152->61493247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.473924:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c44b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61489152->61493247] (req 61489152->61493247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.473932:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760770.473936:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c44b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61489152->61493247] (req 61489152->61493247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.473944:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c44b2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61489152->61493247] (req 61489152->61493247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.473962:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c44b2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61489152->61493247] (req 61489152->61493247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.473970:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c44b2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61489152->61493247] (req 61489152->61493247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.474061:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554686336:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8448us (8806us total) trans 4294982321 rc 0/0 00000100:00100000:2.0:1789760770.474073:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15023 00000100:00100000:2.0:1789760770.474082:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713695476955 time 713711215000 state 0x2 ref 0 00000100:00100000:2.0:1789760770.477197:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760770.486924:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982322 is committed 00002000:00010000:0.0:1789760770.487235:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61493248 -> 61497344 00002000:00010000:0.0:1789760770.487241:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120128 -> 120136 00010000:00010000:0.0:1789760770.487255:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c44b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61493248->61497343] (req 61493248->61497343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.487268:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c44b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61493248->61497343] (req 61493248->61497343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.487278:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c44b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61493248->61497343] (req 61493248->61497343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.487285:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760770.487291:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c44b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61493248->61497343] (req 61493248->61497343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.487300:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c44b9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61493248->61497343] (req 61493248->61497343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.487325:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c44b9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61493248->61497343] (req 61493248->61497343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.487334:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c44b9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61493248->61497343] (req 61493248->61497343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.487600:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554686464:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10164us (10405us total) trans 4294982322 rc 0/0 00000100:00100000:0.0:1789760770.487610:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15024 00000100:00100000:0.0:1789760770.487620:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713711475416 time 713723036988 state 0x2 ref 0 00000100:00100000:0.0:1789760770.491744:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760770.491958:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554686592 00000100:00100000:3.0:1789760770.491968:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554686592 00000100:00100000:3.0:1789760770.492098:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713711475416 time 713723036988 state 0x0 ref 1 00000100:00100000:3.0:1789760770.492115:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713723297404 time 713723036988 state 0x1 ref 1 00000100:00100000:3.0:1789760770.492250:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713723297404 time 713737766254 state 0x0 ref 1 00000100:00100000:3.0:1789760770.492265:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554686592:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760770.492316:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61497344, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d548 00010000:00010000:3.0:1789760770.492325:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44c0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.492341:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760770.492346:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.492352:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61497344->61501439] (req 61497344->61501439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.492366:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61497344->61501439] (req 61497344->61501439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760770.499971:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982323 is committed 00002000:00010000:3.0:1789760770.500205:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61497344 -> 61501440 00002000:00010000:3.0:1789760770.500210:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120136 -> 120144 00010000:00010000:3.0:1789760770.500223:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61497344->61501439] (req 61497344->61501439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.500236:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44c0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61497344->61501439] (req 61497344->61501439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.500245:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61497344->61501439] (req 61497344->61501439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.500253:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760770.500259:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61497344->61501439] (req 61497344->61501439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.500268:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44c0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61497344->61501439] (req 61497344->61501439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.500288:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44c0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61497344->61501439] (req 61497344->61501439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.500296:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44c0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61497344->61501439] (req 61497344->61501439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.500434:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554686592:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8166us (8694us total) trans 4294982323 rc 0/0 00000100:00100000:3.0:1789760770.500446:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15025 00000100:00100000:3.0:1789760770.500456:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713723297404 time 713737766254 state 0x2 ref 0 00000100:00100000:3.0:1789760770.505626:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554686720 00000100:00100000:3.0:1789760770.505636:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554686720 00000100:00100000:3.0:1789760770.505692:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713723297404 time 713737766254 state 0x0 ref 1 00000100:00100000:3.0:1789760770.505709:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713738026670 time 713737766254 state 0x1 ref 1 00000100:00100000:3.0:1789760770.505739:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713738026670 time 713751357350 state 0x0 ref 1 00000100:00100000:3.0:1789760770.505752:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554686720:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760770.505812:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61501440, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d550 00010000:00010000:3.0:1789760770.505821:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44c7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.505838:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760770.505843:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.505849:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61501440->61505535] (req 61501440->61505535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.505862:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61501440->61505535] (req 61501440->61505535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760770.512440:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982324 is committed 00002000:00010000:3.0:1789760770.512900:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61501440 -> 61505536 00002000:00010000:3.0:1789760770.512906:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120144 -> 120152 00010000:00010000:3.0:1789760770.512919:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61501440->61505535] (req 61501440->61505535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.512931:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61501440->61505535] (req 61501440->61505535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.512940:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61501440->61505535] (req 61501440->61505535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.512947:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760770.512951:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61501440->61505535] (req 61501440->61505535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.512960:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61501440->61505535] (req 61501440->61505535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.512980:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44c7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61501440->61505535] (req 61501440->61505535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.512987:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44c7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61501440->61505535] (req 61501440->61505535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.513078:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554686720:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7322us (7583us total) trans 4294982324 rc 0/0 00000100:00100000:3.0:1789760770.513089:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15026 00000100:00100000:3.0:1789760770.513099:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713738026670 time 713751357350 state 0x2 ref 0 00000100:00100000:2.0:1789760770.517864:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760770.523352:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982325 is committed 00000100:00100000:3.0:1789760770.528575:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760770.535478:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982326 is committed 00000100:00100000:3.0:1789760770.538992:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760770.539319:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554687104 00000100:00100000:2.0:1789760770.539330:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554687104 00000100:00100000:2.0:1789760770.539429:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713763991121 time 713774828467 state 0x0 ref 1 00000100:00100000:2.0:1789760770.539446:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713775088883 time 713774828467 state 0x1 ref 1 00000100:00100000:2.0:1789760770.539473:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713775088883 time 713785091918 state 0x0 ref 1 00000100:00100000:2.0:1789760770.539486:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554687104:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760770.539542:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61513728, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d568 00010000:00010000:2.0:1789760770.539551:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c44dc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.539567:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760770.539573:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.539579:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c44dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61513728->61517823] (req 61513728->61517823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.539593:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c44dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61513728->61517823] (req 61513728->61517823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.544799:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982327 is committed 00002000:00010000:2.0:1789760770.544921:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61513728 -> 61517824 00002000:00010000:2.0:1789760770.544927:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120168 -> 120176 00010000:00010000:2.0:1789760770.544943:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c44dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61513728->61517823] (req 61513728->61517823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.544962:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c44dc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61513728->61517823] (req 61513728->61517823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.544976:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c44dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61513728->61517823] (req 61513728->61517823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.544984:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760770.544988:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c44dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61513728->61517823] (req 61513728->61517823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.544996:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c44dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61513728->61517823] (req 61513728->61517823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.545017:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c44dc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61513728->61517823] (req 61513728->61517823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.545025:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c44dc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61513728->61517823] (req 61513728->61517823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.545138:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554687104:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5649us (6150us total) trans 4294982327 rc 0/0 00000100:00100000:2.0:1789760770.545160:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15029 00000100:00100000:2.0:1789760770.545170:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713775088883 time 713785091918 state 0x2 ref 0 00000100:00100000:2.0:1789760770.550281:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760770.550414:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554687232 00000100:00100000:3.0:1789760770.550423:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554687232 00000100:00100000:3.0:1789760770.550476:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713775088883 time 713785091918 state 0x0 ref 1 00000100:00100000:3.0:1789760770.550492:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713785352334 time 713785091918 state 0x1 ref 1 00000100:00100000:3.0:1789760770.550522:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713785352334 time 713796140698 state 0x0 ref 1 00000100:00100000:3.0:1789760770.550534:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554687232:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760770.550580:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61517824, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d570 00010000:00010000:3.0:1789760770.550590:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44e3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.550606:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760770.550611:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.550628:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61517824->61521919] (req 61517824->61521919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.550644:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61517824->61521919] (req 61517824->61521919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.557323:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982328 is committed 00000100:00100000:3.0:1789760770.561427:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760770.569235:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982329 is committed 00000100:00100000:3.0:1789760770.573620:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760770.580848:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982330 is committed 00000100:00100000:2.0:1789760770.595746:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760770.595865:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554687616 00000100:00100000:3.0:1789760770.595874:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554687616 00000100:00100000:3.0:1789760770.595937:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713808347495 time 713819448900 state 0x0 ref 1 00000100:00100000:3.0:1789760770.595954:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713819709316 time 713819448900 state 0x1 ref 1 00000100:00100000:3.0:1789760770.596024:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713819709316 time 713841642110 state 0x0 ref 1 00000100:00100000:3.0:1789760770.596037:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554687616:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760770.596086:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61530112, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d588 00010000:00010000:3.0:1789760770.596097:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44f8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.596115:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760770.596119:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.596125:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61530112->61534207] (req 61530112->61534207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.596139:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61530112->61534207] (req 61530112->61534207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.601216:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982331 is committed 00002000:00010000:3.0:1789760770.601444:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61530112 -> 61534208 00002000:00010000:3.0:1789760770.601450:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120200 -> 120208 00010000:00010000:3.0:1789760770.601465:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61530112->61534207] (req 61530112->61534207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.601479:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44f8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61530112->61534207] (req 61530112->61534207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.601489:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61530112->61534207] (req 61530112->61534207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.601497:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760770.601503:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61530112->61534207] (req 61530112->61534207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.601513:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61530112->61534207] (req 61530112->61534207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.601616:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44f8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61530112->61534207] (req 61530112->61534207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.601625:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c44f8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61530112->61534207] (req 61530112->61534207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.601779:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554687616:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5729us (6082us total) trans 4294982331 rc 0/0 00000100:00100000:3.0:1789760770.601791:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15033 00000100:00100000:3.0:1789760770.601800:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713819709316 time 713841642110 state 0x2 ref 0 00000100:00100000:3.0:1789760770.607324:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760770.612932:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982332 is committed 00002000:00010000:0.0:1789760770.613049:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61534208 -> 61538304 00002000:00010000:0.0:1789760770.613054:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120208 -> 120216 00010000:00010000:0.0:1789760770.613068:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c44ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61534208->61538303] (req 61534208->61538303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.613090:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c44ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61534208->61538303] (req 61534208->61538303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.613100:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c44ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61534208->61538303] (req 61534208->61538303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.613108:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760770.613118:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c44ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61534208->61538303] (req 61534208->61538303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.613127:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c44ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61534208->61538303] (req 61534208->61538303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.613165:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c44ff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61534208->61538303] (req 61534208->61538303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.613174:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c44ff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61534208->61538303] (req 61534208->61538303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.613321:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554687744:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5672us (6001us total) trans 4294982332 rc 0/0 00000100:00100000:0.0:1789760770.613332:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15034 00000100:00100000:0.0:1789760770.613341:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713841902526 time 713853252028 state 0x2 ref 0 00000100:00100000:2.0:1789760770.620281:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554687872 00000100:00100000:2.0:1789760770.620290:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554687872 00000100:00100000:2.0:1789760770.620345:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713841902526 time 713853252028 state 0x0 ref 1 00000100:00100000:2.0:1789760770.620361:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713853512444 time 713853252028 state 0x1 ref 1 00000100:00100000:2.0:1789760770.620419:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713853512444 time 713866037005 state 0x0 ref 1 00000100:00100000:2.0:1789760770.620431:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554687872:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760770.620480:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61538304, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d598 00010000:00010000:2.0:1789760770.620489:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4506 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.620515:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760770.620519:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.620525:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4506 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61538304->61542399] (req 61538304->61542399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.620540:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4506 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61538304->61542399] (req 61538304->61542399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760770.630088:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982333 is committed 00002000:00010000:2.0:1789760770.630217:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61538304 -> 61542400 00002000:00010000:2.0:1789760770.630223:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120216 -> 120224 00010000:00010000:2.0:1789760770.630236:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4506 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61538304->61542399] (req 61538304->61542399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.630255:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4506 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61538304->61542399] (req 61538304->61542399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.630264:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4506 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61538304->61542399] (req 61538304->61542399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.630271:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760770.630275:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4506 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61538304->61542399] (req 61538304->61542399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.630284:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4506 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61538304->61542399] (req 61538304->61542399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.630304:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4506 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61538304->61542399] (req 61538304->61542399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.630312:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4506 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61538304->61542399] (req 61538304->61542399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.630490:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554687872:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10054us (10369us total) trans 4294982333 rc 0/0 00000100:00100000:2.0:1789760770.630501:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15035 00000100:00100000:2.0:1789760770.630510:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713853512444 time 713866037005 state 0x2 ref 0 00000100:00100000:2.0:1789760770.636797:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760770.636879:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554688000 00000100:00100000:0.0:1789760770.636888:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554688000 00000100:00100000:0.0:1789760770.636952:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713853512444 time 713866037005 state 0x0 ref 1 00000100:00100000:0.0:1789760770.636968:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713866297421 time 713866037005 state 0x1 ref 1 00000100:00100000:0.0:1789760770.637001:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713866297421 time 713882619229 state 0x0 ref 1 00000100:00100000:0.0:1789760770.637012:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554688000:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760770.637057:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61542400, mtime: 0x6aad9501, atime: 0x0, ctime: 0x6aad9501, blocks: 0x1d5a0 00010000:00010000:0.0:1789760770.637067:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c450d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.637088:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760770.637093:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.637100:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c450d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61542400->61546495] (req 61542400->61546495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.637113:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c450d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61542400->61546495] (req 61542400->61546495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.642540:0:6508:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760770.645983:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982334 is committed 00000100:00100000:0.0:1789760770.653037:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760770.654480:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554688128 00000100:00100000:3.0:1789760770.654493:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554688128 00000100:00100000:3.0:1789760770.654700:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713866297421 time 713882619229 state 0x0 ref 1 00000100:00100000:3.0:1789760770.654716:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713882879645 time 713882619229 state 0x1 ref 1 00000100:00100000:3.0:1789760770.654812:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713882879645 time 713900429926 state 0x0 ref 1 00000100:00100000:3.0:1789760770.654825:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554688128:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760770.654873:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61546496, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d5a8 00010000:00010000:3.0:1789760770.654883:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c4514 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.654900:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:3.0:1789760770.654905:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.654911:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c4514 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61546496->61550591] (req 61546496->61550591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.654924:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c4514 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61546496->61550591] (req 61546496->61550591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.660808:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982335 is committed 00002000:00010000:2.0:1789760770.665845:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61546496 -> 61550592 00002000:00010000:2.0:1789760770.665855:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120232 -> 120240 00010000:00010000:2.0:1789760770.665868:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c4514 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61546496->61550591] (req 61546496->61550591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.665880:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c4514 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61546496->61550591] (req 61546496->61550591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.665889:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c4514 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61546496->61550591] (req 61546496->61550591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.665897:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760770.665902:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c4514 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61546496->61550591] (req 61546496->61550591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.665911:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c4514 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61546496->61550591] (req 61546496->61550591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.665931:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c4514 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61546496->61550591] (req 61546496->61550591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.665939:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c4514 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61546496->61550591] (req 61546496->61550591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.666066:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554688128:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11238us (13035us total) trans 4294982335 rc 0/0 00000100:00100000:2.0:1789760770.666082:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15037 00000100:00100000:2.0:1789760770.666092:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713882879645 time 713900429926 state 0x2 ref 0 00000100:00100000:3.0:1789760770.671559:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760770.671837:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554688256 00000100:00100000:0.0:1789760770.671848:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554688256 00000100:00100000:0.0:1789760770.671903:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713882879645 time 713900429926 state 0x0 ref 1 00000100:00100000:0.0:1789760770.671919:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713900690342 time 713900429926 state 0x1 ref 1 00000100:00100000:0.0:1789760770.671955:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713900690342 time 713917572792 state 0x0 ref 1 00000100:00100000:0.0:1789760770.671968:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554688256:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760770.672016:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61550592, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d5b0 00010000:00010000:0.0:1789760770.672026:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c451b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.672048:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760770.672053:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.672059:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c451b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61550592->61554687] (req 61550592->61554687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.672073:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c451b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61550592->61554687] (req 61550592->61554687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.677974:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982336 is committed 00000100:00100000:3.0:1789760770.698039:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760770.698988:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554688384 00000100:00100000:2.0:1789760770.699001:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554688384 00000100:00100000:2.0:1789760770.699055:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713900690342 time 713917572792 state 0x0 ref 1 00000100:00100000:2.0:1789760770.699072:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713917833208 time 713917572792 state 0x1 ref 1 00000100:00100000:2.0:1789760770.699105:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713917833208 time 713944723226 state 0x0 ref 1 00000100:00100000:2.0:1789760770.699116:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554688384:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760770.699162:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61554688, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d5b8 00010000:00010000:2.0:1789760770.699171:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4522 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.699187:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789760770.699192:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.699197:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4522 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61554688->61558783] (req 61554688->61558783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.699632:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4522 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61554688->61558783] (req 61554688->61558783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.704950:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982337 is committed 00002000:00010000:2.0:1789760770.705078:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61554688 -> 61558784 00002000:00010000:2.0:1789760770.705084:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120248 -> 120256 00010000:00010000:2.0:1789760770.705099:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4522 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61554688->61558783] (req 61554688->61558783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.705112:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4522 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61554688->61558783] (req 61554688->61558783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.705121:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4522 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61554688->61558783] (req 61554688->61558783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.705129:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760770.705134:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4522 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61554688->61558783] (req 61554688->61558783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.705144:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4522 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61554688->61558783] (req 61554688->61558783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.705166:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4522 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61554688->61558783] (req 61554688->61558783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.705174:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4522 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61554688->61558783] (req 61554688->61558783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.705288:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554688384:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6168us (7253us total) trans 4294982337 rc 0/0 00000100:00100000:2.0:1789760770.705299:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15039 00000100:00100000:2.0:1789760770.705311:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713917833208 time 713944723226 state 0x2 ref 0 00000100:00100000:0.0:1789760770.714963:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:0.0:1789760770.720270:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982338 is committed 00000100:00100000:3.0:1789760770.734508:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760770.734941:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554688640 00000100:00100000:0.0:1789760770.734951:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554688640 00000100:00100000:0.0:1789760770.735009:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713944983642 time 713960819152 state 0x0 ref 1 00000100:00100000:0.0:1789760770.735025:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713961079568 time 713960819152 state 0x1 ref 1 00000100:00100000:0.0:1789760770.735059:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713961079568 time 713980677411 state 0x0 ref 1 00000100:00100000:0.0:1789760770.735074:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a272a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554688640:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760770.735128:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61562880, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d5c8 00010000:00010000:0.0:1789760770.735138:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4530 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.735154:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789760770.735160:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.735166:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4530 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61562880->61566975] (req 61562880->61566975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.735180:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4530 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61562880->61566975] (req 61562880->61566975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760770.740316:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982339 is committed 00002000:00010000:0.0:1789760770.740449:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61562880 -> 61566976 00002000:00010000:0.0:1789760770.740455:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120264 -> 120272 00010000:00010000:0.0:1789760770.740469:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4530 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61562880->61566975] (req 61562880->61566975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.740481:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4530 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61562880->61566975] (req 61562880->61566975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.740687:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4530 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61562880->61566975] (req 61562880->61566975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.740695:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760770.740700:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4530 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61562880->61566975] (req 61562880->61566975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.740710:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4530 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61562880->61566975] (req 61562880->61566975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.740729:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4530 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61562880->61566975] (req 61562880->61566975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.740743:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4530 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61562880->61566975] (req 61562880->61566975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.740895:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a272a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554688640:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5819us (6401us total) trans 4294982339 rc 0/0 00000100:00100000:0.0:1789760770.740905:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15041 00000100:00100000:0.0:1789760770.740914:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713961079568 time 713980677411 state 0x2 ref 0 00000100:00100000:3.0:1789760770.746446:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760770.755241:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982340 is committed 00000100:00100000:3.0:1789760770.761081:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760770.761839:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554688896 00000100:00100000:2.0:1789760770.761849:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554688896 00000100:00100000:2.0:1789760770.761903:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713980937827 time 713994714381 state 0x0 ref 1 00000100:00100000:2.0:1789760770.761920:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713994974797 time 713994714381 state 0x1 ref 1 00000100:00100000:2.0:1789760770.761992:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713994974797 time 714007610167 state 0x0 ref 1 00000100:00100000:2.0:1789760770.762004:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a270700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554688896:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760770.762047:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61571072, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d5d8 00010000:00010000:2.0:1789760770.762057:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c453e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.762073:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760770.762077:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.762082:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c453e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61571072->61575167] (req 61571072->61575167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.762095:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c453e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61571072->61575167] (req 61571072->61575167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760770.769036:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982341 is committed 00000100:00100000:3.0:1789760770.772970:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760770.773340:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554689024 00000100:00100000:2.0:1789760770.773351:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554689024 00000100:00100000:2.0:1789760770.773432:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 713994974797 time 714007610167 state 0x0 ref 1 00000100:00100000:2.0:1789760770.773448:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714007870583 time 714007610167 state 0x1 ref 1 00000100:00100000:2.0:1789760770.773477:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714007870583 time 714019095148 state 0x0 ref 1 00000100:00100000:2.0:1789760770.773490:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554689024:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760770.773539:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61575168, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d5e0 00010000:00010000:2.0:1789760770.773549:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4545 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.773565:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760770.773581:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.773587:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4545 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61575168->61579263] (req 61575168->61579263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.773602:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4545 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61575168->61579263] (req 61575168->61579263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760770.781277:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982342 is committed 00000100:00100000:3.0:1789760770.786682:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760770.786858:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554689152 00000100:00100000:0.0:1789760770.786869:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554689152 00000100:00100000:0.0:1789760770.786926:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714007870583 time 714019095148 state 0x0 ref 1 00000100:00100000:0.0:1789760770.786953:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714019355564 time 714019095148 state 0x1 ref 1 00000100:00100000:0.0:1789760770.786984:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714019355564 time 714032602970 state 0x0 ref 1 00000100:00100000:0.0:1789760770.786996:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554689152:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760770.787061:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61579264, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d5e8 00010000:00010000:0.0:1789760770.787070:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c454c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.787086:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760770.787091:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760770.787098:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c454c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61579264->61583359] (req 61579264->61583359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.787116:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c454c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61579264->61583359] (req 61579264->61583359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760770.803057:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982343 is committed 00000100:00100000:0.0:1789760770.810199:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760770.819247:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982344 is committed 00002000:00010000:2.0:1789760770.821623:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61583360 -> 61587456 00002000:00010000:2.0:1789760770.821632:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120304 -> 120312 00010000:00010000:2.0:1789760770.821646:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4553 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61583360->61587455] (req 61583360->61587455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.821659:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4553 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61583360->61587455] (req 61583360->61587455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.821674:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4553 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61583360->61587455] (req 61583360->61587455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.821681:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760770.821686:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4553 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61583360->61587455] (req 61583360->61587455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.821695:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4553 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61583360->61587455] (req 61583360->61587455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.821716:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4553 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61583360->61587455] (req 61583360->61587455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.821723:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4553 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61583360->61587455] (req 61583360->61587455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.822445:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554689280:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11809us (12248us total) trans 4294982344 rc 0/0 00000100:00100000:2.0:1789760770.822457:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15046 00000100:00100000:2.0:1789760770.822467:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714032863386 time 714056236024 state 0x2 ref 0 00000100:00100000:3.0:1789760770.826707:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760770.837281:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982345 is committed 00000100:00100000:3.0:1789760770.844281:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760770.844950:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554689536 00000100:00100000:2.0:1789760770.844963:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554689536 00000100:00100000:3.0:1789760770.845099:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714056496440 time 714072815861 state 0x0 ref 1 00000100:00100000:3.0:1789760770.845115:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714073076277 time 714072815861 state 0x1 ref 1 00000100:00100000:3.0:1789760770.845238:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714073076277 time 714090855021 state 0x0 ref 1 00000100:00100000:3.0:1789760770.845250:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554689536:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760770.845298:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61591552, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d600 00010000:00010000:3.0:1789760770.845306:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4561 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.845322:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760770.845326:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.845332:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4561 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61591552->61595647] (req 61591552->61595647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.845345:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4561 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61591552->61595647] (req 61591552->61595647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760770.852876:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982346 is committed 00002000:00010000:3.0:1789760770.852999:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61591552 -> 61595648 00002000:00010000:3.0:1789760770.853005:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120320 -> 120328 00010000:00010000:3.0:1789760770.853016:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4561 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61591552->61595647] (req 61591552->61595647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.853029:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4561 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61591552->61595647] (req 61591552->61595647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.853038:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4561 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61591552->61595647] (req 61591552->61595647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.853045:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760770.853050:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4561 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61591552->61595647] (req 61591552->61595647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.853065:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4561 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61591552->61595647] (req 61591552->61595647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.853085:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4561 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61591552->61595647] (req 61591552->61595647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.853093:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4561 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61591552->61595647] (req 61591552->61595647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.853234:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554689536:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7980us (8959us total) trans 4294982346 rc 0/0 00000100:00100000:3.0:1789760770.853244:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15048 00000100:00100000:3.0:1789760770.853253:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714073076277 time 714090855021 state 0x2 ref 0 00000100:00100000:0.0:1789760770.860550:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760770.861024:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554689664 00000100:00100000:2.0:1789760770.861034:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554689664 00000100:00100000:2.0:1789760770.861090:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714073076277 time 714090855021 state 0x0 ref 1 00000100:00100000:2.0:1789760770.861105:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714091115437 time 714090855021 state 0x1 ref 1 00000100:00100000:2.0:1789760770.861138:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714091115437 time 714106756732 state 0x0 ref 1 00000100:00100000:2.0:1789760770.861158:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554689664:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760770.861202:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61595648, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d608 00010000:00010000:2.0:1789760770.861210:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4568 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.861225:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760770.861229:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.861234:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4568 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61595648->61599743] (req 61595648->61599743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.861248:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4568 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61595648->61599743] (req 61595648->61599743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760770.869100:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982347 is committed 00000100:00100000:2.0:1789760770.878488:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554689792 00000100:00100000:2.0:1789760770.878497:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554689792 00000100:00100000:2.0:1789760770.878562:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714091115437 time 714106756732 state 0x0 ref 1 00000100:00100000:2.0:1789760770.878577:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714107017148 time 714106756732 state 0x1 ref 1 00000100:00100000:2.0:1789760770.878607:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714107017148 time 714124225805 state 0x0 ref 1 00000100:00100000:2.0:1789760770.878623:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554689792:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760770.878670:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61599744, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d610 00010000:00010000:2.0:1789760770.878680:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c456f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.878696:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760770.878700:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.878707:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c456f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61599744->61603839] (req 61599744->61603839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.878720:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c456f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61599744->61603839] (req 61599744->61603839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760770.884477:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982348 is committed 00000100:00100000:3.0:1789760770.891999:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:0.0:1789760770.899546:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982349 is committed 00000100:00100000:0.0:1789760770.906803:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760770.907002:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554690048 00000100:00100000:3.0:1789760770.907014:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554690048 00000100:00100000:3.0:1789760770.907078:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714124486221 time 714137871291 state 0x0 ref 1 00000100:00100000:3.0:1789760770.907095:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714138131707 time 714137871291 state 0x1 ref 1 00000100:00100000:3.0:1789760770.907225:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714138131707 time 714152841154 state 0x0 ref 1 00000100:00100000:3.0:1789760770.907241:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554690048:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760770.907290:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61607936, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d620 00010000:00010000:3.0:1789760770.907300:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c457d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.907318:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760770.907322:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760770.907328:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c457d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61607936->61612031] (req 61607936->61612031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.907341:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c457d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61607936->61612031] (req 61607936->61612031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760770.916988:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982350 is committed 00002000:00010000:3.0:1789760770.917153:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61607936 -> 61612032 00002000:00010000:3.0:1789760770.917158:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120352 -> 120360 00010000:00010000:3.0:1789760770.917172:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c457d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61607936->61612031] (req 61607936->61612031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.917186:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c457d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61607936->61612031] (req 61607936->61612031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.917195:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c457d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61607936->61612031] (req 61607936->61612031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.917202:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760770.917317:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c457d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61607936->61612031] (req 61607936->61612031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.917329:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c457d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61607936->61612031] (req 61607936->61612031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.917351:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c457d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61607936->61612031] (req 61607936->61612031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760770.917359:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c457d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61607936->61612031] (req 61607936->61612031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760770.917510:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554690048:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10268us (10713us total) trans 4294982350 rc 0/0 00000100:00100000:3.0:1789760770.917521:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15052 00000100:00100000:3.0:1789760770.917530:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714138131707 time 714152841154 state 0x2 ref 0 00000100:00100000:3.0:1789760770.924173:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760770.924736:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554690176 00000100:00100000:2.0:1789760770.924746:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554690176 00000100:00100000:2.0:1789760770.924812:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714138131707 time 714152841154 state 0x0 ref 1 00000100:00100000:2.0:1789760770.924828:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714153101570 time 714152841154 state 0x1 ref 1 00000100:00100000:2.0:1789760770.924862:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714153101570 time 714170480510 state 0x0 ref 1 00000100:00100000:2.0:1789760770.924875:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554690176:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760770.924939:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61612032, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d628 00010000:00010000:2.0:1789760770.924948:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4584 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.924971:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760770.924976:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760770.924983:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4584 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61612032->61616127] (req 61612032->61616127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.924998:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4584 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61612032->61616127] (req 61612032->61616127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760770.939090:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61612032 -> 61616128 00002000:00010000:2.0:1789760770.939100:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120360 -> 120368 00010000:00010000:2.0:1789760770.939114:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4584 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61612032->61616127] (req 61612032->61616127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.939128:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4584 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61612032->61616127] (req 61612032->61616127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.939137:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4584 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61612032->61616127] (req 61612032->61616127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.939146:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760770.939150:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4584 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61612032->61616127] (req 61612032->61616127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.939160:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4584 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61612032->61616127] (req 61612032->61616127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.939186:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4584 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61612032->61616127] (req 61612032->61616127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.939194:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4584 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61612032->61616127] (req 61612032->61616127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.939291:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554690176:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14413us (15125us total) trans 4294982351 rc 0/0 00000100:00100000:2.0:1789760770.939303:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15053 00000100:00100000:2.0:1789760770.939312:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714153101570 time 714170480510 state 0x2 ref 0 00000100:00100000:2.0:1789760770.945172:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:0.0:1789760770.954569:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61616128 -> 61620224 00002000:00010000:0.0:1789760770.954574:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120368 -> 120376 00010000:00010000:0.0:1789760770.954588:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c458b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61616128->61620223] (req 61616128->61620223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.954599:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c458b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61616128->61620223] (req 61616128->61620223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.955285:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c458b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61616128->61620223] (req 61616128->61620223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.955299:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760770.955304:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c458b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61616128->61620223] (req 61616128->61620223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.955313:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c458b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61616128->61620223] (req 61616128->61620223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.955338:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c458b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61616128->61620223] (req 61616128->61620223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760770.955347:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c458b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61616128->61620223] (req 61616128->61620223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760770.955544:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554690304:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9577us (10376us total) trans 4294982352 rc 0/0 00000100:00100000:0.0:1789760770.955552:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15054 00000100:00100000:0.0:1789760770.955559:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714170740926 time 714191570058 state 0x2 ref 0 00000100:00100000:0.0:1789760770.959572:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:2.0:1789760770.966250:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61620224 -> 61624320 00002000:00010000:2.0:1789760770.966255:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120376 -> 120384 00010000:00010000:2.0:1789760770.966269:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4592 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61620224->61624319] (req 61620224->61624319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.966282:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4592 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61620224->61624319] (req 61620224->61624319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.966291:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4592 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61620224->61624319] (req 61620224->61624319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.966298:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760770.966303:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4592 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61620224->61624319] (req 61620224->61624319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.966312:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4592 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61620224->61624319] (req 61620224->61624319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.966330:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4592 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61620224->61624319] (req 61620224->61624319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.966337:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4592 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61620224->61624319] (req 61620224->61624319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.966463:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554690432:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6681us (6895us total) trans 4294982353 rc 0/0 00000100:00100000:2.0:1789760770.966473:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15055 00000100:00100000:2.0:1789760770.966482:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714191830474 time 714205376614 state 0x2 ref 0 00000100:00100000:3.0:1789760770.970363:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760770.977689:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982354 is committed 00000100:00100000:0.0:1789760770.988015:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:2.0:1789760770.997448:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61628416 -> 61632512 00002000:00010000:2.0:1789760770.997453:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120392 -> 120400 00010000:00010000:2.0:1789760770.997465:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c45a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61628416->61632511] (req 61628416->61632511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.997478:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c45a0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61628416->61632511] (req 61628416->61632511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.997488:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c45a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61628416->61632511] (req 61628416->61632511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.997503:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760770.997507:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c45a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61628416->61632511] (req 61628416->61632511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.997517:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c45a0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61628416->61632511] (req 61628416->61632511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.997538:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c45a0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61628416->61632511] (req 61628416->61632511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760770.997547:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c45a0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61628416->61632511] (req 61628416->61632511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760770.997634:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe1d050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554690688:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9380us (9623us total) trans 4294982355 rc 0/0 00000100:00100000:2.0:1789760770.997644:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15057 00000100:00100000:2.0:1789760770.997654:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714216477782 time 714233854543 state 0x2 ref 0 00000100:00100000:0.0:1789760771.003589:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760771.003672:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554690816 00000100:00100000:2.0:1789760771.003681:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554690816 00000100:00100000:2.0:1789760771.003744:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714216477782 time 714233854543 state 0x0 ref 1 00000100:00100000:2.0:1789760771.003758:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714234114959 time 714233854543 state 0x1 ref 1 00000100:00100000:2.0:1789760771.003795:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714234114959 time 714249413310 state 0x0 ref 1 00000100:00100000:2.0:1789760771.003821:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554690816:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760771.003865:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61632512, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d650 00010000:00010000:2.0:1789760771.003874:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c45a7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.003888:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760771.003892:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.003899:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c45a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61632512->61636607] (req 61632512->61636607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760771.004029:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1789760771.004278:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c45a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61632512->61636607] (req 61632512->61636607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760771.010453:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61632512 -> 61636608 00002000:00010000:2.0:1789760771.010458:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120400 -> 120408 00010000:00010000:2.0:1789760771.010471:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c45a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61632512->61636607] (req 61632512->61636607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.010484:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c45a7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61632512->61636607] (req 61632512->61636607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.010493:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c45a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61632512->61636607] (req 61632512->61636607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.010500:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760771.010505:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c45a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61632512->61636607] (req 61632512->61636607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.010514:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c45a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61632512->61636607] (req 61632512->61636607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.010534:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c45a7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61632512->61636607] (req 61632512->61636607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.010542:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c45a7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61632512->61636607] (req 61632512->61636607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.010736:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554690816:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6912us (7149us total) trans 4294982356 rc 0/0 00000100:00100000:2.0:1789760771.010747:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15058 00000100:00100000:2.0:1789760771.010755:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714234114959 time 714249413310 state 0x2 ref 0 00000100:00100000:3.0:1789760771.016735:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760771.018629:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554690944 00000100:00100000:2.0:1789760771.018686:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554690944 00000100:00100000:2.0:1789760771.018829:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714234114959 time 714249413310 state 0x0 ref 1 00000100:00100000:2.0:1789760771.018846:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714249673726 time 714249413310 state 0x1 ref 1 00000100:00100000:2.0:1789760771.018944:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714249673726 time 714264561546 state 0x0 ref 1 00000100:00100000:2.0:1789760771.018959:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554690944:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760771.019017:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61636608, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d658 00010000:00010000:2.0:1789760771.019027:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c45ae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.019042:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760771.019047:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.019053:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c45ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61636608->61640703] (req 61636608->61640703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.019065:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c45ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61636608->61640703] (req 61636608->61640703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.024727:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982357 is committed 00002000:00010000:2.0:1789760771.024966:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61636608 -> 61640704 00002000:00010000:2.0:1789760771.024972:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120408 -> 120416 00010000:00010000:2.0:1789760771.025058:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c45ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61636608->61640703] (req 61636608->61640703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.025071:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c45ae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61636608->61640703] (req 61636608->61640703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.025080:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c45ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61636608->61640703] (req 61636608->61640703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.025088:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760771.025093:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c45ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61636608->61640703] (req 61636608->61640703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.025102:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c45ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61636608->61640703] (req 61636608->61640703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.025123:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c45ae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61636608->61640703] (req 61636608->61640703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.025131:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c45ae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61636608->61640703] (req 61636608->61640703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.025340:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554690944:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6378us (8608us total) trans 4294982357 rc 0/0 00000100:00100000:2.0:1789760771.025360:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15059 00000100:00100000:2.0:1789760771.025371:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714249673726 time 714264561546 state 0x2 ref 0 00000100:00100000:0.0:1789760771.032144:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760771.039640:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982358 is committed 00000100:00100000:0.0:1789760771.044590:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760771.044871:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554691200 00000100:00100000:2.0:1789760771.044881:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554691200 00000100:00100000:2.0:1789760771.044938:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714264821962 time 714278066790 state 0x0 ref 1 00000100:00100000:2.0:1789760771.044954:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714278327206 time 714278066790 state 0x1 ref 1 00000100:00100000:2.0:1789760771.044994:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714278327206 time 714290612131 state 0x0 ref 1 00000100:00100000:2.0:1789760771.045006:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554691200:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760771.045053:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61644800, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d668 00010000:00010000:2.0:1789760771.045063:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c45bc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.045080:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760771.045085:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.045092:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c45bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61644800->61648895] (req 61644800->61648895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.045106:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c45bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61644800->61648895] (req 61644800->61648895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.052694:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982359 is committed 00000100:00100000:3.0:1789760771.071518:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760771.071605:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554691328 00000100:00100000:2.0:1789760771.071617:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554691328 00000100:00100000:2.0:1789760771.071673:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714278327206 time 714290612131 state 0x0 ref 1 00000100:00100000:2.0:1789760771.071689:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714290872547 time 714290612131 state 0x1 ref 1 00000100:00100000:2.0:1789760771.071721:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714290872547 time 714317339545 state 0x0 ref 1 00000100:00100000:2.0:1789760771.071733:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554691328:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760771.071792:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61648896, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d670 00010000:00010000:2.0:1789760771.071811:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c45c3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.071827:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760771.071832:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.071837:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c45c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61648896->61652991] (req 61648896->61652991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.071850:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c45c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61648896->61652991] (req 61648896->61652991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.081657:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982360 is committed 00002000:00010000:0.0:1789760771.085422:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61648896 -> 61652992 00002000:00010000:0.0:1789760771.085431:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120432 -> 120440 00010000:00010000:0.0:1789760771.085445:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c45c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61648896->61652991] (req 61648896->61652991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.085459:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c45c3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61648896->61652991] (req 61648896->61652991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.085469:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c45c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61648896->61652991] (req 61648896->61652991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.085477:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760771.085482:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c45c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61648896->61652991] (req 61648896->61652991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.085491:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c45c3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61648896->61652991] (req 61648896->61652991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.085509:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c45c3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61648896->61652991] (req 61648896->61652991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.085517:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c45c3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61648896->61652991] (req 61648896->61652991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.085611:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554691328:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13876us (14099us total) trans 4294982360 rc 0/0 00000100:00100000:0.0:1789760771.085622:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15062 00000100:00100000:0.0:1789760771.085631:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714290872547 time 714317339545 state 0x2 ref 0 00000100:00100000:0.0:1789760771.095448:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760771.096014:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554691456 00000100:00100000:3.0:1789760771.096025:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554691456 00000100:00100000:3.0:1789760771.096081:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714290872547 time 714317339545 state 0x0 ref 1 00000100:00100000:3.0:1789760771.096098:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714317599961 time 714317339545 state 0x1 ref 1 00000100:00100000:3.0:1789760771.096180:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714317599961 time 714341797933 state 0x0 ref 1 00000100:00100000:3.0:1789760771.096197:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554691456:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760771.096245:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61652992, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d678 00010000:00010000:3.0:1789760771.096254:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c45ca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.096271:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760771.096275:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.096281:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c45ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61652992->61657087] (req 61652992->61657087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.096295:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c45ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61652992->61657087] (req 61652992->61657087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.108214:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982361 is committed 00002000:00010000:3.0:1789760771.109132:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61652992 -> 61657088 00002000:00010000:3.0:1789760771.109153:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120440 -> 120448 00010000:00010000:3.0:1789760771.109170:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c45ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61652992->61657087] (req 61652992->61657087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.109184:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c45ca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61652992->61657087] (req 61652992->61657087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.109195:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c45ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61652992->61657087] (req 61652992->61657087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.109203:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760771.109209:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c45ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61652992->61657087] (req 61652992->61657087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.109220:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c45ca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61652992->61657087] (req 61652992->61657087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.109242:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c45ca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61652992->61657087] (req 61652992->61657087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.109251:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c45ca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61652992->61657087] (req 61652992->61657087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.109661:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554691456:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13459us (14216us total) trans 4294982361 rc 0/0 00000100:00100000:3.0:1789760771.109674:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15063 00000100:00100000:3.0:1789760771.109685:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714317599961 time 714341797933 state 0x2 ref 0 00000100:00100000:3.0:1789760771.117150:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760771.124345:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982362 is committed 00000100:00100000:3.0:1789760771.132777:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760771.141375:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982363 is committed 00000100:00100000:0.0:1789760771.147787:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760771.147878:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554691840 00000100:00100000:2.0:1789760771.147888:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554691840 00000100:00100000:2.0:1789760771.147939:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714363234518 time 714378601973 state 0x0 ref 1 00000100:00100000:2.0:1789760771.147954:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714378862389 time 714378601973 state 0x1 ref 1 00000100:00100000:2.0:1789760771.147986:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714378862389 time 714393604997 state 0x0 ref 1 00000100:00100000:2.0:1789760771.147998:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554691840:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760771.148045:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61665280, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d690 00010000:00010000:2.0:1789760771.148054:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c45df lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.148069:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760771.148074:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.148080:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c45df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61665280->61669375] (req 61665280->61669375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.148092:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c45df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61665280->61669375] (req 61665280->61669375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.156591:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982364 is committed 00002000:00010000:2.0:1789760771.158543:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61665280 -> 61669376 00002000:00010000:2.0:1789760771.158552:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120464 -> 120472 00010000:00010000:2.0:1789760771.158565:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c45df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61665280->61669375] (req 61665280->61669375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.158578:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c45df lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61665280->61669375] (req 61665280->61669375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.158587:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c45df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61665280->61669375] (req 61665280->61669375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.158594:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760771.158599:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c45df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61665280->61669375] (req 61665280->61669375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.158607:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c45df lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61665280->61669375] (req 61665280->61669375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.158625:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c45df lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61665280->61669375] (req 61665280->61669375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.158633:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c45df lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61665280->61669375] (req 61665280->61669375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.158847:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554691840:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10844us (11060us total) trans 4294982364 rc 0/0 00000100:00100000:2.0:1789760771.158859:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15066 00000100:00100000:2.0:1789760771.158869:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714378862389 time 714393604997 state 0x2 ref 0 00000100:00100000:3.0:1789760771.178741:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760771.181007:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554691968 00000100:00100000:2.0:1789760771.181024:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554691968 00000100:00100000:2.0:1789760771.181092:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714378862389 time 714393604997 state 0x0 ref 1 00000100:00100000:2.0:1789760771.181111:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714393865413 time 714393604997 state 0x1 ref 1 00000100:00100000:2.0:1789760771.181150:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714393865413 time 714426767783 state 0x0 ref 1 00000100:00100000:2.0:1789760771.181164:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554691968:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760771.181214:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61669376, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d698 00010000:00010000:2.0:1789760771.181224:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c45e6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.181240:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760771.181244:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.181251:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c45e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61669376->61673471] (req 61669376->61673471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.181566:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c45e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61669376->61673471] (req 61669376->61673471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.192553:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982365 is committed 00002000:00010000:3.0:1789760771.193117:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61669376 -> 61673472 00002000:00010000:3.0:1789760771.193123:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120472 -> 120480 00010000:00010000:3.0:1789760771.193136:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c45e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61669376->61673471] (req 61669376->61673471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.193158:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c45e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61669376->61673471] (req 61669376->61673471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.193168:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c45e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61669376->61673471] (req 61669376->61673471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.193176:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760771.193180:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c45e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61669376->61673471] (req 61669376->61673471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.193189:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c45e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61669376->61673471] (req 61669376->61673471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.193217:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c45e6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61669376->61673471] (req 61669376->61673471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.193226:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c45e6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61669376->61673471] (req 61669376->61673471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.193427:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554691968:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12261us (14691us total) trans 4294982365 rc 0/0 00000100:00100000:3.0:1789760771.193440:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15067 00000100:00100000:3.0:1789760771.193449:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714393865413 time 714426767783 state 0x2 ref 0 00000100:00100000:3.0:1789760771.199864:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760771.208620:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982366 is committed 00000100:00100000:0.0:1789760771.223507:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:2.0:1789760771.238747:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61677568 -> 61681664 00002000:00010000:2.0:1789760771.238757:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120488 -> 120496 00010000:00010000:2.0:1789760771.238781:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c45f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61677568->61681663] (req 61677568->61681663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.239532:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c45f4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61677568->61681663] (req 61677568->61681663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.239543:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c45f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61677568->61681663] (req 61677568->61681663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.239550:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760771.239555:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c45f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61677568->61681663] (req 61677568->61681663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.239564:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c45f4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61677568->61681663] (req 61677568->61681663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.239584:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c45f4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61677568->61681663] (req 61677568->61681663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.239592:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c45f4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61677568->61681663] (req 61677568->61681663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.239700:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554692224:12345-192.168.203.22@tcp:4:dd.0 Request processed in 15993us (16195us total) trans 4294982367 rc 0/0 00000100:00100000:2.0:1789760771.239711:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15069 00000100:00100000:2.0:1789760771.239720:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714446883608 time 714469310823 state 0x2 ref 0 00000100:00100000:0.0:1789760771.245056:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760771.245128:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554692352 00000100:00100000:2.0:1789760771.245138:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554692352 00000100:00100000:2.0:1789760771.245186:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714446883608 time 714469310823 state 0x0 ref 1 00000100:00100000:2.0:1789760771.245200:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714469571239 time 714469310823 state 0x1 ref 1 00000100:00100000:2.0:1789760771.245271:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714469571239 time 714490889161 state 0x0 ref 1 00000100:00100000:2.0:1789760771.245284:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554692352:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760771.245330:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61681664, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d6b0 00010000:00010000:2.0:1789760771.245339:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c45fb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.245353:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760771.245358:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.245364:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c45fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61681664->61685759] (req 61681664->61685759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.245405:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c45fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61681664->61685759] (req 61681664->61685759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760771.251888:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61681664 -> 61685760 00002000:00010000:2.0:1789760771.251893:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120496 -> 120504 00010000:00010000:2.0:1789760771.251906:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c45fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61681664->61685759] (req 61681664->61685759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.251919:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c45fb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61681664->61685759] (req 61681664->61685759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.251934:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c45fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61681664->61685759] (req 61681664->61685759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.251942:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760771.251946:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c45fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61681664->61685759] (req 61681664->61685759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.251955:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c45fb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61681664->61685759] (req 61681664->61685759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.251973:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c45fb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61681664->61685759] (req 61681664->61685759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.251980:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c45fb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61681664->61685759] (req 61681664->61685759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.252069:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554692352:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6783us (7017us total) trans 4294982368 rc 0/0 00000100:00100000:2.0:1789760771.252080:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15070 00000100:00100000:2.0:1789760771.252089:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714469571239 time 714490889161 state 0x2 ref 0 00000100:00100000:0.0:1789760771.261849:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760771.263131:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554692480 00000100:00100000:3.0:1789760771.263143:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554692480 00000100:00100000:3.0:1789760771.263209:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714469571239 time 714490889161 state 0x0 ref 1 00000100:00100000:3.0:1789760771.263227:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714491149577 time 714490889161 state 0x1 ref 1 00000100:00100000:3.0:1789760771.263262:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714491149577 time 714508880156 state 0x0 ref 1 00000100:00100000:3.0:1789760771.263275:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554692480:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760771.263319:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61685760, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d6b8 00010000:00010000:3.0:1789760771.263328:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4602 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.263345:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760771.263349:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.263355:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4602 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61685760->61689855] (req 61685760->61689855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.263408:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4602 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61685760->61689855] (req 61685760->61689855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760771.271629:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61685760 -> 61689856 00002000:00010000:0.0:1789760771.271635:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120504 -> 120512 00010000:00010000:0.0:1789760771.271648:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4602 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61685760->61689855] (req 61685760->61689855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.271662:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4602 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61685760->61689855] (req 61685760->61689855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.271671:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4602 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61685760->61689855] (req 61685760->61689855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.271678:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760771.271683:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4602 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61685760->61689855] (req 61685760->61689855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.271693:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4602 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61685760->61689855] (req 61685760->61689855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.271713:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4602 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61685760->61689855] (req 61685760->61689855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.271721:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4602 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61685760->61689855] (req 61685760->61689855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.272620:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554692480:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9338us (10771us total) trans 4294982369 rc 0/0 00000100:00100000:0.0:1789760771.272633:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15071 00000100:00100000:0.0:1789760771.272642:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714491149577 time 714508880156 state 0x2 ref 0 00000100:00100000:0.0:1789760771.275834:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760771.281684:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982370 is committed 00000100:00100000:2.0:1789760771.286806:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554692736 00000100:00100000:2.0:1789760771.286816:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554692736 00000100:00100000:2.0:1789760771.286871:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714509140572 time 714521792875 state 0x0 ref 1 00000100:00100000:2.0:1789760771.286889:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714522053291 time 714521792875 state 0x1 ref 1 00000100:00100000:2.0:1789760771.286926:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714522053291 time 714532545457 state 0x0 ref 1 00000100:00100000:2.0:1789760771.286937:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554692736:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760771.286978:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61693952, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d6c8 00010000:00010000:2.0:1789760771.286985:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4610 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.286998:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760771.287001:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.287006:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4610 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61693952->61698047] (req 61693952->61698047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.287017:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4610 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61693952->61698047] (req 61693952->61698047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760771.295074:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61693952 -> 61698048 00002000:00010000:2.0:1789760771.295079:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120520 -> 120528 00010000:00010000:2.0:1789760771.295092:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4610 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61693952->61698047] (req 61693952->61698047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.295105:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4610 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61693952->61698047] (req 61693952->61698047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.295114:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4610 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61693952->61698047] (req 61693952->61698047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.295123:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760771.295127:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4610 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61693952->61698047] (req 61693952->61698047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.295135:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4610 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61693952->61698047] (req 61693952->61698047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.295754:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4610 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61693952->61698047] (req 61693952->61698047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.295775:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4610 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61693952->61698047] (req 61693952->61698047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.295957:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554692736:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9014us (9319us total) trans 4294982371 rc 0/0 00000100:00100000:0.0:1789760771.295968:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15073 00000100:00100000:0.0:1789760771.295977:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714522053291 time 714532545457 state 0x2 ref 0 00000100:00100000:0.0:1789760771.301794:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760771.301880:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554692864 00000100:00100000:2.0:1789760771.301889:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554692864 00000100:00100000:2.0:1789760771.301945:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714522053291 time 714532545457 state 0x0 ref 1 00000100:00100000:2.0:1789760771.301959:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714532805873 time 714532545457 state 0x1 ref 1 00000100:00100000:2.0:1789760771.301991:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714532805873 time 714547610409 state 0x0 ref 1 00000100:00100000:2.0:1789760771.302001:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554692864:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760771.302045:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61698048, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d6d0 00010000:00010000:2.0:1789760771.302053:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4617 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.302071:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760771.302076:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.302082:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4617 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61698048->61702143] (req 61698048->61702143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.302094:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4617 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61698048->61702143] (req 61698048->61702143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.307103:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982372 is committed 00002000:00010000:2.0:1789760771.307262:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61698048 -> 61702144 00002000:00010000:2.0:1789760771.307267:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120528 -> 120536 00010000:00010000:2.0:1789760771.307281:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4617 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61698048->61702143] (req 61698048->61702143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.307295:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4617 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61698048->61702143] (req 61698048->61702143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.307304:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4617 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61698048->61702143] (req 61698048->61702143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.307313:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760771.307317:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4617 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61698048->61702143] (req 61698048->61702143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.307326:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4617 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61698048->61702143] (req 61698048->61702143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.307346:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4617 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61698048->61702143] (req 61698048->61702143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.307351:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4617 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61698048->61702143] (req 61698048->61702143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.307605:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554692864:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5598us (5814us total) trans 4294982372 rc 0/0 00000100:00100000:2.0:1789760771.307615:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15074 00000100:00100000:2.0:1789760771.307623:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714532805873 time 714547610409 state 0x2 ref 0 00000100:00100000:0.0:1789760771.314142:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760771.314374:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554692992 00000100:00100000:2.0:1789760771.314420:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554692992 00000100:00100000:2.0:1789760771.314679:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714532805873 time 714547610409 state 0x0 ref 1 00000100:00100000:2.0:1789760771.314694:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714547870825 time 714547610409 state 0x1 ref 1 00000100:00100000:0.0:1789760771.314861:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714547870825 time 714560476511 state 0x0 ref 1 00000100:00100000:0.0:1789760771.314878:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554692992:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760771.314992:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61702144, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d6d8 00010000:00010000:0.0:1789760771.315002:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c461e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.315018:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760771.315023:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.315029:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c461e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61702144->61706239] (req 61702144->61706239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.315044:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c461e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61702144->61706239] (req 61702144->61706239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.323039:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982373 is committed 00000100:00100000:3.0:1789760771.334158:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554693120 00000100:00100000:3.0:1789760771.334169:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554693120 00000100:00100000:3.0:1789760771.334230:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714547870825 time 714560476511 state 0x0 ref 1 00000100:00100000:3.0:1789760771.334244:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714560736927 time 714560476511 state 0x1 ref 1 00000100:00100000:3.0:1789760771.334276:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714560736927 time 714579894635 state 0x0 ref 1 00000100:00100000:3.0:1789760771.334288:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554693120:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760771.334336:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61706240, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d6e0 00010000:00010000:3.0:1789760771.334344:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4625 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.334369:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760771.334373:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.334419:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4625 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61706240->61710335] (req 61706240->61710335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.334491:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4625 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61706240->61710335] (req 61706240->61710335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.340509:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982374 is committed 00002000:00010000:3.0:1789760771.340651:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61706240 -> 61710336 00002000:00010000:3.0:1789760771.340658:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120544 -> 120552 00010000:00010000:3.0:1789760771.340673:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4625 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61706240->61710335] (req 61706240->61710335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.340688:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4625 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61706240->61710335] (req 61706240->61710335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.340697:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4625 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61706240->61710335] (req 61706240->61710335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.340704:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760771.340710:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4625 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61706240->61710335] (req 61706240->61710335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.340725:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4625 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61706240->61710335] (req 61706240->61710335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.340749:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4625 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61706240->61710335] (req 61706240->61710335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.340756:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4625 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61706240->61710335] (req 61706240->61710335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.340870:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554693120:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6580us (6837us total) trans 4294982374 rc 0/0 00000100:00100000:3.0:1789760771.340882:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15076 00000100:00100000:3.0:1789760771.340891:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714560736927 time 714579894635 state 0x2 ref 0 00000100:00100000:0.0:1789760771.345797:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:0.0:1789760771.355072:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982375 is committed 00000100:00100000:0.0:1789760771.363622:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760771.363716:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554693376 00000100:00100000:2.0:1789760771.363726:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554693376 00000100:00100000:2.0:1789760771.363793:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714580155051 time 714591734644 state 0x0 ref 1 00000100:00100000:2.0:1789760771.363808:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714591995060 time 714591734644 state 0x1 ref 1 00000100:00100000:2.0:1789760771.363836:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714591995060 time 714609454322 state 0x0 ref 1 00000100:00100000:2.0:1789760771.363857:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554693376:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760771.363912:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61714432, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d6f0 00010000:00010000:2.0:1789760771.363921:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4633 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.363937:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760771.363942:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.363949:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4633 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61714432->61718527] (req 61714432->61718527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.363963:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4633 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61714432->61718527] (req 61714432->61718527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760771.370171:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61714432 -> 61718528 00002000:00010000:2.0:1789760771.370176:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120560 -> 120568 00010000:00010000:2.0:1789760771.370187:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4633 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61714432->61718527] (req 61714432->61718527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.370199:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4633 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61714432->61718527] (req 61714432->61718527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.370208:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4633 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61714432->61718527] (req 61714432->61718527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.370216:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760771.370221:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4633 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61714432->61718527] (req 61714432->61718527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.370234:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4633 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61714432->61718527] (req 61714432->61718527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.370251:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4633 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61714432->61718527] (req 61714432->61718527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.370258:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4633 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61714432->61718527] (req 61714432->61718527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.370350:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554693376:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6492us (6734us total) trans 4294982376 rc 0/0 00000100:00100000:2.0:1789760771.370360:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15078 00000100:00100000:2.0:1789760771.370368:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714591995060 time 714609454322 state 0x2 ref 0 00000100:00100000:0.0:1789760771.376048:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554693504 00000100:00100000:0.0:1789760771.376058:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554693504 00000100:00100000:0.0:1789760771.376109:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714591995060 time 714609454322 state 0x0 ref 1 00000100:00100000:0.0:1789760771.376124:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714609714738 time 714609454322 state 0x1 ref 1 00000100:00100000:0.0:1789760771.376162:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714609714738 time 714621781083 state 0x0 ref 1 00000100:00100000:0.0:1789760771.376175:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554693504:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760771.376218:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61718528, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d6f8 00010000:00010000:0.0:1789760771.376228:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c463a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.376244:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760771.376248:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.376254:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c463a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61718528->61722623] (req 61718528->61722623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.376268:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c463a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61718528->61722623] (req 61718528->61722623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760771.382548:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61718528 -> 61722624 00002000:00010000:0.0:1789760771.382554:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120568 -> 120576 00010000:00010000:0.0:1789760771.382566:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c463a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61718528->61722623] (req 61718528->61722623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.382578:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c463a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61718528->61722623] (req 61718528->61722623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.382588:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c463a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61718528->61722623] (req 61718528->61722623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.382595:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760771.382600:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c463a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61718528->61722623] (req 61718528->61722623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.382608:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c463a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61718528->61722623] (req 61718528->61722623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.382629:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c463a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61718528->61722623] (req 61718528->61722623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.382637:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c463a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61718528->61722623] (req 61718528->61722623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.382730:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554693504:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6553us (6922us total) trans 4294982377 rc 0/0 00000100:00100000:0.0:1789760771.382740:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15079 00000100:00100000:0.0:1789760771.382749:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714609714738 time 714621781083 state 0x2 ref 0 00000100:00100000:0.0:1789760771.388196:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760771.388289:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554693632 00000100:00100000:3.0:1789760771.388299:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554693632 00000100:00100000:3.0:1789760771.388351:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714609714738 time 714621781083 state 0x0 ref 1 00000100:00100000:3.0:1789760771.388366:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714622041499 time 714621781083 state 0x1 ref 1 00000100:00100000:3.0:1789760771.388495:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714622041499 time 714634113297 state 0x0 ref 1 00000100:00100000:3.0:1789760771.388508:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554693632:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760771.388678:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61722624, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d700 00010000:00010000:3.0:1789760771.388688:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4641 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.388704:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760771.388708:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.388714:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4641 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61722624->61726719] (req 61722624->61726719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.388728:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4641 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61722624->61726719] (req 61722624->61726719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.395818:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982378 is committed 00000100:00100000:2.0:1789760771.397000:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554693632:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8487us (8864us total) trans 4294982378 rc 0/0 00000100:00100000:2.0:1789760771.397012:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15080 00000100:00100000:2.0:1789760771.397022:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714622041499 time 714634113297 state 0x2 ref 0 00000100:00100000:2.0:1789760771.401724:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760771.410054:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982379 is committed 00000100:00100000:2.0:1789760771.411218:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554693760:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9267us (9487us total) trans 4294982379 rc 0/0 00000100:00100000:2.0:1789760771.411231:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15081 00000100:00100000:2.0:1789760771.411241:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714634373713 time 714647543718 state 0x2 ref 0 00000100:00100000:2.0:1789760771.419332:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554693888 00000100:00100000:2.0:1789760771.419343:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554693888 00000100:00100000:2.0:1789760771.420942:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714634373713 time 714647543718 state 0x0 ref 1 00000100:00100000:2.0:1789760771.420962:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714647804134 time 714647543718 state 0x1 ref 1 00000100:00100000:2.0:1789760771.421004:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714647804134 time 714666621796 state 0x0 ref 1 00000100:00100000:2.0:1789760771.421022:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554693888:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760771.421068:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61730816, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d710 00010000:00010000:2.0:1789760771.421077:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c464f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.421092:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760771.421095:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.421101:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c464f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61730816->61734911] (req 61730816->61734911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.421114:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c464f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61730816->61734911] (req 61730816->61734911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.428249:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982380 is committed 00002000:00010000:0.0:1789760771.435100:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61730816 -> 61734912 00002000:00010000:0.0:1789760771.435110:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120592 -> 120600 00010000:00010000:0.0:1789760771.435124:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c464f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61730816->61734911] (req 61730816->61734911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.435196:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c464f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61730816->61734911] (req 61730816->61734911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.435208:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c464f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61730816->61734911] (req 61730816->61734911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.435216:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760771.435222:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c464f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61730816->61734911] (req 61730816->61734911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.435232:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c464f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61730816->61734911] (req 61730816->61734911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.435254:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c464f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61730816->61734911] (req 61730816->61734911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.435263:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c464f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61730816->61734911] (req 61730816->61734911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.435450:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554693888:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14430us (16656us total) trans 4294982380 rc 0/0 00000100:00100000:0.0:1789760771.435464:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15082 00000100:00100000:0.0:1789760771.435474:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714647804134 time 714666621796 state 0x2 ref 0 00000100:00100000:0.0:1789760771.443478:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760771.449553:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982381 is committed 00000100:00100000:2.0:1789760771.453644:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760771.453726:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554694144 00000100:00100000:3.0:1789760771.453735:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554694144 00000100:00100000:3.0:1789760771.453796:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714666882212 time 714689538070 state 0x0 ref 1 00000100:00100000:3.0:1789760771.453812:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714689798486 time 714689538070 state 0x1 ref 1 00000100:00100000:3.0:1789760771.453849:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714689798486 time 714699467184 state 0x0 ref 1 00000100:00100000:3.0:1789760771.453862:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554694144:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760771.453905:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61739008, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d720 00010000:00010000:3.0:1789760771.453915:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c465d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.453930:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760771.453935:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.453941:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c465d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61739008->61743103] (req 61739008->61743103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.453960:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c465d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61739008->61743103] (req 61739008->61743103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.458878:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982382 is committed 00002000:00010000:2.0:1789760771.458995:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61739008 -> 61743104 00002000:00010000:2.0:1789760771.458999:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120608 -> 120616 00010000:00010000:2.0:1789760771.459013:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c465d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61739008->61743103] (req 61739008->61743103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.459026:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c465d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61739008->61743103] (req 61739008->61743103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.459035:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c465d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61739008->61743103] (req 61739008->61743103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.459041:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760771.459046:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c465d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61739008->61743103] (req 61739008->61743103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.459054:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c465d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61739008->61743103] (req 61739008->61743103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.459072:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c465d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61739008->61743103] (req 61739008->61743103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.459080:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c465d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61739008->61743103] (req 61739008->61743103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.459268:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554694144:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5403us (5628us total) trans 4294982382 rc 0/0 00000100:00100000:2.0:1789760771.459278:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15084 00000100:00100000:2.0:1789760771.459288:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714689798486 time 714699467184 state 0x2 ref 0 00000100:00100000:2.0:1789760771.467218:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760771.467305:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554694272 00000100:00100000:0.0:1789760771.467319:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554694272 00000100:00100000:0.0:1789760771.467414:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714689798486 time 714699467184 state 0x0 ref 1 00000100:00100000:0.0:1789760771.467431:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714699727600 time 714699467184 state 0x1 ref 1 00000100:00100000:0.0:1789760771.467453:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714699727600 time 714713072315 state 0x0 ref 1 00000100:00100000:0.0:1789760771.467466:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554694272:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760771.467510:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61743104, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d728 00010000:00010000:0.0:1789760771.467525:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4664 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.467538:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760771.467541:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.467545:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4664 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61743104->61747199] (req 61743104->61747199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.467556:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4664 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61743104->61747199] (req 61743104->61747199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.473282:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982383 is committed 00002000:00010000:0.0:1789760771.473487:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61743104 -> 61747200 00002000:00010000:0.0:1789760771.473493:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120616 -> 120624 00010000:00010000:0.0:1789760771.473509:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4664 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61743104->61747199] (req 61743104->61747199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.473534:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4664 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61743104->61747199] (req 61743104->61747199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.473543:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4664 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61743104->61747199] (req 61743104->61747199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.473551:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760771.473556:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4664 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61743104->61747199] (req 61743104->61747199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.473565:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4664 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61743104->61747199] (req 61743104->61747199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.473584:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4664 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61743104->61747199] (req 61743104->61747199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.473591:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4664 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61743104->61747199] (req 61743104->61747199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.473688:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554694272:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6220us (6475us total) trans 4294982383 rc 0/0 00000100:00100000:0.0:1789760771.473700:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15085 00000100:00100000:0.0:1789760771.473710:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714699727600 time 714713072315 state 0x2 ref 0 00000100:00100000:0.0:1789760771.479368:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554694400 00000100:00100000:0.0:1789760771.479511:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554694400 00000100:00100000:0.0:1789760771.479574:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714699727600 time 714713072315 state 0x0 ref 1 00000100:00100000:0.0:1789760771.479590:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714713332731 time 714713072315 state 0x1 ref 1 00000100:00100000:0.0:1789760771.479717:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714713332731 time 714725334230 state 0x0 ref 1 00000100:00100000:0.0:1789760771.479732:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554694400:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760771.479793:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61747200, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d730 00010000:00010000:0.0:1789760771.479802:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c466b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.479818:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760771.479822:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.479828:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c466b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61747200->61751295] (req 61747200->61751295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.479842:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c466b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61747200->61751295] (req 61747200->61751295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.484061:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982384 is committed 00002000:00010000:0.0:1789760771.484176:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61747200 -> 61751296 00002000:00010000:0.0:1789760771.484182:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120624 -> 120632 00010000:00010000:0.0:1789760771.484194:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c466b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61747200->61751295] (req 61747200->61751295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.484223:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c466b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61747200->61751295] (req 61747200->61751295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.484232:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c466b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61747200->61751295] (req 61747200->61751295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.484239:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760771.484244:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c466b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61747200->61751295] (req 61747200->61751295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.484253:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c466b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61747200->61751295] (req 61747200->61751295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.484273:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c466b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61747200->61751295] (req 61747200->61751295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.484280:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c466b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61747200->61751295] (req 61747200->61751295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.484369:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554694400:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4636us (6372us total) trans 4294982384 rc 0/0 00000100:00100000:0.0:1789760771.484426:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15086 00000100:00100000:0.0:1789760771.484436:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714713332731 time 714725334230 state 0x2 ref 0 00000100:00100000:3.0:1789760771.489793:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760771.495922:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982385 is committed 00000100:00100000:2.0:1789760771.501432:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760771.501511:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554694656 00000100:00100000:3.0:1789760771.501528:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554694656 00000100:00100000:3.0:1789760771.501583:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714725594646 time 714735709484 state 0x0 ref 1 00000100:00100000:3.0:1789760771.501597:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714735969900 time 714735709484 state 0x1 ref 1 00000100:00100000:3.0:1789760771.501629:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714735969900 time 714747246788 state 0x0 ref 1 00000100:00100000:3.0:1789760771.501641:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554694656:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760771.501689:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61755392, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d740 00010000:00010000:3.0:1789760771.501699:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4679 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.501715:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:3.0:1789760771.501720:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.501727:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4679 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61755392->61759487] (req 61755392->61759487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.501741:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4679 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61755392->61759487] (req 61755392->61759487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.505809:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982386 is committed 00002000:00010000:2.0:1789760771.512438:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61755392 -> 61759488 00002000:00010000:2.0:1789760771.512447:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120640 -> 120648 00010000:00010000:2.0:1789760771.512460:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4679 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61755392->61759487] (req 61755392->61759487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.512471:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4679 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61755392->61759487] (req 61755392->61759487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.512479:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4679 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61755392->61759487] (req 61755392->61759487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.512486:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760771.512490:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4679 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61755392->61759487] (req 61755392->61759487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.512499:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4679 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61755392->61759487] (req 61755392->61759487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.512575:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4679 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61755392->61759487] (req 61755392->61759487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.512585:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4679 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61755392->61759487] (req 61755392->61759487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.512685:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554694656:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11041us (11268us total) trans 4294982386 rc 0/0 00000100:00100000:2.0:1789760771.512697:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15088 00000100:00100000:2.0:1789760771.512707:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714735969900 time 714747246788 state 0x2 ref 0 00000100:00100000:3.0:1789760771.517426:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760771.517897:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554694784 00000100:00100000:0.0:1789760771.517910:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554694784 00000100:00100000:0.0:1789760771.517967:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714735969900 time 714747246788 state 0x0 ref 1 00000100:00100000:0.0:1789760771.517983:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714747507204 time 714747246788 state 0x1 ref 1 00000100:00100000:0.0:1789760771.518019:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714747507204 time 714763637636 state 0x0 ref 1 00000100:00100000:0.0:1789760771.518037:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554694784:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760771.518086:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61759488, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d748 00010000:00010000:0.0:1789760771.518095:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4680 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.518113:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760771.518118:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.518125:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4680 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61759488->61763583] (req 61759488->61763583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.518150:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4680 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61759488->61763583] (req 61759488->61763583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.525625:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982387 is committed 00002000:00010000:0.0:1789760771.525724:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61759488 -> 61763584 00002000:00010000:0.0:1789760771.525729:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120648 -> 120656 00010000:00010000:0.0:1789760771.525742:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4680 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61759488->61763583] (req 61759488->61763583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.525753:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4680 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61759488->61763583] (req 61759488->61763583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.525779:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4680 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61759488->61763583] (req 61759488->61763583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.525786:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760771.525790:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4680 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61759488->61763583] (req 61759488->61763583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.525799:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4680 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61759488->61763583] (req 61759488->61763583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.525819:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4680 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61759488->61763583] (req 61759488->61763583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.525828:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4680 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61759488->61763583] (req 61759488->61763583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.525936:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554694784:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7897us (8559us total) trans 4294982387 rc 0/0 00000100:00100000:0.0:1789760771.525946:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15089 00000100:00100000:0.0:1789760771.525955:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714747507204 time 714763637636 state 0x2 ref 0 00000100:00100000:3.0:1789760771.529109:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760771.529183:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554694912 00000100:00100000:0.0:1789760771.529193:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554694912 00000100:00100000:0.0:1789760771.529256:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714747507204 time 714763637636 state 0x0 ref 1 00000100:00100000:0.0:1789760771.529272:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714763898052 time 714763637636 state 0x1 ref 1 00000100:00100000:0.0:1789760771.529305:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714763898052 time 714774923826 state 0x0 ref 1 00000100:00100000:0.0:1789760771.529317:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554694912:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760771.529369:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61763584, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d750 00010000:00010000:0.0:1789760771.529510:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4687 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.529582:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760771.529588:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.529595:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4687 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61763584->61767679] (req 61763584->61767679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.529610:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4687 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61763584->61767679] (req 61763584->61767679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.535807:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982388 is committed 00000100:00100000:2.0:1789760771.539456:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760771.544072:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982389 is committed 00000100:00100000:3.0:1789760771.552269:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760771.552408:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554695168 00000100:00100000:2.0:1789760771.552418:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554695168 00000100:00100000:2.0:1789760771.552474:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714775184242 time 714785269468 state 0x0 ref 1 00000100:00100000:2.0:1789760771.552490:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714785529884 time 714785269468 state 0x1 ref 1 00000100:00100000:2.0:1789760771.552532:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714785529884 time 714798150885 state 0x0 ref 1 00000100:00100000:2.0:1789760771.552546:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554695168:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760771.552591:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61771776, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d760 00010000:00010000:2.0:1789760771.552601:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4695 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.552616:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760771.552621:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.552627:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4695 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61771776->61775871] (req 61771776->61775871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.552647:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4695 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61771776->61775871] (req 61771776->61775871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760771.567345:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61771776 -> 61775872 00002000:00010000:2.0:1789760771.567354:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120672 -> 120680 00010000:00010000:2.0:1789760771.567368:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4695 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61771776->61775871] (req 61771776->61775871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.567395:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4695 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61771776->61775871] (req 61771776->61775871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.568081:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4695 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61771776->61775871] (req 61771776->61775871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.568092:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760771.568097:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4695 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61771776->61775871] (req 61771776->61775871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.568107:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4695 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61771776->61775871] (req 61771776->61775871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.568129:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4695 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61771776->61775871] (req 61771776->61775871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.568138:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4695 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61771776->61775871] (req 61771776->61775871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.568526:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554695168:12345-192.168.203.22@tcp:4:dd.0 Request processed in 15976us (16261us total) trans 4294982390 rc 0/0 00000100:00100000:2.0:1789760771.568539:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15092 00000100:00100000:2.0:1789760771.568548:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714785529884 time 714798150885 state 0x2 ref 0 00000100:00100000:3.0:1789760771.579164:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760771.579261:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554695296 00000100:00100000:0.0:1789760771.579270:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554695296 00000100:00100000:0.0:1789760771.579318:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714785529884 time 714798150885 state 0x0 ref 1 00000100:00100000:0.0:1789760771.579331:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714798411301 time 714798150885 state 0x1 ref 1 00000100:00100000:0.0:1789760771.579360:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714798411301 time 714824979169 state 0x0 ref 1 00000100:00100000:0.0:1789760771.579371:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554695296:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760771.579461:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61775872, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d768 00010000:00010000:0.0:1789760771.579470:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c469c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.579486:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760771.579490:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.579496:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c469c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61775872->61779967] (req 61775872->61779967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.579510:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c469c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61775872->61779967] (req 61775872->61779967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.584399:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982391 is committed 00002000:00010000:2.0:1789760771.584684:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61775872 -> 61779968 00002000:00010000:2.0:1789760771.584689:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120680 -> 120688 00010000:00010000:2.0:1789760771.584703:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c469c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61775872->61779967] (req 61775872->61779967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.584715:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c469c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61775872->61779967] (req 61775872->61779967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.584725:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c469c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61775872->61779967] (req 61775872->61779967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.584733:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760771.584738:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c469c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61775872->61779967] (req 61775872->61779967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.584748:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c469c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61775872->61779967] (req 61775872->61779967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.584778:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c469c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61775872->61779967] (req 61775872->61779967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.584792:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c469c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61775872->61779967] (req 61775872->61779967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.584942:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554695296:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5566us (5780us total) trans 4294982391 rc 0/0 00000100:00100000:2.0:1789760771.584953:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15093 00000100:00100000:2.0:1789760771.584963:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714798411301 time 714824979169 state 0x2 ref 0 00000100:00100000:2.0:1789760771.589352:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554695424 00000100:00100000:2.0:1789760771.589362:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554695424 00000100:00100000:2.0:1789760771.592313:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714798411301 time 714824979169 state 0x0 ref 1 00000100:00100000:2.0:1789760771.592332:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714825239585 time 714824979169 state 0x1 ref 1 00000100:00100000:2.0:1789760771.592450:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714825239585 time 714838066605 state 0x0 ref 1 00000100:00100000:2.0:1789760771.592467:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554695424:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760771.592526:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61779968, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d770 00010000:00010000:2.0:1789760771.592537:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c46a3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.592553:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760771.592559:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.592565:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c46a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61779968->61784063] (req 61779968->61784063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.592585:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c46a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61779968->61784063] (req 61779968->61784063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.597680:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982392 is committed 00002000:00010000:2.0:1789760771.597890:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61779968 -> 61784064 00002000:00010000:2.0:1789760771.597896:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120688 -> 120696 00010000:00010000:2.0:1789760771.597910:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c46a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61779968->61784063] (req 61779968->61784063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.597922:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c46a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61779968->61784063] (req 61779968->61784063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.597932:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c46a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61779968->61784063] (req 61779968->61784063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.597944:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760771.597950:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c46a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61779968->61784063] (req 61779968->61784063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.597962:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c46a3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61779968->61784063] (req 61779968->61784063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.597985:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c46a3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61779968->61784063] (req 61779968->61784063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.597998:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c46a3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61779968->61784063] (req 61779968->61784063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.598103:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554695424:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5636us (9016us total) trans 4294982392 rc 0/0 00000100:00100000:2.0:1789760771.598115:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15094 00000100:00100000:2.0:1789760771.598125:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714825239585 time 714838066605 state 0x2 ref 0 00000100:00100000:3.0:1789760771.602113:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760771.602246:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554695552 00000100:00100000:0.0:1789760771.602255:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554695552 00000100:00100000:0.0:1789760771.602307:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714825239585 time 714838066605 state 0x0 ref 1 00000100:00100000:0.0:1789760771.602322:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714838327021 time 714838066605 state 0x1 ref 1 00000100:00100000:0.0:1789760771.602352:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714838327021 time 714847970877 state 0x0 ref 1 00000100:00100000:0.0:1789760771.602364:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0644a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554695552:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760771.602429:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61784064, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d778 00010000:00010000:0.0:1789760771.602438:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46aa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.602455:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760771.602461:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.602467:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61784064->61788159] (req 61784064->61788159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.602481:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61784064->61788159] (req 61784064->61788159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.609805:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982393 is committed 00002000:00010000:0.0:1789760771.609973:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61784064 -> 61788160 00002000:00010000:0.0:1789760771.609978:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120696 -> 120704 00010000:00010000:0.0:1789760771.609991:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61784064->61788159] (req 61784064->61788159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.610005:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46aa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61784064->61788159] (req 61784064->61788159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.610015:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61784064->61788159] (req 61784064->61788159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.610023:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760771.610028:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61784064->61788159] (req 61784064->61788159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.610038:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46aa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61784064->61788159] (req 61784064->61788159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.610056:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46aa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61784064->61788159] (req 61784064->61788159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.610069:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46aa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61784064->61788159] (req 61784064->61788159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.610330:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0644a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554695552:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7961us (8220us total) trans 4294982393 rc 0/0 00000100:00100000:0.0:1789760771.610341:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15095 00000100:00100000:0.0:1789760771.610352:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714838327021 time 714847970877 state 0x2 ref 0 00000100:00100000:2.0:1789760771.615354:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760771.615720:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554695680 00000100:00100000:0.0:1789760771.615732:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554695680 00000100:00100000:0.0:1789760771.615805:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714838327021 time 714847970877 state 0x0 ref 1 00000100:00100000:0.0:1789760771.615821:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714848231293 time 714847970877 state 0x1 ref 1 00000100:00100000:0.0:1789760771.615889:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714848231293 time 714861506409 state 0x0 ref 1 00000100:00100000:0.0:1789760771.615904:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554695680:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760771.615951:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61788160, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d780 00010000:00010000:0.0:1789760771.615960:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46b1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.615977:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760771.615982:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.615989:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61788160->61792255] (req 61788160->61792255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.616009:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61788160->61792255] (req 61788160->61792255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.622347:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982394 is committed 00002000:00010000:0.0:1789760771.624899:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61788160 -> 61792256 00002000:00010000:0.0:1789760771.624908:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120704 -> 120712 00010000:00010000:0.0:1789760771.624921:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61788160->61792255] (req 61788160->61792255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.624932:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46b1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61788160->61792255] (req 61788160->61792255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.624940:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61788160->61792255] (req 61788160->61792255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.624946:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760771.624951:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61788160->61792255] (req 61788160->61792255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.624959:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46b1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61788160->61792255] (req 61788160->61792255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.624976:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46b1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61788160->61792255] (req 61788160->61792255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.624983:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46b1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61788160->61792255] (req 61788160->61792255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.625117:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554695680:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9211us (9772us total) trans 4294982394 rc 0/0 00000100:00100000:0.0:1789760771.625127:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15096 00000100:00100000:0.0:1789760771.625196:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714848231293 time 714861506409 state 0x2 ref 0 00000100:00100000:2.0:1789760771.630741:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554695808 00000100:00100000:2.0:1789760771.630750:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554695808 00000100:00100000:2.0:1789760771.630814:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714848231293 time 714861506409 state 0x0 ref 1 00000100:00100000:2.0:1789760771.630829:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714861766825 time 714861506409 state 0x1 ref 1 00000100:00100000:2.0:1789760771.630901:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714861766825 time 714876518731 state 0x0 ref 1 00000100:00100000:2.0:1789760771.630914:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554695808:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760771.630962:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61792256, mtime: 0x6aad9502, atime: 0x0, ctime: 0x6aad9502, blocks: 0x1d788 00010000:00010000:2.0:1789760771.630972:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c46b8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.630987:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760771.630992:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.630999:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c46b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61792256->61796351] (req 61792256->61796351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.631017:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c46b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61792256->61796351] (req 61792256->61796351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.644103:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982395 is committed 00002000:00010000:0.0:1789760771.644255:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61792256 -> 61796352 00002000:00010000:0.0:1789760771.644260:0:8441:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760770 -> 1789760771 00002000:00010000:0.0:1789760771.644263:0:8441:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760770 -> 1789760771 00002000:00010000:0.0:1789760771.644265:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120712 -> 120720 00010000:00010000:0.0:1789760771.644280:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c46b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61792256->61796351] (req 61792256->61796351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.644292:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c46b8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61792256->61796351] (req 61792256->61796351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.644301:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c46b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61792256->61796351] (req 61792256->61796351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.644308:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760771.644313:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c46b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61792256->61796351] (req 61792256->61796351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.644322:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c46b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61792256->61796351] (req 61792256->61796351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.644348:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c46b8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61792256->61796351] (req 61792256->61796351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.644356:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c46b8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61792256->61796351] (req 61792256->61796351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.644895:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554695808:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13976us (14255us total) trans 4294982395 rc 0/0 00000100:00100000:0.0:1789760771.644906:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15097 00000100:00100000:0.0:1789760771.644917:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714861766825 time 714876518731 state 0x2 ref 0 00000100:00100000:2.0:1789760771.652223:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554695936 00000100:00100000:2.0:1789760771.652234:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554695936 00000100:00100000:2.0:1789760771.652294:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714861766825 time 714876518731 state 0x0 ref 1 00000100:00100000:2.0:1789760771.652309:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714876779147 time 714876518731 state 0x1 ref 1 00000100:00100000:2.0:1789760771.652410:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714876779147 time 714898027173 state 0x0 ref 1 00000100:00100000:2.0:1789760771.652424:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554695936:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760771.652472:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61796352, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d790 00010000:00010000:2.0:1789760771.652482:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c46bf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.652496:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760771.652501:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.652507:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c46bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61796352->61800447] (req 61796352->61800447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.652520:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c46bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61796352->61800447] (req 61796352->61800447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.661221:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982396 is committed 00010000:00010000:0.0:1789760771.666839:0:8453:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760771.675677:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554696064 00000100:00100000:0.0:1789760771.675689:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554696064 00000100:00100000:0.0:1789760771.675752:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714876779147 time 714898027173 state 0x0 ref 1 00000100:00100000:0.0:1789760771.675778:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714898287589 time 714898027173 state 0x1 ref 1 00000100:00100000:0.0:1789760771.675859:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714898287589 time 714921476637 state 0x0 ref 1 00000100:00100000:0.0:1789760771.675872:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554696064:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760771.675927:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61800448, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d798 00010000:00010000:0.0:1789760771.675937:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46c6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.675958:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760771.675962:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.675968:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61800448->61804543] (req 61800448->61804543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.675982:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61800448->61804543] (req 61800448->61804543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.683688:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982397 is committed 00002000:00010000:0.0:1789760771.683901:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61800448 -> 61804544 00002000:00010000:0.0:1789760771.683906:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120728 -> 120736 00010000:00010000:0.0:1789760771.683919:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61800448->61804543] (req 61800448->61804543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.683932:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61800448->61804543] (req 61800448->61804543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.683941:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61800448->61804543] (req 61800448->61804543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.683948:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760771.683952:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61800448->61804543] (req 61800448->61804543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.683961:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61800448->61804543] (req 61800448->61804543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.683987:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46c6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61800448->61804543] (req 61800448->61804543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.683994:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46c6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61800448->61804543] (req 61800448->61804543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.684354:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554696064:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8478us (9177us total) trans 4294982397 rc 0/0 00000100:00100000:0.0:1789760771.684367:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15099 00000100:00100000:0.0:1789760771.684425:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714898287589 time 714921476637 state 0x2 ref 0 00000100:00100000:3.0:1789760771.691368:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760771.691650:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554696192 00000100:00100000:0.0:1789760771.691660:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554696192 00000100:00100000:0.0:1789760771.691713:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714898287589 time 714921476637 state 0x0 ref 1 00000100:00100000:0.0:1789760771.691729:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714921737053 time 714921476637 state 0x1 ref 1 00000100:00100000:0.0:1789760771.691812:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714921737053 time 714937428809 state 0x0 ref 1 00000100:00100000:0.0:1789760771.691824:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06449c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554696192:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760771.691871:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61804544, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d7a0 00010000:00010000:0.0:1789760771.691880:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46cd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.691898:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760771.691903:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.691910:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61804544->61808639] (req 61804544->61808639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.691923:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61804544->61808639] (req 61804544->61808639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760771.699776:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982398 is committed 00002000:00010000:0.0:1789760771.700029:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61804544 -> 61808640 00002000:00010000:0.0:1789760771.700035:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120736 -> 120744 00010000:00010000:0.0:1789760771.700047:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61804544->61808639] (req 61804544->61808639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.700059:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61804544->61808639] (req 61804544->61808639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.700068:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61804544->61808639] (req 61804544->61808639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.700076:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760771.700081:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61804544->61808639] (req 61804544->61808639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.700091:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46cd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61804544->61808639] (req 61804544->61808639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.700109:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46cd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61804544->61808639] (req 61804544->61808639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.700121:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46cd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61804544->61808639] (req 61804544->61808639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.700226:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06449c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554696192:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8399us (8862us total) trans 4294982398 rc 0/0 00000100:00100000:0.0:1789760771.700236:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15100 00000100:00100000:0.0:1789760771.700256:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714921737053 time 714937428809 state 0x2 ref 0 00000100:00100000:2.0:1789760771.707042:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554696320 00000100:00100000:2.0:1789760771.707052:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554696320 00000100:00100000:2.0:1789760771.707107:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714921737053 time 714937428809 state 0x0 ref 1 00000100:00100000:2.0:1789760771.707124:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714937689225 time 714937428809 state 0x1 ref 1 00000100:00100000:2.0:1789760771.707190:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714937689225 time 714952807893 state 0x0 ref 1 00000100:00100000:2.0:1789760771.707201:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554696320:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760771.707253:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61808640, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d7a8 00010000:00010000:2.0:1789760771.707262:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46d4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.707277:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760771.707281:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.707287:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61808640->61812735] (req 61808640->61812735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.707300:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61808640->61812735] (req 61808640->61812735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760771.712663:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61808640 -> 61812736 00002000:00010000:2.0:1789760771.712669:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120744 -> 120752 00010000:00010000:2.0:1789760771.712682:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61808640->61812735] (req 61808640->61812735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.712695:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61808640->61812735] (req 61808640->61812735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.712703:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61808640->61812735] (req 61808640->61812735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.712710:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760771.712715:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61808640->61812735] (req 61808640->61812735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.712723:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46d4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61808640->61812735] (req 61808640->61812735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.712741:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46d4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61808640->61812735] (req 61808640->61812735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.712749:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46d4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61808640->61812735] (req 61808640->61812735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.713091:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554696320:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5885us (6299us total) trans 4294982399 rc 0/0 00000100:00100000:2.0:1789760771.713102:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15101 00000100:00100000:2.0:1789760771.713111:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714937689225 time 714952807893 state 0x2 ref 0 00000100:00100000:2.0:1789760771.722483:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554696448 00000100:00100000:2.0:1789760771.722628:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554696448 00000100:00100000:2.0:1789760771.722787:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714937689225 time 714952807893 state 0x0 ref 1 00000100:00100000:2.0:1789760771.722807:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714953068309 time 714952807893 state 0x1 ref 1 00000100:00100000:2.0:1789760771.722844:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714953068309 time 714968461570 state 0x0 ref 1 00000100:00100000:2.0:1789760771.722859:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554696448:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760771.722929:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61812736, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d7b0 00010000:00010000:2.0:1789760771.722942:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46db lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.723099:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:2.0:1789760771.723105:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.723113:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61812736->61816831] (req 61812736->61816831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.723130:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61812736->61816831] (req 61812736->61816831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760771.731066:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61812736 -> 61816832 00002000:00010000:2.0:1789760771.731073:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120752 -> 120760 00010000:00010000:2.0:1789760771.731090:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61812736->61816831] (req 61812736->61816831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.731111:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61812736->61816831] (req 61812736->61816831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.731122:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61812736->61816831] (req 61812736->61816831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.731130:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760771.731136:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61812736->61816831] (req 61812736->61816831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.731146:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46db lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61812736->61816831] (req 61812736->61816831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.731171:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46db lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61812736->61816831] (req 61812736->61816831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.731180:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46db lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61812736->61816831] (req 61812736->61816831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.731308:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554696448:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8445us (9118us total) trans 4294982400 rc 0/0 00000100:00100000:2.0:1789760771.731321:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15102 00000100:00100000:2.0:1789760771.731333:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714953068309 time 714968461570 state 0x2 ref 0 00000100:00100000:0.0:1789760771.740237:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554696576 00000100:00100000:0.0:1789760771.740251:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554696576 00000100:00100000:0.0:1789760771.740335:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714953068309 time 714968461570 state 0x0 ref 1 00000100:00100000:0.0:1789760771.740353:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714968721986 time 714968461570 state 0x1 ref 1 00000100:00100000:0.0:1789760771.740429:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714968721986 time 714986046529 state 0x0 ref 1 00000100:00100000:0.0:1789760771.740444:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554696576:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760771.740521:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61816832, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d7b8 00010000:00010000:0.0:1789760771.740533:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46e2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.740554:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760771.740560:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.740567:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61816832->61820927] (req 61816832->61820927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.740589:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61816832->61820927] (req 61816832->61820927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760771.747119:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61816832 -> 61820928 00002000:00010000:0.0:1789760771.747125:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120760 -> 120768 00010000:00010000:0.0:1789760771.747143:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61816832->61820927] (req 61816832->61820927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.747161:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61816832->61820927] (req 61816832->61820927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.747171:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61816832->61820927] (req 61816832->61820927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.747179:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760771.747185:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61816832->61820927] (req 61816832->61820927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.747195:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61816832->61820927] (req 61816832->61820927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.747219:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46e2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61816832->61820927] (req 61816832->61820927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.747227:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c46e2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61816832->61820927] (req 61816832->61820927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.747353:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554696576:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6904us (7240us total) trans 4294982401 rc 0/0 00000100:00100000:0.0:1789760771.747367:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15103 00000100:00100000:0.0:1789760771.747442:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714968721986 time 714986046529 state 0x2 ref 0 00000100:00100000:0.0:1789760771.753205:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554696704 00000100:00100000:0.0:1789760771.753219:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554696704 00000100:00100000:0.0:1789760771.753301:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714968721986 time 714986046529 state 0x0 ref 1 00000100:00100000:0.0:1789760771.753321:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714986306945 time 714986046529 state 0x1 ref 1 00000100:00100000:0.0:1789760771.753440:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714986306945 time 714999057702 state 0x0 ref 1 00000100:00100000:0.0:1789760771.753456:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554696704:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760771.753536:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61820928, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d7c0 00010000:00010000:0.0:1789760771.753548:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46e9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.753568:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=714s v=(0 0) 00010000:00010000:0.0:1789760771.753573:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.753582:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61820928->61825023] (req 61820928->61825023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.753600:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61820928->61825023] (req 61820928->61825023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.761611:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982402 is committed 00002000:00010000:0.0:1789760771.761749:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61820928 -> 61825024 00002000:00010000:0.0:1789760771.761770:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120768 -> 120776 00010000:00010000:0.0:1789760771.761787:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61820928->61825023] (req 61820928->61825023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.761802:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61820928->61825023] (req 61820928->61825023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.761813:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61820928->61825023] (req 61820928->61825023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.761821:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760771.761827:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61820928->61825023] (req 61820928->61825023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.761838:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61820928->61825023] (req 61820928->61825023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.761866:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46e9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61820928->61825023] (req 61820928->61825023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.761874:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46e9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61820928->61825023] (req 61820928->61825023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.762001:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554696704:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8541us (9059us total) trans 4294982402 rc 0/0 00000100:00100000:0.0:1789760771.762016:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15104 00000100:00100000:0.0:1789760771.762029:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714986306945 time 714999057702 state 0x2 ref 0 00000100:00100000:2.0:1789760771.772621:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554696832 00000100:00100000:2.0:1789760771.772640:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554696832 00000100:00100000:2.0:1789760771.773043:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714986306945 time 714999057702 state 0x0 ref 1 00000100:00100000:2.0:1789760771.773063:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714999318118 time 714999057702 state 0x1 ref 1 00000100:00100000:2.0:1789760771.773151:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714999318118 time 715018768251 state 0x0 ref 1 00000100:00100000:2.0:1789760771.773167:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554696832:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760771.773234:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61825024, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d7c8 00010000:00010000:2.0:1789760771.773247:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46f0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.773269:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760771.773274:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.773282:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61825024->61829119] (req 61825024->61829119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.773299:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61825024->61829119] (req 61825024->61829119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760771.783862:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982403 is committed 00002000:00010000:2.0:1789760771.784220:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61825024 -> 61829120 00002000:00010000:2.0:1789760771.784230:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120776 -> 120784 00010000:00010000:2.0:1789760771.784247:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61825024->61829119] (req 61825024->61829119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.784262:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46f0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61825024->61829119] (req 61825024->61829119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.784279:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61825024->61829119] (req 61825024->61829119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.784288:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760771.784295:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61825024->61829119] (req 61825024->61829119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.784305:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46f0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61825024->61829119] (req 61825024->61829119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.784330:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46f0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61825024->61829119] (req 61825024->61829119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.784339:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46f0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61825024->61829119] (req 61825024->61829119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.784803:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554696832:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11632us (12332us total) trans 4294982403 rc 0/0 00000100:00100000:2.0:1789760771.784816:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15105 00000100:00100000:2.0:1789760771.784828:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714999318118 time 715018768251 state 0x2 ref 0 00000100:00100000:2.0:1789760771.798304:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554696960 00000100:00100000:2.0:1789760771.798318:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554696960 00000100:00100000:2.0:1789760771.798375:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 714999318118 time 715018768251 state 0x0 ref 1 00000100:00100000:2.0:1789760771.798407:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715019028667 time 715018768251 state 0x1 ref 1 00000100:00100000:2.0:1789760771.798443:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715019028667 time 715044061721 state 0x0 ref 1 00000100:00100000:2.0:1789760771.798457:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554696960:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760771.798503:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61829120, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d7d0 00010000:00010000:2.0:1789760771.798512:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46f7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.798527:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760771.798532:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.798539:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61829120->61833215] (req 61829120->61833215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.798553:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61829120->61833215] (req 61829120->61833215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760771.811692:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61829120 -> 61833216 00002000:00010000:2.0:1789760771.811697:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120784 -> 120792 00010000:00010000:2.0:1789760771.811709:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61829120->61833215] (req 61829120->61833215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.811722:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46f7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61829120->61833215] (req 61829120->61833215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.811733:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61829120->61833215] (req 61829120->61833215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.811747:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760771.811752:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61829120->61833215] (req 61829120->61833215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.811775:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46f7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61829120->61833215] (req 61829120->61833215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.811794:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46f7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61829120->61833215] (req 61829120->61833215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.811802:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c46f7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61829120->61833215] (req 61829120->61833215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.811991:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554696960:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13531us (15500us total) trans 4294982404 rc 0/0 00000100:00100000:2.0:1789760771.812003:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15106 00000100:00100000:2.0:1789760771.812012:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715019028667 time 715044061721 state 0x2 ref 0 00000100:00100000:0.0:1789760771.828332:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554697088 00000100:00100000:0.0:1789760771.828343:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554697088 00000100:00100000:0.0:1789760771.828462:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715019028667 time 715044061721 state 0x0 ref 1 00000100:00100000:0.0:1789760771.828477:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715044322137 time 715044061721 state 0x1 ref 1 00000100:00100000:0.0:1789760771.828508:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715044322137 time 715074125780 state 0x0 ref 1 00000100:00100000:0.0:1789760771.828627:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554697088:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760771.828672:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61833216, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d7d8 00010000:00010000:0.0:1789760771.828680:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46fe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.828696:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760771.828700:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.828707:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61833216->61837311] (req 61833216->61837311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.828720:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61833216->61837311] (req 61833216->61837311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760771.837512:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61833216 -> 61837312 00002000:00010000:0.0:1789760771.837518:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120792 -> 120800 00010000:00010000:0.0:1789760771.837532:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61833216->61837311] (req 61833216->61837311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.837544:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46fe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61833216->61837311] (req 61833216->61837311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.837552:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61833216->61837311] (req 61833216->61837311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.837560:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760771.837565:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61833216->61837311] (req 61833216->61837311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.837575:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46fe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61833216->61837311] (req 61833216->61837311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.837595:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46fe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61833216->61837311] (req 61833216->61837311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.837603:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c46fe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61833216->61837311] (req 61833216->61837311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.837726:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554697088:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9098us (9725us total) trans 4294982405 rc 0/0 00000100:00100000:0.0:1789760771.837736:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15107 00000100:00100000:0.0:1789760771.837745:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715044322137 time 715074125780 state 0x2 ref 0 00000100:00100000:0.0:1789760771.842470:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554697216 00000100:00100000:0.0:1789760771.842480:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554697216 00000100:00100000:0.0:1789760771.842529:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715044322137 time 715074125780 state 0x0 ref 1 00000100:00100000:0.0:1789760771.842543:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715074386196 time 715074125780 state 0x1 ref 1 00000100:00100000:0.0:1789760771.842572:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715074386196 time 715088190444 state 0x0 ref 1 00000100:00100000:0.0:1789760771.842583:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554697216:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760771.842629:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61837312, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d7e0 00010000:00010000:0.0:1789760771.842648:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4705 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.842665:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760771.842669:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.842675:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4705 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61837312->61841407] (req 61837312->61841407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.842688:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4705 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61837312->61841407] (req 61837312->61841407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760771.849646:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61837312 -> 61841408 00002000:00010000:0.0:1789760771.849652:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120800 -> 120808 00010000:00010000:0.0:1789760771.849665:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4705 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61837312->61841407] (req 61837312->61841407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.849684:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4705 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61837312->61841407] (req 61837312->61841407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.849693:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4705 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61837312->61841407] (req 61837312->61841407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.849701:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760771.849706:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4705 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61837312->61841407] (req 61837312->61841407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.849715:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4705 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61837312->61841407] (req 61837312->61841407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.849735:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4705 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61837312->61841407] (req 61837312->61841407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.849743:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4705 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61837312->61841407] (req 61837312->61841407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.849846:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554697216:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7259us (7482us total) trans 4294982406 rc 0/0 00000100:00100000:0.0:1789760771.849856:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15108 00000100:00100000:0.0:1789760771.849865:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715074386196 time 715088190444 state 0x2 ref 0 00000100:00100000:0.0:1789760771.854721:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760771.855043:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554697344 00000100:00100000:3.0:1789760771.855053:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554697344 00000100:00100000:3.0:1789760771.855111:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715074386196 time 715088190444 state 0x0 ref 1 00000100:00100000:3.0:1789760771.855126:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715088450860 time 715088190444 state 0x1 ref 1 00000100:00100000:3.0:1789760771.855167:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715088450860 time 715100784956 state 0x0 ref 1 00000100:00100000:3.0:1789760771.855179:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554697344:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760771.855227:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61841408, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d7e8 00010000:00010000:3.0:1789760771.855237:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c470c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.855253:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760771.855259:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.855265:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c470c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61841408->61845503] (req 61841408->61845503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.855280:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c470c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61841408->61845503] (req 61841408->61845503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760771.861688:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61841408 -> 61845504 00002000:00010000:3.0:1789760771.861692:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120808 -> 120816 00010000:00010000:3.0:1789760771.861705:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c470c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61841408->61845503] (req 61841408->61845503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.861719:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c470c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61841408->61845503] (req 61841408->61845503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.861728:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c470c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61841408->61845503] (req 61841408->61845503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.861736:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760771.861741:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c470c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61841408->61845503] (req 61841408->61845503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.861751:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c470c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61841408->61845503] (req 61841408->61845503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.861790:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c470c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61841408->61845503] (req 61841408->61845503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.861803:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c470c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61841408->61845503] (req 61841408->61845503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.861896:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554697344:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6714us (7181us total) trans 4294982407 rc 0/0 00000100:00100000:3.0:1789760771.861907:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15109 00000100:00100000:3.0:1789760771.861917:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715088450860 time 715100784956 state 0x2 ref 0 00000100:00100000:2.0:1789760771.872861:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554697472 00000100:00100000:2.0:1789760771.872871:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554697472 00000100:00100000:2.0:1789760771.872928:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715088450860 time 715100784956 state 0x0 ref 1 00000100:00100000:2.0:1789760771.872943:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715101045372 time 715100784956 state 0x1 ref 1 00000100:00100000:2.0:1789760771.872977:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715101045372 time 715118595075 state 0x0 ref 1 00000100:00100000:2.0:1789760771.872993:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554697472:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760771.873039:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61845504, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d7f0 00010000:00010000:2.0:1789760771.873047:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4713 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.873062:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760771.873066:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.873072:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4713 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61845504->61849599] (req 61845504->61849599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.873087:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4713 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61845504->61849599] (req 61845504->61849599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760771.881697:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61845504 -> 61849600 00002000:00010000:2.0:1789760771.881704:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120816 -> 120824 00010000:00010000:2.0:1789760771.881717:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4713 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61845504->61849599] (req 61845504->61849599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.881731:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4713 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61845504->61849599] (req 61845504->61849599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.881740:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4713 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61845504->61849599] (req 61845504->61849599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.881747:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760771.881753:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4713 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61845504->61849599] (req 61845504->61849599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.881774:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4713 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61845504->61849599] (req 61845504->61849599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.881794:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4713 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61845504->61849599] (req 61845504->61849599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.881802:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4713 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61845504->61849599] (req 61845504->61849599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.881942:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554697472:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8945us (9183us total) trans 4294982408 rc 0/0 00000100:00100000:2.0:1789760771.881953:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15110 00000100:00100000:2.0:1789760771.881963:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715101045372 time 715118595075 state 0x2 ref 0 00000100:00100000:0.0:1789760771.898263:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554697600 00000100:00100000:0.0:1789760771.898276:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554697600 00000100:00100000:0.0:1789760771.898337:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715101045372 time 715118595075 state 0x0 ref 1 00000100:00100000:0.0:1789760771.898353:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715118855491 time 715118595075 state 0x1 ref 1 00000100:00100000:0.0:1789760771.898423:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715118855491 time 715144040383 state 0x0 ref 1 00000100:00100000:0.0:1789760771.898437:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554697600:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760771.898485:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61849600, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d7f8 00010000:00010000:0.0:1789760771.898494:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c471a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.898524:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760771.898529:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760771.898536:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c471a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61849600->61853695] (req 61849600->61853695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760771.898551:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c471a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61849600->61853695] (req 61849600->61853695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.904060:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982409 is committed 00000100:00100000:0.0:1789760771.922928:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760771.923066:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554697728 00000100:00100000:2.0:1789760771.923076:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554697728 00000100:00100000:2.0:1789760771.923132:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715118855491 time 715144040383 state 0x0 ref 1 00000100:00100000:2.0:1789760771.923159:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715144300799 time 715144040383 state 0x1 ref 1 00000100:00100000:2.0:1789760771.923191:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715144300799 time 715168809273 state 0x0 ref 1 00000100:00100000:2.0:1789760771.923202:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554697728:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760771.923246:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61853696, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d800 00010000:00010000:2.0:1789760771.923255:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4721 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.923269:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760771.923273:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.923280:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4721 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61853696->61857791] (req 61853696->61857791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.923294:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4721 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61853696->61857791] (req 61853696->61857791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.928413:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982410 is committed 00002000:00010000:2.0:1789760771.928567:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61853696 -> 61857792 00002000:00010000:2.0:1789760771.928573:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120832 -> 120840 00010000:00010000:2.0:1789760771.928586:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4721 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61853696->61857791] (req 61853696->61857791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.928599:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4721 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61853696->61857791] (req 61853696->61857791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.928608:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4721 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61853696->61857791] (req 61853696->61857791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.928615:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760771.928620:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4721 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61853696->61857791] (req 61853696->61857791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.928629:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4721 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61853696->61857791] (req 61853696->61857791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.928649:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4721 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61853696->61857791] (req 61853696->61857791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.928658:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4721 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61853696->61857791] (req 61853696->61857791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.929228:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554697728:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5544us (5826us total) trans 4294982410 rc 0/0 00000100:00100000:2.0:1789760771.929239:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15112 00000100:00100000:2.0:1789760771.929248:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715144300799 time 715168809273 state 0x2 ref 0 00000100:00100000:2.0:1789760771.938341:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760771.938481:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554697856 00000100:00100000:3.0:1789760771.938492:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554697856 00000100:00100000:3.0:1789760771.938544:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715144300799 time 715168809273 state 0x0 ref 1 00000100:00100000:3.0:1789760771.938559:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715169069689 time 715168809273 state 0x1 ref 1 00000100:00100000:3.0:1789760771.938591:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715169069689 time 715184209431 state 0x0 ref 1 00000100:00100000:3.0:1789760771.938604:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554697856:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760771.938651:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61857792, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d808 00010000:00010000:3.0:1789760771.938660:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4728 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.938677:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760771.938681:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.938688:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4728 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61857792->61861887] (req 61857792->61861887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.938708:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4728 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61857792->61861887] (req 61857792->61861887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760771.951839:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61857792 -> 61861888 00002000:00010000:3.0:1789760771.951844:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120840 -> 120848 00010000:00010000:3.0:1789760771.951857:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4728 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61857792->61861887] (req 61857792->61861887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.951870:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4728 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61857792->61861887] (req 61857792->61861887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.951879:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4728 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61857792->61861887] (req 61857792->61861887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.951886:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760771.951891:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4728 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61857792->61861887] (req 61857792->61861887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.951900:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4728 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61857792->61861887] (req 61857792->61861887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.951918:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4728 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61857792->61861887] (req 61857792->61861887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.951925:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4728 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61857792->61861887] (req 61857792->61861887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760771.952025:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554697856:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13418us (13689us total) trans 4294982411 rc 0/0 00000100:00100000:3.0:1789760771.952046:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15113 00000100:00100000:3.0:1789760771.952058:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715169069689 time 715184209431 state 0x2 ref 0 00000100:00100000:2.0:1789760771.959806:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554697984 00000100:00100000:2.0:1789760771.959821:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554697984 00000100:00100000:2.0:1789760771.959883:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715169069689 time 715184209431 state 0x0 ref 1 00000100:00100000:2.0:1789760771.959898:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715184469847 time 715184209431 state 0x1 ref 1 00000100:00100000:2.0:1789760771.959930:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715184469847 time 715205549340 state 0x0 ref 1 00000100:00100000:2.0:1789760771.959943:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe1fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554697984:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760771.959987:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61861888, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d810 00010000:00010000:2.0:1789760771.959996:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c472f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.960017:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760771.960021:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760771.960026:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c472f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61861888->61865983] (req 61861888->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.960039:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c472f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61861888->61865983] (req 61861888->61865983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760771.969278:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982412 is committed 00002000:00010000:2.0:1789760771.969869:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61861888 -> 61865984 00002000:00010000:2.0:1789760771.969874:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120848 -> 120856 00010000:00010000:2.0:1789760771.969887:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c472f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61861888->61865983] (req 61861888->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.969899:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c472f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61861888->61865983] (req 61861888->61865983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.969908:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c472f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61861888->61865983] (req 61861888->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.969915:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760771.969920:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c472f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61861888->61865983] (req 61861888->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.969929:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c472f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61861888->61865983] (req 61861888->61865983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.969953:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c472f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61861888->61865983] (req 61861888->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760771.969961:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c472f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61861888->61865983] (req 61861888->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760771.970067:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe1fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554697984:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10122us (14936us total) trans 4294982412 rc 0/0 00000100:00100000:2.0:1789760771.970078:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15114 00000100:00100000:2.0:1789760771.970087:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715184469847 time 715205549340 state 0x2 ref 0 00000100:00100000:0.0:1789760771.977751:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:0.0:1789760771.983978:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982413 is committed 00000100:00100000:2.0:1789760771.987350:0:3648:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789760771.987372:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03e5ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3648:1876699573912576:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760771.987815:0:3647:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760771.987826:0:3647:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb06448a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3647:1876699573912704:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760771.987903:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760771.987976:0:3647:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb06449f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3647:1876699573912832:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789760771.988001:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760771.988028:0:7831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573912832 00000100:00100000:3.0:1789760771.988034:0:7831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573912832 00000100:00100000:3.0:1789760771.988079:0:7831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 380 00000100:00100000:3.0:1789760771.988088:0:7831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0644b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3647:x1876699573912832:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1789760771.988119:0:7831:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446227968 avail 1299988480 00000100:00100000:3.0:1789760771.988166:0:7831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0644b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3647:x1876699573912832:12345-0@lo:41:osp-pre-0-1.0 Request processed in 79us (163us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760771.988173:0:7831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 380 00000100:00100000:3.0:1789760771.988210:0:3647:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb06449f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3647:1876699573912832:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789760771.988236:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573912704 00000100:00100000:3.0:1789760771.988239:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573912704 00000100:00100000:3.0:1789760771.988267:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 354 00000100:00100000:3.0:1789760771.988273:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0644b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3647:x1876699573912704:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760771.988289:0:8440:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:3.0:1789760771.988336:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0644b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3647:x1876699573912704:12345-0@lo:13:osp-pre-1-0.0 Request processed in 63us (432us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760771.988342:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 354 00000100:00080000:3.0:1789760771.988356:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789760771.988402:0:3647:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb06448a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3647:1876699573912704:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760771.988839:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760771.988869:0:8439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573912576 00000100:00100000:2.0:1789760771.988873:0:8439:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573912576 00000100:00100000:2.0:1789760771.988899:0:8439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 355 00000100:00100000:2.0:1789760771.988905:0:8439:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3648:x1876699573912576:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1789760771.988915:0:8439:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:2.0:1789760771.988941:0:8439:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3648:x1876699573912576:12345-0@lo:13:osp-pre-1-1.0 Request processed in 37us (103us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760771.988947:0:8439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 355 00000100:00080000:2.0:1789760771.988959:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789760771.988977:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03e5ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3648:1876699573912576:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789760771.992242:0:3648:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789760771.992263:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb031b4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3648:1876699573913088:0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.992330:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760771.993412:0:6522:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573913088 00000100:00100000:3.0:1789760771.993433:0:6522:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573913088 00000100:00100000:3.0:1789760771.993491:0:6522:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 859 00000100:00100000:3.0:1789760771.993501:0:6522:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3648:x1876699573913088:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760771.993589:0:6522:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3648:x1876699573913088:12345-0@lo:400:kworker.0 Request processed in 89us (1258us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760771.993597:0:6522:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 859 00000100:00100000:2.0:1789760771.993802:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760771.993853:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb031b4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3648:1876699573913088:0@lo:400:kworker.0 00000100:00100000:3.0:1789760771.994269:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554698240 00000100:00100000:3.0:1789760771.994275:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554698240 00000100:00100000:3.0:1789760771.994317:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715205809756 time 715223580526 state 0x0 ref 1 00000100:00100000:3.0:1789760771.994333:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715223840942 time 715223580526 state 0x1 ref 1 00000100:00100000:3.0:1789760771.994364:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715223840942 time 715239982112 state 0x0 ref 1 00000100:00100000:3.0:1789760771.994374:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554698240:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760771.994613:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61870080, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d820 00010000:00010000:3.0:1789760771.994623:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c473d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.994640:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760771.994644:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760771.994651:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c473d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61870080->61874175] (req 61870080->61874175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760771.994664:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c473d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61870080->61874175] (req 61870080->61874175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760771.996008:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb031b7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3646:1876699573913216:0@lo:400:kworker.0 00000100:00100000:0.0:1789760771.996044:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760771.996067:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb044b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573913728:0@lo:400:kworker.0 00000100:00100000:0.0:1789760771.996089:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760771.996110:0:3646:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1789760771.996115:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb031b6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573913344:0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.996227:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb031b5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3648:1876699573913472:0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.996258:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760771.996279:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb049d4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3648:1876699573913984:0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.996299:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760771.996316:0:3648:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1789760771.996320:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb031b4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3648:1876699573913600:0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.996340:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760771.996355:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb049d6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3648:1876699573914112:0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.996374:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760771.996479:0:7470:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573913216 00000100:00100000:2.0:1789760771.996486:0:7470:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573913216 00000100:00100000:2.0:1789760771.996530:0:7470:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573913472 00000100:00100000:2.0:1789760771.996533:0:7470:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573913472 00000100:00100000:2.0:1789760771.996553:0:7470:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573913984 00000100:00100000:2.0:1789760771.996555:0:7470:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573913984 00000100:00100000:2.0:1789760771.996564:0:7470:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573913600 00000100:00100000:2.0:1789760771.996566:0:7470:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573913600 00000100:00100000:2.0:1789760771.996577:0:7470:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573914112 00000100:00100000:2.0:1789760771.996579:0:7470:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573914112 00000100:00100000:2.0:1789760771.996592:0:7470:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 860 00000100:00100000:2.0:1789760771.996600:0:7470:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3646:x1876699573913216:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.996646:0:7470:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3646:x1876699573913216:12345-0@lo:400:kworker.0 Request processed in 46us (601us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760771.996657:0:7470:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 860 00000100:00100000:2.0:1789760771.996672:0:7470:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 861 00000100:00100000:2.0:1789760771.996677:0:7470:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3648:x1876699573913472:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.996702:0:7470:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3648:x1876699573913472:12345-0@lo:400:kworker.0 Request processed in 25us (443us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760771.996707:0:7470:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 861 00000100:00100000:2.0:1789760771.996716:0:7470:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 862 00000100:00100000:2.0:1789760771.996720:0:7470:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3648:x1876699573913984:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.996740:0:7470:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3648:x1876699573913984:12345-0@lo:400:kworker.0 Request processed in 20us (441us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760771.996745:0:7470:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 862 00000100:00100000:2.0:1789760771.996754:0:7470:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 863 00000100:00100000:2.0:1789760771.996771:0:7470:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3648:x1876699573913600:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.996793:0:7470:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3648:x1876699573913600:12345-0@lo:400:kworker.0 Request processed in 35us (453us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760771.996799:0:7470:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 863 00000100:00100000:2.0:1789760771.996808:0:7470:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 864 00000100:00100000:2.0:1789760771.996814:0:7470:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3648:x1876699573914112:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.996833:0:7470:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3648:x1876699573914112:12345-0@lo:400:kworker.0 Request processed in 20us (459us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760771.996838:0:7470:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 864 00000100:00080000:2.0:1789760771.996859:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:2.0:1789760771.996863:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760771.996886:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb031b4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3648:1876699573913600:0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.996902:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb049d6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3648:1876699573914112:0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.996909:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb031b5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3648:1876699573913472:0@lo:400:kworker.0 00000100:00100000:2.0:1789760771.996915:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb049d4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3648:1876699573913984:0@lo:400:kworker.0 00000100:00100000:3.0:1789760771.997106:0:6514:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573912960 00000100:00100000:3.0:1789760771.997112:0:6514:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573912960 00000100:00100000:3.0:1789760771.997145:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 467 00000100:00100000:3.0:1789760771.997162:0:6514:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc320b7450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5711db52-8702-4d6b-b42c-8543e9bf1cb2+24:3649:x1876699573912960:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760771.997204:0:6514:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc320b7450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5711db52-8702-4d6b-b42c-8543e9bf1cb2+24:3649:x1876699573912960:12345-0@lo:400:kworker.0 Request processed in 44us (1494us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760771.997211:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 467 00000100:00100000:3.0:1789760771.997271:0:8438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573913728 00000100:00100000:3.0:1789760771.997276:0:8438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573913728 00000100:00080000:2.0:1789760771.997319:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760771.997344:0:8438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 292 00000100:00100000:3.0:1789760771.997353:0:8438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3646:x1876699573913728:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760771.997408:0:8438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3646:x1876699573913728:12345-0@lo:400:kworker.0 Request processed in 56us (1316us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760771.997415:0:8438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 292 00000100:00100000:0.0:1789760771.997712:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760771.997750:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb049d4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573913856:0@lo:400:kworker.0 00000100:00100000:0.0:1789760771.997789:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760771.997822:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb031b7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3646:1876699573913216:0@lo:400:kworker.0 00000100:00100000:0.0:1789760771.997831:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb044b3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573913728:0@lo:400:kworker.0 00000100:00100000:0.0:1789760771.998046:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb031b6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573913344:0@lo:400:kworker.0 00000100:00100000:3.0:1789760771.998104:0:8436:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573913856 00000100:00100000:3.0:1789760771.998109:0:8436:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573913856 00000100:00080000:0.0:1789760771.998218:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760771.998271:0:8436:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 293 00000100:00100000:3.0:1789760771.998278:0:8436:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573913856:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760771.998328:0:8436:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573913856:12345-0@lo:400:kworker.0 Request processed in 50us (538us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760771.998334:0:8436:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 293 00000001:00080000:0.0:1789760772.000069:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982414 is committed 00002000:00010000:3.0:1789760772.000275:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61870080 -> 61874176 00002000:00010000:3.0:1789760772.000281:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120864 -> 120872 00010000:00010000:3.0:1789760772.000293:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c473d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61870080->61874175] (req 61870080->61874175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.000305:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c473d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61870080->61874175] (req 61870080->61874175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.000313:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c473d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61870080->61874175] (req 61870080->61874175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.000321:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760772.000325:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c473d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61870080->61874175] (req 61870080->61874175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.000334:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c473d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61870080->61874175] (req 61870080->61874175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.000354:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c473d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61870080->61874175] (req 61870080->61874175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.000362:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c473d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61870080->61874175] (req 61870080->61874175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.000770:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554698240:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6386us (6958us total) trans 4294982414 rc 0/0 00000100:00100000:3.0:1789760772.000781:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15116 00000100:00100000:3.0:1789760772.000791:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715223840942 time 715239982112 state 0x2 ref 0 00000100:00100000:3.0:1789760772.005745:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554698368 00000100:00100000:3.0:1789760772.005764:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554698368 00000100:00100000:3.0:1789760772.005819:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715223840942 time 715239982112 state 0x0 ref 1 00000100:00100000:3.0:1789760772.005834:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715240242528 time 715239982112 state 0x1 ref 1 00000100:00100000:3.0:1789760772.005864:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715240242528 time 715251482288 state 0x0 ref 1 00000100:00100000:3.0:1789760772.005878:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554698368:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760772.005924:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61874176, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d828 00010000:00010000:3.0:1789760772.005933:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4744 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.005949:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760772.005954:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.005960:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4744 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61874176->61878271] (req 61874176->61878271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.005974:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4744 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61874176->61878271] (req 61874176->61878271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.012048:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982415 is committed 00002000:00010000:3.0:1789760772.012472:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61874176 -> 61878272 00002000:00010000:3.0:1789760772.012478:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120872 -> 120880 00010000:00010000:3.0:1789760772.012491:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4744 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61874176->61878271] (req 61874176->61878271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.012517:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4744 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61874176->61878271] (req 61874176->61878271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.012527:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4744 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61874176->61878271] (req 61874176->61878271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.012534:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760772.012539:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4744 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61874176->61878271] (req 61874176->61878271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.012549:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4744 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61874176->61878271] (req 61874176->61878271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.012568:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4744 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61874176->61878271] (req 61874176->61878271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.012576:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4744 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61874176->61878271] (req 61874176->61878271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.013213:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554698368:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7331us (7892us total) trans 4294982415 rc 0/0 00000100:00100000:3.0:1789760772.013224:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15117 00000100:00100000:3.0:1789760772.013234:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715240242528 time 715251482288 state 0x2 ref 0 00000100:00100000:0.0:1789760772.017579:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760772.017702:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554698496 00000100:00100000:2.0:1789760772.017712:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554698496 00000100:00100000:2.0:1789760772.017775:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715240242528 time 715251482288 state 0x0 ref 1 00000100:00100000:2.0:1789760772.017790:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715251742704 time 715251482288 state 0x1 ref 1 00000100:00100000:2.0:1789760772.017851:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715251742704 time 715263468937 state 0x0 ref 1 00000100:00100000:2.0:1789760772.017864:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554698496:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760772.017908:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61878272, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d830 00010000:00010000:2.0:1789760772.017917:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c474b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.017932:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760772.017937:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.017943:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c474b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61878272->61882367] (req 61878272->61882367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.017957:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c474b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61878272->61882367] (req 61878272->61882367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.022723:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982416 is committed 00002000:00010000:2.0:1789760772.022990:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61878272 -> 61882368 00002000:00010000:2.0:1789760772.022995:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120880 -> 120888 00010000:00010000:2.0:1789760772.023009:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c474b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61878272->61882367] (req 61878272->61882367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.023022:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c474b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61878272->61882367] (req 61878272->61882367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.023032:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c474b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61878272->61882367] (req 61878272->61882367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.023040:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760772.023045:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c474b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61878272->61882367] (req 61878272->61882367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.023055:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c474b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61878272->61882367] (req 61878272->61882367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.023076:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c474b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61878272->61882367] (req 61878272->61882367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.023083:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c474b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61878272->61882367] (req 61878272->61882367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.023273:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554698496:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5405us (5697us total) trans 4294982416 rc 0/0 00000100:00100000:2.0:1789760772.023285:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15118 00000100:00100000:2.0:1789760772.023295:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715251742704 time 715263468937 state 0x2 ref 0 00000100:00100000:2.0:1789760772.027475:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760772.027649:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554698624 00000100:00100000:0.0:1789760772.027658:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554698624 00000100:00100000:0.0:1789760772.027712:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715251742704 time 715263468937 state 0x0 ref 1 00000100:00100000:0.0:1789760772.027727:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715263729353 time 715263468937 state 0x1 ref 1 00000100:00100000:0.0:1789760772.027764:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715263729353 time 715273373968 state 0x0 ref 1 00000100:00100000:0.0:1789760772.027776:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554698624:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760772.027820:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61882368, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d838 00010000:00010000:0.0:1789760772.027829:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4752 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.027845:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760772.027849:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.027855:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4752 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61882368->61886463] (req 61882368->61886463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.027868:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4752 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61882368->61886463] (req 61882368->61886463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.039010:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982417 is committed 00000100:00100000:3.0:1789760772.047407:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554698752 00000100:00100000:3.0:1789760772.047415:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554698752 00000100:00100000:3.0:1789760772.047460:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715263729353 time 715273373968 state 0x0 ref 1 00000100:00100000:3.0:1789760772.047472:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715273634384 time 715273373968 state 0x1 ref 1 00000100:00100000:3.0:1789760772.047540:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715273634384 time 715293158882 state 0x0 ref 1 00000100:00100000:3.0:1789760772.047549:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554698752:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760772.047585:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61886464, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d840 00010000:00010000:3.0:1789760772.047592:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4759 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.047604:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760772.047607:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.047612:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4759 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61886464->61890559] (req 61886464->61890559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.047622:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4759 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61886464->61890559] (req 61886464->61890559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.053808:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982418 is committed 00002000:00010000:3.0:1789760772.054068:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61886464 -> 61890560 00002000:00010000:3.0:1789760772.054084:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120896 -> 120904 00010000:00010000:3.0:1789760772.054100:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4759 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61886464->61890559] (req 61886464->61890559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.054114:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4759 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61886464->61890559] (req 61886464->61890559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.054129:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4759 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61886464->61890559] (req 61886464->61890559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.054144:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760772.054149:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4759 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61886464->61890559] (req 61886464->61890559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.054158:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4759 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61886464->61890559] (req 61886464->61890559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.054176:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4759 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61886464->61890559] (req 61886464->61890559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.054184:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4759 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61886464->61890559] (req 61886464->61890559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.054275:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554698752:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6722us (6935us total) trans 4294982418 rc 0/0 00000100:00100000:3.0:1789760772.054285:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15120 00000100:00100000:3.0:1789760772.054295:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715273634384 time 715293158882 state 0x2 ref 0 00000100:00100000:3.0:1789760772.059300:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554698880 00000100:00100000:3.0:1789760772.059308:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554698880 00000100:00100000:3.0:1789760772.059356:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715273634384 time 715293158882 state 0x0 ref 1 00000100:00100000:3.0:1789760772.059371:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715293419298 time 715293158882 state 0x1 ref 1 00000001:00080000:2.0:1789760772.066309:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982419 is committed 00000100:00100000:2.0:1789760772.074122:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760772.079925:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982420 is committed 00000100:00100000:2.0:1789760772.091323:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760772.091548:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554699136 00000100:00100000:3.0:1789760772.091558:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554699136 00000100:00100000:3.0:1789760772.091611:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715305527710 time 715319903293 state 0x0 ref 1 00000100:00100000:3.0:1789760772.091627:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715320163709 time 715319903293 state 0x1 ref 1 00000100:00100000:3.0:1789760772.091660:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715320163709 time 715337278957 state 0x0 ref 1 00000100:00100000:3.0:1789760772.091671:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554699136:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760772.091716:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61898752, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d858 00010000:00010000:3.0:1789760772.091725:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c476e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.091742:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760772.091746:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.091752:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c476e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61898752->61902847] (req 61898752->61902847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.091780:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c476e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61898752->61902847] (req 61898752->61902847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.100703:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982421 is committed 00002000:00010000:2.0:1789760772.100875:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61898752 -> 61902848 00002000:00010000:2.0:1789760772.100881:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120920 -> 120928 00010000:00010000:2.0:1789760772.100895:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c476e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61898752->61902847] (req 61898752->61902847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.100909:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c476e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61898752->61902847] (req 61898752->61902847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.100918:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c476e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61898752->61902847] (req 61898752->61902847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.100925:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760772.100930:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c476e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61898752->61902847] (req 61898752->61902847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.100939:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c476e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61898752->61902847] (req 61898752->61902847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.100959:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c476e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61898752->61902847] (req 61898752->61902847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.100967:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c476e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61898752->61902847] (req 61898752->61902847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.101063:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554699136:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9388us (9745us total) trans 4294982421 rc 0/0 00000100:00100000:2.0:1789760772.101073:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15123 00000100:00100000:2.0:1789760772.101082:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715320163709 time 715337278957 state 0x2 ref 0 00000100:00100000:3.0:1789760772.104692:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760772.105149:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554699264 00000100:00100000:0.0:1789760772.105159:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554699264 00000100:00100000:0.0:1789760772.105213:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715320163709 time 715337278957 state 0x0 ref 1 00000100:00100000:0.0:1789760772.105229:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715337539373 time 715337278957 state 0x1 ref 1 00000100:00100000:0.0:1789760772.105264:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715337539373 time 715350882536 state 0x0 ref 1 00000100:00100000:0.0:1789760772.105277:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06448a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554699264:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760772.105327:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61902848, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d860 00010000:00010000:0.0:1789760772.105336:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4775 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.105352:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760772.105357:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.105364:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4775 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61902848->61906943] (req 61902848->61906943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.105628:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4775 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61902848->61906943] (req 61902848->61906943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.112558:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982422 is committed 00002000:00010000:3.0:1789760772.112687:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61902848 -> 61906944 00002000:00010000:3.0:1789760772.112692:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120928 -> 120936 00010000:00010000:3.0:1789760772.112705:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4775 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61902848->61906943] (req 61902848->61906943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.112717:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4775 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61902848->61906943] (req 61902848->61906943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.112736:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4775 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61902848->61906943] (req 61902848->61906943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.112751:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760772.112765:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4775 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61902848->61906943] (req 61902848->61906943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.112774:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4775 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61902848->61906943] (req 61902848->61906943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.112792:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4775 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61902848->61906943] (req 61902848->61906943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.112800:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4775 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61902848->61906943] (req 61902848->61906943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.112900:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06448a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554699264:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7620us (8212us total) trans 4294982422 rc 0/0 00000100:00100000:3.0:1789760772.112911:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15124 00000100:00100000:3.0:1789760772.112921:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715337539373 time 715350882536 state 0x2 ref 0 00000100:00100000:2.0:1789760772.122362:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554699392 00000100:00100000:2.0:1789760772.122373:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554699392 00000100:00100000:2.0:1789760772.122468:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715337539373 time 715350882536 state 0x0 ref 1 00000100:00100000:2.0:1789760772.122483:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715351142952 time 715350882536 state 0x1 ref 1 00000100:00100000:2.0:1789760772.122568:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715351142952 time 715368185519 state 0x0 ref 1 00000100:00100000:2.0:1789760772.122583:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554699392:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760772.122633:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61906944, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d868 00010000:00010000:2.0:1789760772.122643:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c477c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.122661:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760772.122666:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.122673:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c477c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61906944->61911039] (req 61906944->61911039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.122687:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c477c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61906944->61911039] (req 61906944->61911039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.133073:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982423 is committed 00000100:00100000:2.0:1789760772.140254:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760772.146484:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982424 is committed 00000100:00100000:0.0:1789760772.152802:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760772.152887:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554699648 00000100:00100000:3.0:1789760772.152898:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554699648 00000100:00100000:3.0:1789760772.152954:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715368445935 time 715386230315 state 0x0 ref 1 00000100:00100000:3.0:1789760772.152970:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715386490731 time 715386230315 state 0x1 ref 1 00000100:00100000:3.0:1789760772.153031:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715386490731 time 715398649683 state 0x0 ref 1 00000100:00100000:3.0:1789760772.153042:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554699648:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760772.153091:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61915136, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d878 00010000:00010000:3.0:1789760772.153099:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c478a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.153114:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760772.153118:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.153124:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c478a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61915136->61919231] (req 61915136->61919231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.153146:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c478a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61915136->61919231] (req 61915136->61919231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.158740:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982425 is committed 00002000:00010000:3.0:1789760772.158854:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61915136 -> 61919232 00002000:00010000:3.0:1789760772.158858:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120952 -> 120960 00010000:00010000:3.0:1789760772.158869:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c478a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61915136->61919231] (req 61915136->61919231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.158878:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c478a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61915136->61919231] (req 61915136->61919231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.158889:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c478a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61915136->61919231] (req 61915136->61919231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.158894:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760772.158897:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c478a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61915136->61919231] (req 61915136->61919231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.158903:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c478a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61915136->61919231] (req 61915136->61919231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.158917:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c478a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61915136->61919231] (req 61915136->61919231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.158923:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c478a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61915136->61919231] (req 61915136->61919231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.159025:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554699648:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5982us (6230us total) trans 4294982425 rc 0/0 00000100:00100000:3.0:1789760772.159033:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15127 00000100:00100000:3.0:1789760772.159041:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715386490731 time 715398649683 state 0x2 ref 0 00000100:00100000:2.0:1789760772.162484:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760772.162547:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554699776 00000100:00100000:3.0:1789760772.162558:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554699776 00000100:00100000:3.0:1789760772.162616:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715386490731 time 715398649683 state 0x0 ref 1 00000100:00100000:3.0:1789760772.162632:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715398910099 time 715398649683 state 0x1 ref 1 00000100:00100000:3.0:1789760772.162663:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715398910099 time 715408281817 state 0x0 ref 1 00000100:00100000:3.0:1789760772.162673:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554699776:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760772.162716:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61919232, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d880 00010000:00010000:3.0:1789760772.162725:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4791 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.162740:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760772.162745:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.162751:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4791 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61919232->61923327] (req 61919232->61923327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.162777:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4791 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61919232->61923327] (req 61919232->61923327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.171254:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982426 is committed 00002000:00010000:3.0:1789760772.171442:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61919232 -> 61923328 00002000:00010000:3.0:1789760772.171448:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120960 -> 120968 00010000:00010000:3.0:1789760772.171461:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4791 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61919232->61923327] (req 61919232->61923327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.171474:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4791 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61919232->61923327] (req 61919232->61923327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.171488:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4791 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61919232->61923327] (req 61919232->61923327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.171501:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760772.171506:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4791 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61919232->61923327] (req 61919232->61923327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.171515:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4791 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61919232->61923327] (req 61919232->61923327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.171530:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4791 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61919232->61923327] (req 61919232->61923327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.171535:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4791 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61919232->61923327] (req 61919232->61923327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.171804:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554699776:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9127us (9324us total) trans 4294982426 rc 0/0 00000100:00100000:3.0:1789760772.171815:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15128 00000100:00100000:3.0:1789760772.171824:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715398910099 time 715408281817 state 0x2 ref 0 00000100:00100000:2.0:1789760772.177514:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:0.0:1789760772.183942:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982427 is committed 00000100:00100000:0.0:1789760772.198514:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:2.0:1789760772.207996:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61927424 -> 61931520 00002000:00010000:2.0:1789760772.208005:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120976 -> 120984 00010000:00010000:2.0:1789760772.208019:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c479f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61927424->61931519] (req 61927424->61931519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.208033:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c479f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61927424->61931519] (req 61927424->61931519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.208043:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c479f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61927424->61931519] (req 61927424->61931519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.208050:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760772.208055:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c479f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61927424->61931519] (req 61927424->61931519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.208064:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c479f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61927424->61931519] (req 61927424->61931519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.208085:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c479f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61927424->61931519] (req 61927424->61931519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.208094:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c479f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61927424->61931519] (req 61927424->61931519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.208329:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554700032:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9566us (9819us total) trans 4294982428 rc 0/0 00000100:00100000:2.0:1789760772.208341:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15130 00000100:00100000:2.0:1789760772.208350:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715423568938 time 715444365298 state 0x2 ref 0 00000100:00100000:2.0:1789760772.218646:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760772.218782:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554700160 00000100:00100000:3.0:1789760772.218792:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554700160 00000100:00100000:3.0:1789760772.218845:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715423568938 time 715444365298 state 0x0 ref 1 00000100:00100000:3.0:1789760772.218860:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715444625714 time 715444365298 state 0x1 ref 1 00000100:00100000:3.0:1789760772.218893:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715444625714 time 715464511329 state 0x0 ref 1 00000100:00100000:3.0:1789760772.218909:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554700160:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760772.218958:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61931520, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d898 00010000:00010000:3.0:1789760772.218968:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47a6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.218985:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760772.218989:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.218995:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61931520->61935615] (req 61931520->61935615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.219008:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61931520->61935615] (req 61931520->61935615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760772.225470:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61931520 -> 61935616 00002000:00010000:3.0:1789760772.225475:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120984 -> 120992 00010000:00010000:3.0:1789760772.225488:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61931520->61935615] (req 61931520->61935615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.225498:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47a6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61931520->61935615] (req 61931520->61935615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.225506:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61931520->61935615] (req 61931520->61935615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.225523:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760772.225528:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61931520->61935615] (req 61931520->61935615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.225536:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47a6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61931520->61935615] (req 61931520->61935615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.225553:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47a6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61931520->61935615] (req 61931520->61935615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.225560:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47a6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61931520->61935615] (req 61931520->61935615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.225653:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554700160:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6742us (7014us total) trans 4294982429 rc 0/0 00000100:00100000:3.0:1789760772.225663:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15131 00000100:00100000:3.0:1789760772.225671:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715444625714 time 715464511329 state 0x2 ref 0 00000100:00100000:3.0:1789760772.231330:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554700288 00000100:00100000:3.0:1789760772.231341:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554700288 00000100:00100000:3.0:1789760772.231492:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715444625714 time 715464511329 state 0x0 ref 1 00000100:00100000:3.0:1789760772.231507:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715464771745 time 715464511329 state 0x1 ref 1 00000100:00100000:3.0:1789760772.232313:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715464771745 time 715477928154 state 0x0 ref 1 00000100:00100000:3.0:1789760772.232331:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554700288:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760772.232683:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61935616, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d8a0 00010000:00010000:3.0:1789760772.232695:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47ad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.232712:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760772.232717:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.232723:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61935616->61939711] (req 61935616->61939711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.232738:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61935616->61939711] (req 61935616->61939711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760772.238364:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61935616 -> 61939712 00002000:00010000:3.0:1789760772.238371:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 120992 -> 121000 00010000:00010000:3.0:1789760772.238470:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61935616->61939711] (req 61935616->61939711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.238484:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47ad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61935616->61939711] (req 61935616->61939711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.238494:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61935616->61939711] (req 61935616->61939711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.238501:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760772.238507:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61935616->61939711] (req 61935616->61939711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.238888:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61935616->61939711] (req 61935616->61939711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.238909:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47ad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61935616->61939711] (req 61935616->61939711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.238918:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47ad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61935616->61939711] (req 61935616->61939711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.239069:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554700288:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6737us (7836us total) trans 4294982430 rc 0/0 00000100:00100000:3.0:1789760772.239081:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15132 00000100:00100000:3.0:1789760772.239091:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715464771745 time 715477928154 state 0x2 ref 0 00000100:00100000:0.0:1789760772.248115:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760772.254454:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982431 is committed 00000100:00100000:2.0:1789760772.260255:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:2.0:1789760772.266866:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61943808 -> 61947904 00002000:00010000:2.0:1789760772.266871:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121008 -> 121016 00010000:00010000:2.0:1789760772.266885:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c47bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61943808->61947903] (req 61943808->61947903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.266897:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c47bb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61943808->61947903] (req 61943808->61947903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.266905:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c47bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61943808->61947903] (req 61943808->61947903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.266912:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760772.266917:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c47bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61943808->61947903] (req 61943808->61947903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.266925:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c47bb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61943808->61947903] (req 61943808->61947903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.266944:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c47bb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61943808->61947903] (req 61943808->61947903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.266952:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c47bb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61943808->61947903] (req 61943808->61947903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.267104:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554700544:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5869us (6851us total) trans 4294982432 rc 0/0 00000100:00100000:2.0:1789760772.267115:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15134 00000100:00100000:2.0:1789760772.267124:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715494369949 time 715506835801 state 0x2 ref 0 00000100:00100000:2.0:1789760772.274098:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760772.274987:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554700672 00000100:00100000:0.0:1789760772.275002:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554700672 00000100:00100000:0.0:1789760772.275070:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715494369949 time 715506835801 state 0x0 ref 1 00000100:00100000:0.0:1789760772.275089:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715507096217 time 715506835801 state 0x1 ref 1 00000100:00100000:0.0:1789760772.275131:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715507096217 time 715520748160 state 0x0 ref 1 00000100:00100000:0.0:1789760772.275149:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554700672:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760772.275206:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61947904, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d8b8 00010000:00010000:0.0:1789760772.275232:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47c2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.275251:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760772.275256:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.275263:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61947904->61951999] (req 61947904->61951999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.275277:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61947904->61951999] (req 61947904->61951999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760772.290037:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61947904 -> 61952000 00002000:00010000:0.0:1789760772.290043:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121016 -> 121024 00010000:00010000:0.0:1789760772.290056:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61947904->61951999] (req 61947904->61951999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.290078:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47c2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61947904->61951999] (req 61947904->61951999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.290094:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61947904->61951999] (req 61947904->61951999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.290102:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760772.290107:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61947904->61951999] (req 61947904->61951999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.290115:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47c2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61947904->61951999] (req 61947904->61951999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.290141:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47c2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61947904->61951999] (req 61947904->61951999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.290155:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47c2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61947904->61951999] (req 61947904->61951999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.290258:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554700672:12345-192.168.203.22@tcp:4:dd.0 Request processed in 15106us (16161us total) trans 4294982433 rc 0/0 00000100:00100000:0.0:1789760772.290269:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15135 00000100:00100000:0.0:1789760772.290280:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715507096217 time 715520748160 state 0x2 ref 0 00000100:00100000:0.0:1789760772.297018:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760772.297241:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554700800 00000100:00100000:3.0:1789760772.297252:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554700800 00000100:00100000:3.0:1789760772.297308:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715507096217 time 715520748160 state 0x0 ref 1 00000100:00100000:3.0:1789760772.297324:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715521008576 time 715520748160 state 0x1 ref 1 00000100:00100000:3.0:1789760772.297359:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715521008576 time 715542977216 state 0x0 ref 1 00000100:00100000:3.0:1789760772.297373:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554700800:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760772.297475:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61952000, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d8c0 00010000:00010000:3.0:1789760772.297485:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47c9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.297502:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760772.297508:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.297514:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61952000->61956095] (req 61952000->61956095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.297529:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61952000->61956095] (req 61952000->61956095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760772.305682:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61952000 -> 61956096 00002000:00010000:3.0:1789760772.305687:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121024 -> 121032 00010000:00010000:3.0:1789760772.305699:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61952000->61956095] (req 61952000->61956095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.305713:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47c9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61952000->61956095] (req 61952000->61956095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.305722:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61952000->61956095] (req 61952000->61956095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.305729:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760772.305734:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61952000->61956095] (req 61952000->61956095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.305743:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47c9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61952000->61956095] (req 61952000->61956095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.305771:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47c9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61952000->61956095] (req 61952000->61956095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.305783:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47c9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61952000->61956095] (req 61952000->61956095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.307294:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554700800:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9909us (10273us total) trans 4294982434 rc 0/0 00000100:00100000:3.0:1789760772.307309:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15136 00000100:00100000:3.0:1789760772.307319:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715521008576 time 715542977216 state 0x2 ref 0 00000100:00100000:2.0:1789760772.320575:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760772.332120:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982435 is committed 00000100:00100000:2.0:1789760772.339153:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:2.0:1789760772.349189:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61960192 -> 61964288 00002000:00010000:2.0:1789760772.349198:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121040 -> 121048 00010000:00010000:2.0:1789760772.349211:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c47d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61960192->61964287] (req 61960192->61964287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.349224:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c47d7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61960192->61964287] (req 61960192->61964287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.349235:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c47d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61960192->61964287] (req 61960192->61964287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.349243:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760772.349248:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c47d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61960192->61964287] (req 61960192->61964287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.349257:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c47d7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61960192->61964287] (req 61960192->61964287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.349273:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c47d7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61960192->61964287] (req 61960192->61964287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.349281:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c47d7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61960192->61964287] (req 61960192->61964287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.349438:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554701056:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9899us (10288us total) trans 4294982436 rc 0/0 00000100:00100000:2.0:1789760772.349449:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15138 00000100:00100000:2.0:1789760772.349457:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715566961529 time 715585138544 state 0x2 ref 0 00000100:00100000:3.0:1789760772.352859:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760772.352922:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554701184 00000100:00100000:0.0:1789760772.352932:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554701184 00000100:00100000:0.0:1789760772.352975:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715566961529 time 715585138544 state 0x0 ref 1 00000100:00100000:0.0:1789760772.352986:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715585398960 time 715585138544 state 0x1 ref 1 00000100:00100000:0.0:1789760772.353010:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715585398960 time 715598629820 state 0x0 ref 1 00000100:00100000:0.0:1789760772.353018:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06449f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554701184:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760772.353052:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61964288, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d8d8 00010000:00010000:0.0:1789760772.353059:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47de lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.353070:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760772.353073:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.353078:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61964288->61968383] (req 61964288->61968383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.353087:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61964288->61968383] (req 61964288->61968383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760772.359042:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61964288 -> 61968384 00002000:00010000:0.0:1789760772.359048:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121048 -> 121056 00010000:00010000:0.0:1789760772.359062:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61964288->61968383] (req 61964288->61968383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.359074:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47de lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61964288->61968383] (req 61964288->61968383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.359084:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61964288->61968383] (req 61964288->61968383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.359092:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760772.359098:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61964288->61968383] (req 61964288->61968383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.359107:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47de lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61964288->61968383] (req 61964288->61968383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.359127:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47de lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61964288->61968383] (req 61964288->61968383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.359196:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47de lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61964288->61968383] (req 61964288->61968383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.359284:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06449f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554701184:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6262us (6429us total) trans 4294982437 rc 0/0 00000100:00100000:0.0:1789760772.359295:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15139 00000100:00100000:0.0:1789760772.359305:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715585398960 time 715598629820 state 0x2 ref 0 00000100:00100000:3.0:1789760772.365023:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554701312 00000100:00100000:3.0:1789760772.365033:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554701312 00000100:00100000:3.0:1789760772.365086:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715585398960 time 715598629820 state 0x0 ref 1 00000100:00100000:3.0:1789760772.365100:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715598890236 time 715598629820 state 0x1 ref 1 00000100:00100000:3.0:1789760772.365190:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715598890236 time 715610748466 state 0x0 ref 1 00000100:00100000:3.0:1789760772.365206:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554701312:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760772.365254:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61968384, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d8e0 00010000:00010000:3.0:1789760772.365263:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47e5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.365279:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760772.365282:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.365288:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61968384->61972479] (req 61968384->61972479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.365301:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61968384->61972479] (req 61968384->61972479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760772.373933:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61968384 -> 61972480 00002000:00010000:2.0:1789760772.373939:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121056 -> 121064 00010000:00010000:2.0:1789760772.373953:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61968384->61972479] (req 61968384->61972479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.373970:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47e5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61968384->61972479] (req 61968384->61972479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.373979:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61968384->61972479] (req 61968384->61972479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.373985:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760772.373990:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61968384->61972479] (req 61968384->61972479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.373998:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61968384->61972479] (req 61968384->61972479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.374016:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47e5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61968384->61972479] (req 61968384->61972479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.374023:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c47e5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61968384->61972479] (req 61968384->61972479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.374117:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554701312:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8910us (9633us total) trans 4294982438 rc 0/0 00000100:00100000:2.0:1789760772.374127:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15140 00000100:00100000:2.0:1789760772.374137:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715598890236 time 715610748466 state 0x2 ref 0 00000100:00100000:2.0:1789760772.380486:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760772.390548:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982439 is committed 00000100:00100000:3.0:1789760772.404125:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760772.404259:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554701568 00000100:00100000:2.0:1789760772.404267:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554701568 00000100:00100000:2.0:1789760772.404325:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715611008882 time 715629567272 state 0x0 ref 1 00000100:00100000:2.0:1789760772.404339:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715629827688 time 715629567272 state 0x1 ref 1 00000100:00100000:2.0:1789760772.404370:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715629827688 time 715649988933 state 0x0 ref 1 00000100:00100000:2.0:1789760772.404419:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0644b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554701568:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760772.404461:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61976576, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d8f0 00010000:00010000:2.0:1789760772.404469:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c47f3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.404496:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760772.404500:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.404506:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c47f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61976576->61980671] (req 61976576->61980671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.404520:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c47f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61976576->61980671] (req 61976576->61980671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760772.409784:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61976576 -> 61980672 00002000:00010000:2.0:1789760772.409789:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121072 -> 121080 00010000:00010000:2.0:1789760772.409802:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c47f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61976576->61980671] (req 61976576->61980671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.409821:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c47f3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61976576->61980671] (req 61976576->61980671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.409831:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c47f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61976576->61980671] (req 61976576->61980671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.409838:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760772.409843:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c47f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61976576->61980671] (req 61976576->61980671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.409852:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c47f3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61976576->61980671] (req 61976576->61980671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.409871:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c47f3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61976576->61980671] (req 61976576->61980671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.409879:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c47f3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61976576->61980671] (req 61976576->61980671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.409972:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0644b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554701568:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5549us (5853us total) trans 4294982440 rc 0/0 00000100:00100000:2.0:1789760772.409983:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15142 00000100:00100000:2.0:1789760772.409992:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715629827688 time 715649988933 state 0x2 ref 0 00000100:00100000:2.0:1789760772.413707:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760772.413815:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554701696 00000100:00100000:0.0:1789760772.413826:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554701696 00000100:00100000:0.0:1789760772.413879:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715629827688 time 715649988933 state 0x0 ref 1 00000100:00100000:0.0:1789760772.413894:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715650249349 time 715649988933 state 0x1 ref 1 00000100:00100000:0.0:1789760772.413927:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715650249349 time 715659545147 state 0x0 ref 1 00000100:00100000:0.0:1789760772.413939:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554701696:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760772.413985:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61980672, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d8f8 00010000:00010000:0.0:1789760772.413994:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47fa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.414010:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760772.414014:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.414021:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61980672->61984767] (req 61980672->61984767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.414035:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61980672->61984767] (req 61980672->61984767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760772.419604:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61980672 -> 61984768 00002000:00010000:0.0:1789760772.419610:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121080 -> 121088 00010000:00010000:0.0:1789760772.419625:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61980672->61984767] (req 61980672->61984767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.419639:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47fa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61980672->61984767] (req 61980672->61984767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.419648:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61980672->61984767] (req 61980672->61984767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.419657:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760772.419662:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61980672->61984767] (req 61980672->61984767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.419671:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47fa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61980672->61984767] (req 61980672->61984767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.419691:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47fa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61980672->61984767] (req 61980672->61984767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.419699:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c47fa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61980672->61984767] (req 61980672->61984767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.419802:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554701696:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5861us (6100us total) trans 4294982441 rc 0/0 00000100:00100000:0.0:1789760772.419813:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15143 00000100:00100000:0.0:1789760772.419822:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715650249349 time 715659545147 state 0x2 ref 0 00000100:00100000:0.0:1789760772.427406:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554701824 00000100:00100000:0.0:1789760772.427417:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554701824 00000100:00100000:0.0:1789760772.427470:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715650249349 time 715659545147 state 0x0 ref 1 00000100:00100000:0.0:1789760772.427495:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715659805563 time 715659545147 state 0x1 ref 1 00000100:00100000:0.0:1789760772.427533:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715659805563 time 715673151417 state 0x0 ref 1 00000100:00100000:0.0:1789760772.427545:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554701824:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760772.427591:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61984768, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d900 00010000:00010000:0.0:1789760772.427599:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c4801 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.427615:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760772.427619:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.427625:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c4801 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61984768->61988863] (req 61984768->61988863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.427650:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c4801 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61984768->61988863] (req 61984768->61988863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.437006:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982442 is committed 00002000:00010000:2.0:1789760772.437123:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61984768 -> 61988864 00002000:00010000:2.0:1789760772.437139:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121088 -> 121096 00010000:00010000:2.0:1789760772.437153:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c4801 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61984768->61988863] (req 61984768->61988863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.437167:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c4801 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61984768->61988863] (req 61984768->61988863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.437176:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c4801 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61984768->61988863] (req 61984768->61988863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.437184:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760772.437189:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c4801 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61984768->61988863] (req 61984768->61988863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.437198:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c4801 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61984768->61988863] (req 61984768->61988863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.437216:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c4801 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61984768->61988863] (req 61984768->61988863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.437223:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c4801 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61984768->61988863] (req 61984768->61988863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.437356:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554701824:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9808us (10114us total) trans 4294982442 rc 0/0 00000100:00100000:2.0:1789760772.437367:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15144 00000100:00100000:2.0:1789760772.437413:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715659805563 time 715673151417 state 0x2 ref 0 00000100:00100000:3.0:1789760772.443444:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:0.0:1789760772.457412:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982443 is committed 00002000:00010000:0.0:1789760772.457786:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61988864 -> 61992960 00002000:00010000:0.0:1789760772.457792:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121096 -> 121104 00010000:00010000:0.0:1789760772.457804:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4808 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61988864->61992959] (req 61988864->61992959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.457816:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4808 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61988864->61992959] (req 61988864->61992959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.457824:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4808 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61988864->61992959] (req 61988864->61992959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.457832:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760772.457837:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4808 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61988864->61992959] (req 61988864->61992959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.457845:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4808 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61988864->61992959] (req 61988864->61992959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.457864:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4808 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61988864->61992959] (req 61988864->61992959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.457871:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4808 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61988864->61992959] (req 61988864->61992959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.458014:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06448380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554701952:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14360us (14577us total) trans 4294982443 rc 0/0 00000100:00100000:0.0:1789760772.458024:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15145 00000100:00100000:0.0:1789760772.458033:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715673411833 time 715689258764 state 0x2 ref 0 00000100:00100000:2.0:1789760772.464501:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554702080 00000100:00100000:2.0:1789760772.464511:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554702080 00000100:00100000:2.0:1789760772.464563:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715673411833 time 715689258764 state 0x0 ref 1 00000100:00100000:2.0:1789760772.464579:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715689519180 time 715689258764 state 0x1 ref 1 00000100:00100000:2.0:1789760772.464616:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715689519180 time 715710234928 state 0x0 ref 1 00000100:00100000:2.0:1789760772.464627:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554702080:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760772.464670:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61992960, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d910 00010000:00010000:2.0:1789760772.464678:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c480f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.464694:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760772.464712:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.464719:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c480f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61992960->61997055] (req 61992960->61997055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.464732:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c480f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61992960->61997055] (req 61992960->61997055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.473823:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982444 is committed 00002000:00010000:2.0:1789760772.473990:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61992960 -> 61997056 00002000:00010000:2.0:1789760772.473995:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121104 -> 121112 00010000:00010000:2.0:1789760772.474008:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c480f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61992960->61997055] (req 61992960->61997055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.474029:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c480f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61992960->61997055] (req 61992960->61997055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.474039:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c480f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61992960->61997055] (req 61992960->61997055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.474046:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760772.474051:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c480f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61992960->61997055] (req 61992960->61997055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.474061:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c480f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61992960->61997055] (req 61992960->61997055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.474081:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c480f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61992960->61997055] (req 61992960->61997055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.474089:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c480f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61992960->61997055] (req 61992960->61997055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.474434:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554702080:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9803us (10227us total) trans 4294982444 rc 0/0 00000100:00100000:2.0:1789760772.474445:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15146 00000100:00100000:2.0:1789760772.474455:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715689519180 time 715710234928 state 0x2 ref 0 00000100:00100000:0.0:1789760772.487644:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760772.488036:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554702208 00000100:00100000:3.0:1789760772.488053:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554702208 00000100:00100000:3.0:1789760772.488103:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715689519180 time 715710234928 state 0x0 ref 1 00000100:00100000:3.0:1789760772.488118:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715710495344 time 715710234928 state 0x1 ref 1 00000100:00100000:3.0:1789760772.488151:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715710495344 time 715733769883 state 0x0 ref 1 00000100:00100000:3.0:1789760772.488163:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554702208:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760772.488211:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 61997056, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d918 00010000:00010000:3.0:1789760772.488221:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4816 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.488236:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760772.488241:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.488247:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4816 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61997056->62001151] (req 61997056->62001151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.488261:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4816 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61997056->62001151] (req 61997056->62001151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.494283:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982445 is committed 00002000:00010000:3.0:1789760772.494540:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 61997056 -> 62001152 00002000:00010000:3.0:1789760772.494546:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121112 -> 121120 00010000:00010000:3.0:1789760772.494559:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4816 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61997056->62001151] (req 61997056->62001151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.494572:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4816 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61997056->62001151] (req 61997056->62001151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.494581:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4816 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61997056->62001151] (req 61997056->62001151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.494589:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760772.494594:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4816 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61997056->62001151] (req 61997056->62001151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.494604:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4816 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61997056->62001151] (req 61997056->62001151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.494623:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4816 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61997056->62001151] (req 61997056->62001151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.494630:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4816 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [61997056->62001151] (req 61997056->62001151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.494779:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554702208:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6612us (7139us total) trans 4294982445 rc 0/0 00000100:00100000:3.0:1789760772.494790:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15147 00000100:00100000:3.0:1789760772.494800:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715710495344 time 715733769883 state 0x2 ref 0 00000100:00100000:3.0:1789760772.503032:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760772.503115:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554702336 00000100:00100000:0.0:1789760772.503126:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554702336 00000100:00100000:0.0:1789760772.503241:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715710495344 time 715733769883 state 0x0 ref 1 00000100:00100000:0.0:1789760772.503259:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715734030299 time 715733769883 state 0x1 ref 1 00000100:00100000:0.0:1789760772.503342:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715734030299 time 715748959224 state 0x0 ref 1 00000100:00100000:0.0:1789760772.503356:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0644a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554702336:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760772.503449:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62001152, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d920 00010000:00010000:0.0:1789760772.503461:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c481d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.503486:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760772.503491:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.503496:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c481d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62001152->62005247] (req 62001152->62005247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.503510:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c481d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62001152->62005247] (req 62001152->62005247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.509009:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982446 is committed 00002000:00010000:0.0:1789760772.509191:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62001152 -> 62005248 00002000:00010000:0.0:1789760772.509196:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121120 -> 121128 00010000:00010000:0.0:1789760772.509209:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c481d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62001152->62005247] (req 62001152->62005247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.509222:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c481d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62001152->62005247] (req 62001152->62005247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.509231:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c481d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62001152->62005247] (req 62001152->62005247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.509239:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760772.509244:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c481d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62001152->62005247] (req 62001152->62005247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.509252:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c481d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62001152->62005247] (req 62001152->62005247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.509271:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c481d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62001152->62005247] (req 62001152->62005247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.509279:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c481d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62001152->62005247] (req 62001152->62005247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.509370:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0644a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554702336:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6012us (6344us total) trans 4294982446 rc 0/0 00000100:00100000:0.0:1789760772.509427:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15148 00000100:00100000:0.0:1789760772.509438:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715734030299 time 715748959224 state 0x2 ref 0 00000100:00100000:0.0:1789760772.515168:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554702464 00000100:00100000:0.0:1789760772.515183:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554702464 00000100:00100000:0.0:1789760772.515251:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715734030299 time 715748959224 state 0x0 ref 1 00000100:00100000:0.0:1789760772.515267:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715749219640 time 715748959224 state 0x1 ref 1 00000100:00100000:0.0:1789760772.515300:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715749219640 time 715760918142 state 0x0 ref 1 00000100:00100000:0.0:1789760772.515313:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554702464:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760772.515357:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62005248, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d928 00010000:00010000:0.0:1789760772.515366:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4824 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.515412:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760772.515417:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.515423:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4824 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62005248->62009343] (req 62005248->62009343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.515438:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4824 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62005248->62009343] (req 62005248->62009343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.522678:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982447 is committed 00002000:00010000:0.0:1789760772.523185:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62005248 -> 62009344 00002000:00010000:0.0:1789760772.523190:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121128 -> 121136 00010000:00010000:0.0:1789760772.523204:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4824 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62005248->62009343] (req 62005248->62009343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.523222:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4824 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62005248->62009343] (req 62005248->62009343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.523231:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4824 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62005248->62009343] (req 62005248->62009343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.523238:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760772.523243:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4824 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62005248->62009343] (req 62005248->62009343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.523253:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4824 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62005248->62009343] (req 62005248->62009343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.523274:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4824 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62005248->62009343] (req 62005248->62009343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.523283:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4824 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62005248->62009343] (req 62005248->62009343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.523456:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554702464:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8140us (9988us total) trans 4294982447 rc 0/0 00000100:00100000:0.0:1789760772.523469:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15149 00000100:00100000:0.0:1789760772.523480:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715749219640 time 715760918142 state 0x2 ref 0 00000100:00100000:2.0:1789760772.529113:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760772.529890:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715761178558 time 715775505315 state 0x0 ref 1 00000100:00100000:0.0:1789760772.529906:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554702592:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760772.529959:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62009344, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d930 00010000:00010000:0.0:1789760772.529974:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c482b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.529990:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760772.529995:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.530001:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c482b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62009344->62013439] (req 62009344->62013439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.530014:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c482b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62009344->62013439] (req 62009344->62013439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.535175:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982448 is committed 00002000:00010000:0.0:1789760772.535331:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62009344 -> 62013440 00002000:00010000:0.0:1789760772.535336:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121136 -> 121144 00010000:00010000:0.0:1789760772.535348:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c482b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62009344->62013439] (req 62009344->62013439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.535361:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c482b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62009344->62013439] (req 62009344->62013439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.535370:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c482b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62009344->62013439] (req 62009344->62013439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.535412:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760772.535417:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c482b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62009344->62013439] (req 62009344->62013439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.535427:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c482b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62009344->62013439] (req 62009344->62013439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.535446:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c482b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62009344->62013439] (req 62009344->62013439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.535453:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c482b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62009344->62013439] (req 62009344->62013439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.535932:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554702592:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6023us (6821us total) trans 4294982448 rc 0/0 00000100:00100000:0.0:1789760772.535943:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15150 00000100:00100000:0.0:1789760772.535952:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715761178558 time 715775505315 state 0x2 ref 0 00000100:00100000:3.0:1789760772.544930:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760772.545701:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554702720 00000100:00100000:0.0:1789760772.545778:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554702720 00000100:00100000:0.0:1789760772.545838:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715761178558 time 715775505315 state 0x0 ref 1 00000100:00100000:0.0:1789760772.545853:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715775765731 time 715775505315 state 0x1 ref 1 00000100:00100000:0.0:1789760772.545888:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715775765731 time 715791506720 state 0x0 ref 1 00000100:00100000:0.0:1789760772.545901:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06449c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554702720:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760772.545953:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62013440, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d938 00010000:00010000:0.0:1789760772.545963:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4832 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.545978:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760772.545982:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.545989:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4832 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62013440->62017535] (req 62013440->62017535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.546003:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4832 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62013440->62017535] (req 62013440->62017535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.550975:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982449 is committed 00002000:00010000:0.0:1789760772.551073:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62013440 -> 62017536 00002000:00010000:0.0:1789760772.551078:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121144 -> 121152 00010000:00010000:0.0:1789760772.551090:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4832 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62013440->62017535] (req 62013440->62017535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.551111:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4832 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62013440->62017535] (req 62013440->62017535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.551120:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4832 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62013440->62017535] (req 62013440->62017535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.551133:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760772.551138:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4832 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62013440->62017535] (req 62013440->62017535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.551147:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4832 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62013440->62017535] (req 62013440->62017535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.551167:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4832 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62013440->62017535] (req 62013440->62017535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.551175:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4832 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62013440->62017535] (req 62013440->62017535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.551264:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06449c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554702720:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5361us (6339us total) trans 4294982449 rc 0/0 00000100:00100000:0.0:1789760772.551274:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15151 00000100:00100000:0.0:1789760772.551283:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715775765731 time 715791506720 state 0x2 ref 0 00000100:00100000:2.0:1789760772.559961:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554702848 00000100:00100000:2.0:1789760772.559976:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554702848 00000100:00100000:2.0:1789760772.560041:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715775765731 time 715791506720 state 0x0 ref 1 00000100:00100000:2.0:1789760772.560057:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715791767136 time 715791506720 state 0x1 ref 1 00000100:00100000:2.0:1789760772.560194:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715791767136 time 715805750053 state 0x0 ref 1 00000100:00100000:2.0:1789760772.560211:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554702848:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760772.560261:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62017536, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d940 00010000:00010000:2.0:1789760772.560271:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4839 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.560288:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:2.0:1789760772.560293:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.560299:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4839 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62017536->62021631] (req 62017536->62021631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.560314:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4839 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62017536->62021631] (req 62017536->62021631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.565313:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982450 is committed 00002000:00010000:2.0:1789760772.565629:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62017536 -> 62021632 00002000:00010000:2.0:1789760772.565635:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121152 -> 121160 00010000:00010000:2.0:1789760772.565649:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4839 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62017536->62021631] (req 62017536->62021631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.565667:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4839 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62017536->62021631] (req 62017536->62021631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.565814:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4839 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62017536->62021631] (req 62017536->62021631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.565822:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760772.565827:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4839 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62017536->62021631] (req 62017536->62021631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.565836:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4839 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62017536->62021631] (req 62017536->62021631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.565856:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4839 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62017536->62021631] (req 62017536->62021631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.565864:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4839 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62017536->62021631] (req 62017536->62021631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.566003:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554702848:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5790us (8974us total) trans 4294982450 rc 0/0 00000100:00100000:2.0:1789760772.566015:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15152 00000100:00100000:2.0:1789760772.566024:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715791767136 time 715805750053 state 0x2 ref 0 00000100:00100000:2.0:1789760772.570228:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760772.570337:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554702976 00000100:00100000:0.0:1789760772.570347:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554702976 00000100:00100000:0.0:1789760772.570721:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715791767136 time 715805750053 state 0x0 ref 1 00000100:00100000:0.0:1789760772.570736:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715806010469 time 715805750053 state 0x1 ref 1 00000100:00100000:0.0:1789760772.570780:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715806010469 time 715816398437 state 0x0 ref 1 00000100:00100000:0.0:1789760772.570794:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554702976:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760772.570839:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62021632, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d948 00010000:00010000:0.0:1789760772.570848:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4840 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.570870:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760772.570874:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.570880:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4840 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62021632->62025727] (req 62021632->62025727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.570893:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4840 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62021632->62025727] (req 62021632->62025727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.580588:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982451 is committed 00000100:00100000:3.0:1789760772.590187:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760772.592786:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554703104 00000100:00100000:0.0:1789760772.592800:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554703104 00000100:00100000:0.0:1789760772.592863:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715806010469 time 715816398437 state 0x0 ref 1 00000100:00100000:0.0:1789760772.592879:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715816658853 time 715816398437 state 0x1 ref 1 00000100:00100000:0.0:1789760772.592920:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715816658853 time 715838537744 state 0x0 ref 1 00000100:00100000:0.0:1789760772.592934:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554703104:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760772.592982:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62025728, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d950 00010000:00010000:0.0:1789760772.592990:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4847 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.593007:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760772.593011:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.593017:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4847 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62025728->62029823] (req 62025728->62029823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.593031:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4847 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62025728->62029823] (req 62025728->62029823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.600733:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982452 is committed 00002000:00010000:0.0:1789760772.601502:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62025728 -> 62029824 00002000:00010000:0.0:1789760772.601573:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121168 -> 121176 00010000:00010000:0.0:1789760772.601589:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4847 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62025728->62029823] (req 62025728->62029823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.601603:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4847 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62025728->62029823] (req 62025728->62029823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.601611:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4847 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62025728->62029823] (req 62025728->62029823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.601619:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760772.601624:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4847 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62025728->62029823] (req 62025728->62029823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.601632:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4847 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62025728->62029823] (req 62025728->62029823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.601652:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4847 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62025728->62029823] (req 62025728->62029823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.601658:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c4847 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62025728->62029823] (req 62025728->62029823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.601769:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554703104:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8834us (11590us total) trans 4294982452 rc 0/0 00000100:00100000:0.0:1789760772.601780:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15154 00000100:00100000:0.0:1789760772.601789:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715816658853 time 715838537744 state 0x2 ref 0 00000100:00100000:2.0:1789760772.606489:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760772.606621:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554703232 00000100:00100000:3.0:1789760772.606632:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554703232 00000100:00100000:3.0:1789760772.606681:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715816658853 time 715838537744 state 0x0 ref 1 00000100:00100000:3.0:1789760772.606894:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715838798160 time 715838537744 state 0x1 ref 1 00000100:00100000:3.0:1789760772.606928:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715838798160 time 715852546085 state 0x0 ref 1 00000100:00100000:3.0:1789760772.606941:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554703232:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760772.606989:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62029824, mtime: 0x6aad9503, atime: 0x0, ctime: 0x6aad9503, blocks: 0x1d958 00010000:00010000:3.0:1789760772.606999:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c484e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.607015:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760772.607019:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.607026:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c484e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62029824->62033919] (req 62029824->62033919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.607042:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c484e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62029824->62033919] (req 62029824->62033919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.613589:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982453 is committed 00002000:00010000:3.0:1789760772.613697:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62029824 -> 62033920 00002000:00010000:3.0:1789760772.615026:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121176 -> 121184 00010000:00010000:3.0:1789760772.615168:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c484e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62029824->62033919] (req 62029824->62033919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.615183:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c484e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62029824->62033919] (req 62029824->62033919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.615193:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c484e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62029824->62033919] (req 62029824->62033919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.615200:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760772.615205:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c484e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62029824->62033919] (req 62029824->62033919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.615214:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c484e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62029824->62033919] (req 62029824->62033919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.615234:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c484e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62029824->62033919] (req 62029824->62033919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.615243:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c484e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62029824->62033919] (req 62029824->62033919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.615341:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554703232:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8397us (8855us total) trans 4294982453 rc 0/0 00000100:00100000:3.0:1789760772.615352:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15155 00000100:00100000:3.0:1789760772.615361:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715838798160 time 715852546085 state 0x2 ref 0 00000100:00100000:2.0:1789760772.621871:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760772.627646:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982454 is committed 00000100:00100000:0.0:1789760772.633827:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760772.648441:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982455 is committed 00000100:00100000:2.0:1789760772.660347:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760772.660472:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554703616 00000100:00100000:0.0:1789760772.660483:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554703616 00000100:00100000:0.0:1789760772.660548:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715868004468 time 715885303238 state 0x0 ref 1 00000100:00100000:0.0:1789760772.660565:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715885563654 time 715885303238 state 0x1 ref 1 00000100:00100000:0.0:1789760772.660598:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715885563654 time 715906216514 state 0x0 ref 1 00000100:00100000:0.0:1789760772.660610:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554703616:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760772.660654:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62042112, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1d970 00010000:00010000:0.0:1789760772.660664:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4863 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.660679:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760772.660683:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.660690:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4863 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62042112->62046207] (req 62042112->62046207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.660709:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4863 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62042112->62046207] (req 62042112->62046207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.668551:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982456 is committed 00002000:00010000:0.0:1789760772.668693:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62042112 -> 62046208 00002000:00010000:0.0:1789760772.668698:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121200 -> 121208 00010000:00010000:0.0:1789760772.668711:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4863 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62042112->62046207] (req 62042112->62046207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.668724:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4863 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62042112->62046207] (req 62042112->62046207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.668732:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4863 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62042112->62046207] (req 62042112->62046207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.668736:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760772.668740:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4863 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62042112->62046207] (req 62042112->62046207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.668746:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4863 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62042112->62046207] (req 62042112->62046207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.668772:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4863 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62042112->62046207] (req 62042112->62046207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.668779:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4863 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62042112->62046207] (req 62042112->62046207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.668913:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554703616:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8299us (8571us total) trans 4294982456 rc 0/0 00000100:00100000:0.0:1789760772.668927:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15158 00000100:00100000:0.0:1789760772.668936:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715885563654 time 715906216514 state 0x2 ref 0 00000100:00100000:2.0:1789760772.674034:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760772.674372:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554703744 00000100:00100000:3.0:1789760772.674561:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554703744 00000100:00100000:3.0:1789760772.674617:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715885563654 time 715906216514 state 0x0 ref 1 00000100:00100000:3.0:1789760772.674634:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715906476930 time 715906216514 state 0x1 ref 1 00000100:00100000:3.0:1789760772.674665:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715906476930 time 715920283649 state 0x0 ref 1 00000100:00100000:3.0:1789760772.674677:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554703744:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760772.674726:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62046208, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1d978 00010000:00010000:3.0:1789760772.674737:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c486a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.674761:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:3.0:1789760772.674766:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.674772:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c486a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62046208->62050303] (req 62046208->62050303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.674787:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c486a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62046208->62050303] (req 62046208->62050303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.687836:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982457 is committed 00002000:00010000:3.0:1789760772.688292:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62046208 -> 62050304 00002000:00010000:3.0:1789760772.688298:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121208 -> 121216 00010000:00010000:3.0:1789760772.688312:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c486a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62046208->62050303] (req 62046208->62050303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.688326:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c486a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62046208->62050303] (req 62046208->62050303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.688336:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c486a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62046208->62050303] (req 62046208->62050303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.688344:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760772.688349:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c486a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62046208->62050303] (req 62046208->62050303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.688357:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c486a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62046208->62050303] (req 62046208->62050303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.688399:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c486a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62046208->62050303] (req 62046208->62050303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.688408:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c486a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62046208->62050303] (req 62046208->62050303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.688546:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554703744:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13864us (14516us total) trans 4294982457 rc 0/0 00000100:00100000:3.0:1789760772.688557:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15159 00000100:00100000:3.0:1789760772.688567:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715906476930 time 715920283649 state 0x2 ref 0 00010000:00010000:0.0:1789760772.689634:0:6509:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760772.702185:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760772.718920:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982458 is committed 00000100:00100000:2.0:1789760772.723958:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:0.0:1789760772.737489:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982459 is committed 00000100:00100000:2.0:1789760772.744330:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760772.744436:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554704128 00000100:00100000:0.0:1789760772.744447:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554704128 00000100:00100000:0.0:1789760772.744503:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715951443252 time 715970337539 state 0x0 ref 1 00000100:00100000:0.0:1789760772.744521:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715970597955 time 715970337539 state 0x1 ref 1 00000100:00100000:0.0:1789760772.744557:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715970597955 time 715990174969 state 0x0 ref 1 00000100:00100000:0.0:1789760772.744570:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554704128:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760772.744616:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62058496, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1d990 00010000:00010000:0.0:1789760772.744626:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c487f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.744643:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=715s v=(0 0) 00010000:00010000:0.0:1789760772.744647:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.744653:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c487f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62058496->62062591] (req 62058496->62062591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.744674:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c487f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62058496->62062591] (req 62058496->62062591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.755239:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982460 is committed 00002000:00010000:0.0:1789760772.755356:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62058496 -> 62062592 00002000:00010000:0.0:1789760772.755361:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121232 -> 121240 00010000:00010000:0.0:1789760772.755373:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c487f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62058496->62062591] (req 62058496->62062591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.755426:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c487f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62058496->62062591] (req 62058496->62062591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.755436:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c487f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62058496->62062591] (req 62058496->62062591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.755444:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760772.755448:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c487f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62058496->62062591] (req 62058496->62062591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.755457:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c487f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62058496->62062591] (req 62058496->62062591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.755478:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c487f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62058496->62062591] (req 62058496->62062591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.755487:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c487f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62058496->62062591] (req 62058496->62062591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.755577:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554704128:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11005us (11252us total) trans 4294982460 rc 0/0 00000100:00100000:0.0:1789760772.755588:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15162 00000100:00100000:0.0:1789760772.755597:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715970597955 time 715990174969 state 0x2 ref 0 00000100:00100000:3.0:1789760772.763118:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760772.763345:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554704256 00000100:00100000:0.0:1789760772.763356:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554704256 00000100:00100000:0.0:1789760772.763441:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715970597955 time 715990174969 state 0x0 ref 1 00000100:00100000:0.0:1789760772.763456:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715990435385 time 715990174969 state 0x1 ref 1 00000100:00100000:0.0:1789760772.763477:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715990435385 time 716009095631 state 0x0 ref 1 00000100:00100000:0.0:1789760772.763490:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554704256:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760772.763536:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62062592, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1d998 00010000:00010000:0.0:1789760772.763546:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c4886 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.763562:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760772.763567:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.763573:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c4886 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62062592->62066687] (req 62062592->62066687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.763587:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c4886 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62062592->62066687] (req 62062592->62066687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.774215:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982461 is committed 00002000:00010000:0.0:1789760772.774614:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62062592 -> 62066688 00002000:00010000:0.0:1789760772.774618:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121240 -> 121248 00010000:00010000:0.0:1789760772.774629:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c4886 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62062592->62066687] (req 62062592->62066687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.774639:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c4886 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62062592->62066687] (req 62062592->62066687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.774646:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c4886 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62062592->62066687] (req 62062592->62066687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.774651:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760772.774654:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c4886 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62062592->62066687] (req 62062592->62066687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.774661:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c4886 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62062592->62066687] (req 62062592->62066687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.774677:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c4886 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62062592->62066687] (req 62062592->62066687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.774681:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c4886 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62062592->62066687] (req 62062592->62066687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.774781:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554704256:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11288us (11667us total) trans 4294982461 rc 0/0 00000100:00100000:0.0:1789760772.774792:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15163 00000100:00100000:0.0:1789760772.774802:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 715990435385 time 716009095631 state 0x2 ref 0 00000100:00100000:2.0:1789760772.780812:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760772.788349:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982462 is committed 00000100:00100000:2.0:1789760772.806292:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760772.814019:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982463 is committed 00000100:00100000:0.0:1789760772.820345:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760772.820728:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554704640 00000100:00100000:2.0:1789760772.820738:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554704640 00000100:00100000:2.0:1789760772.820801:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716026959726 time 716052290221 state 0x0 ref 1 00000100:00100000:2.0:1789760772.820816:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716052550637 time 716052290221 state 0x1 ref 1 00000100:00100000:2.0:1789760772.820849:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716052550637 time 716066467076 state 0x0 ref 1 00000100:00100000:2.0:1789760772.820862:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e54e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554704640:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760772.820910:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62074880, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1d9b0 00010000:00010000:2.0:1789760772.820919:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c489b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.820935:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760772.820940:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.820947:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c489b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62074880->62078975] (req 62074880->62078975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.820961:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c489b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62074880->62078975] (req 62074880->62078975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.826942:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982464 is committed 00002000:00010000:0.0:1789760772.827056:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62074880 -> 62078976 00002000:00010000:0.0:1789760772.827062:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121264 -> 121272 00010000:00010000:0.0:1789760772.827075:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c489b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62074880->62078975] (req 62074880->62078975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.827087:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c489b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62074880->62078975] (req 62074880->62078975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.827096:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c489b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62074880->62078975] (req 62074880->62078975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.827104:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760772.827109:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c489b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62074880->62078975] (req 62074880->62078975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.827118:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c489b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62074880->62078975] (req 62074880->62078975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.827149:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c489b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62074880->62078975] (req 62074880->62078975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.827156:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c489b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62074880->62078975] (req 62074880->62078975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.827251:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554704640:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6387us (6912us total) trans 4294982464 rc 0/0 00000100:00100000:0.0:1789760772.827261:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15166 00000100:00100000:0.0:1789760772.827271:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716052550637 time 716066467076 state 0x2 ref 0 00000100:00100000:2.0:1789760772.831874:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760772.831976:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554704768 00000100:00100000:0.0:1789760772.831987:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554704768 00000100:00100000:0.0:1789760772.832040:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716052550637 time 716066467076 state 0x0 ref 1 00000100:00100000:0.0:1789760772.832055:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716066727492 time 716066467076 state 0x1 ref 1 00000100:00100000:0.0:1789760772.832075:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716066727492 time 716077694489 state 0x0 ref 1 00000100:00100000:0.0:1789760772.832087:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554704768:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760772.832147:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62078976, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1d9b8 00010000:00010000:0.0:1789760772.832156:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c48a2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.832173:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760772.832178:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.832184:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c48a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62078976->62083071] (req 62078976->62083071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.832197:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c48a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62078976->62083071] (req 62078976->62083071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.841437:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982465 is committed 00002000:00010000:0.0:1789760772.841556:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62078976 -> 62083072 00002000:00010000:0.0:1789760772.841561:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121272 -> 121280 00010000:00010000:0.0:1789760772.841576:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c48a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62078976->62083071] (req 62078976->62083071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.841589:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c48a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62078976->62083071] (req 62078976->62083071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.841598:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c48a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62078976->62083071] (req 62078976->62083071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.841605:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760772.841610:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c48a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62078976->62083071] (req 62078976->62083071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.841619:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c48a2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62078976->62083071] (req 62078976->62083071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.841637:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c48a2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62078976->62083071] (req 62078976->62083071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.841644:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c48a2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62078976->62083071] (req 62078976->62083071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.841736:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554704768:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9645us (9865us total) trans 4294982465 rc 0/0 00000100:00100000:0.0:1789760772.841747:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15167 00000100:00100000:0.0:1789760772.841764:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716066727492 time 716077694489 state 0x2 ref 0 00000100:00100000:2.0:1789760772.845792:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760772.853688:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982466 is committed 00000100:00100000:3.0:1789760772.860928:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760772.866020:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982467 is committed 00000100:00100000:2.0:1789760772.870505:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760772.870600:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554705152 00000100:00100000:0.0:1789760772.870611:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554705152 00000100:00100000:0.0:1789760772.870667:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716091870083 time 716106749421 state 0x0 ref 1 00000100:00100000:0.0:1789760772.870682:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716107009837 time 716106749421 state 0x1 ref 1 00000100:00100000:0.0:1789760772.870708:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716107009837 time 716116326260 state 0x0 ref 1 00000100:00100000:0.0:1789760772.870719:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554705152:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760772.870772:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62091264, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1d9d0 00010000:00010000:0.0:1789760772.870781:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c48b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.870796:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760772.870800:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.870806:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c48b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62091264->62095359] (req 62091264->62095359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.870828:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c48b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62091264->62095359] (req 62091264->62095359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760772.875208:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982468 is committed 00002000:00010000:0.0:1789760772.875343:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62091264 -> 62095360 00002000:00010000:0.0:1789760772.875349:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121296 -> 121304 00010000:00010000:0.0:1789760772.875369:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c48b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62091264->62095359] (req 62091264->62095359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.875470:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c48b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62091264->62095359] (req 62091264->62095359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.875479:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c48b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62091264->62095359] (req 62091264->62095359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.875486:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760772.875490:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c48b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62091264->62095359] (req 62091264->62095359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.875498:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c48b7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62091264->62095359] (req 62091264->62095359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.875517:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c48b7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62091264->62095359] (req 62091264->62095359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.875525:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c48b7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62091264->62095359] (req 62091264->62095359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.875660:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554705152:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4937us (5159us total) trans 4294982468 rc 0/0 00000100:00100000:0.0:1789760772.875671:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15170 00000100:00100000:0.0:1789760772.875680:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716107009837 time 716116326260 state 0x2 ref 0 00000100:00100000:2.0:1789760772.878483:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760772.878613:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554705280 00000100:00100000:0.0:1789760772.878624:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554705280 00000100:00100000:0.0:1789760772.878674:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716107009837 time 716116326260 state 0x0 ref 1 00000100:00100000:0.0:1789760772.878689:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716116586676 time 716116326260 state 0x1 ref 1 00000100:00100000:0.0:1789760772.878710:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716116586676 time 716124329208 state 0x0 ref 1 00000100:00100000:0.0:1789760772.878721:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe1e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554705280:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760772.878779:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62095360, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1d9d8 00010000:00010000:0.0:1789760772.878789:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c48be lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.878805:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760772.878809:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.878813:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c48be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62095360->62099455] (req 62095360->62099455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.878823:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c48be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62095360->62099455] (req 62095360->62099455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.883641:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982469 is committed 00002000:00010000:0.0:1789760772.883768:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62095360 -> 62099456 00002000:00010000:0.0:1789760772.883773:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121304 -> 121312 00010000:00010000:0.0:1789760772.883787:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c48be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62095360->62099455] (req 62095360->62099455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.883804:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c48be lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62095360->62099455] (req 62095360->62099455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.883813:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c48be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62095360->62099455] (req 62095360->62099455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.883820:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760772.883824:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c48be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62095360->62099455] (req 62095360->62099455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.883833:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c48be lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62095360->62099455] (req 62095360->62099455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.883851:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c48be lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62095360->62099455] (req 62095360->62099455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.883859:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c48be lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62095360->62099455] (req 62095360->62099455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.883996:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe1e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554705280:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5271us (5516us total) trans 4294982469 rc 0/0 00000100:00100000:0.0:1789760772.884008:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15171 00000100:00100000:0.0:1789760772.884018:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716116586676 time 716124329208 state 0x2 ref 0 00000100:00100000:3.0:1789760772.888428:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760772.895964:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982470 is committed 00000100:00100000:2.0:1789760772.899515:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760772.905179:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982471 is committed 00000100:00100000:2.0:1789760772.919227:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760772.919839:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554705664 00000100:00100000:0.0:1789760772.919850:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554705664 00000100:00100000:0.0:1789760772.919899:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716134723353 time 716145368986 state 0x0 ref 1 00000100:00100000:0.0:1789760772.919912:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716145629402 time 716145368986 state 0x1 ref 1 00000100:00100000:0.0:1789760772.919931:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716145629402 time 716165550132 state 0x0 ref 1 00000100:00100000:0.0:1789760772.919940:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554705664:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760772.919981:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62107648, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1d9f0 00010000:00010000:0.0:1789760772.919990:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c48d3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.920004:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760772.920008:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760772.920013:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c48d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62107648->62111743] (req 62107648->62111743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.920026:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c48d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62107648->62111743] (req 62107648->62111743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760772.925473:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982472 is committed 00002000:00010000:0.0:1789760772.925594:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62107648 -> 62111744 00002000:00010000:0.0:1789760772.925600:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121328 -> 121336 00010000:00010000:0.0:1789760772.925615:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c48d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62107648->62111743] (req 62107648->62111743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.925627:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c48d3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62107648->62111743] (req 62107648->62111743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.925644:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c48d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62107648->62111743] (req 62107648->62111743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.925651:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760772.925656:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c48d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62107648->62111743] (req 62107648->62111743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.925665:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c48d3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62107648->62111743] (req 62107648->62111743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.925683:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c48d3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62107648->62111743] (req 62107648->62111743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760772.925691:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c48d3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62107648->62111743] (req 62107648->62111743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760772.925789:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554705664:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5845us (6570us total) trans 4294982472 rc 0/0 00000100:00100000:0.0:1789760772.925799:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15174 00000100:00100000:0.0:1789760772.925809:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716145629402 time 716165550132 state 0x2 ref 0 00000100:00100000:0.0:1789760772.931862:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760772.931971:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554705792 00000100:00100000:3.0:1789760772.931980:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554705792 00000100:00100000:3.0:1789760772.932033:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716145629402 time 716165550132 state 0x0 ref 1 00000100:00100000:3.0:1789760772.932047:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716165810548 time 716165550132 state 0x1 ref 1 00000100:00100000:3.0:1789760772.932116:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716165810548 time 716177733485 state 0x0 ref 1 00000100:00100000:3.0:1789760772.932140:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e54e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554705792:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760772.932184:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62111744, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1d9f8 00010000:00010000:3.0:1789760772.932193:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c48da lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.932208:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760772.932212:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.932219:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c48da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62111744->62115839] (req 62111744->62115839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.932233:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c48da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62111744->62115839] (req 62111744->62115839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.936250:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982473 is committed 00002000:00010000:3.0:1789760772.936476:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62111744 -> 62115840 00002000:00010000:3.0:1789760772.936481:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121336 -> 121344 00010000:00010000:3.0:1789760772.936496:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c48da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62111744->62115839] (req 62111744->62115839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.936979:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c48da lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62111744->62115839] (req 62111744->62115839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.936990:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c48da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62111744->62115839] (req 62111744->62115839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.936997:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760772.937002:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c48da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62111744->62115839] (req 62111744->62115839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.937011:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c48da lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62111744->62115839] (req 62111744->62115839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.937031:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c48da lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62111744->62115839] (req 62111744->62115839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.937045:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c48da lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62111744->62115839] (req 62111744->62115839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.937310:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554705792:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5178us (5452us total) trans 4294982473 rc 0/0 00000100:00100000:3.0:1789760772.937322:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15175 00000100:00100000:3.0:1789760772.937331:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716165810548 time 716177733485 state 0x2 ref 0 00000100:00100000:2.0:1789760772.941915:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:0.0:1789760772.946875:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982474 is committed 00000100:00100000:2.0:1789760772.953179:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:0.0:1789760772.957835:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982475 is committed 00000100:00100000:0.0:1789760772.963334:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760772.963498:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554706176 00000100:00100000:2.0:1789760772.963521:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554706176 00000100:00100000:2.0:1789760772.963576:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716188105304 time 716199453734 state 0x0 ref 1 00000100:00100000:2.0:1789760772.963592:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716199714150 time 716199453734 state 0x1 ref 1 00000100:00100000:2.0:1789760772.963619:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716199714150 time 716209237869 state 0x0 ref 1 00000100:00100000:2.0:1789760772.963636:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554706176:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760772.963681:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62124032, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1da10 00010000:00010000:2.0:1789760772.963690:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c48ef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.963706:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760772.963710:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760772.963716:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c48ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62124032->62128127] (req 62124032->62128127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.963730:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c48ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62124032->62128127] (req 62124032->62128127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.968161:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982476 is committed 00002000:00010000:2.0:1789760772.968320:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62124032 -> 62128128 00002000:00010000:2.0:1789760772.968324:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121360 -> 121368 00010000:00010000:2.0:1789760772.968338:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c48ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62124032->62128127] (req 62124032->62128127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.968351:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c48ef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62124032->62128127] (req 62124032->62128127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.968360:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c48ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62124032->62128127] (req 62124032->62128127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.968367:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760772.968373:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c48ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62124032->62128127] (req 62124032->62128127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.968605:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c48ef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62124032->62128127] (req 62124032->62128127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.968626:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c48ef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62124032->62128127] (req 62124032->62128127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760772.968633:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c48ef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62124032->62128127] (req 62124032->62128127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760772.968726:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554706176:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5087us (5398us total) trans 4294982476 rc 0/0 00000100:00100000:2.0:1789760772.968737:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15178 00000100:00100000:2.0:1789760772.968760:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716199714150 time 716209237869 state 0x2 ref 0 00000100:00100000:2.0:1789760772.972036:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760772.972128:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554706304 00000100:00100000:3.0:1789760772.972146:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554706304 00000100:00100000:3.0:1789760772.972201:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716199714150 time 716209237869 state 0x0 ref 1 00000100:00100000:3.0:1789760772.972216:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716209498285 time 716209237869 state 0x1 ref 1 00000100:00100000:3.0:1789760772.972249:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716209498285 time 716217867600 state 0x0 ref 1 00000100:00100000:3.0:1789760772.972261:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554706304:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760772.972307:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62128128, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1da18 00010000:00010000:3.0:1789760772.972317:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c48f6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.972334:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760772.972338:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760772.972345:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c48f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62128128->62132223] (req 62128128->62132223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.972363:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c48f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62128128->62132223] (req 62128128->62132223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760772.978163:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982477 is committed 00002000:00010000:3.0:1789760772.979046:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62128128 -> 62132224 00002000:00010000:3.0:1789760772.979053:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121368 -> 121376 00010000:00010000:3.0:1789760772.979066:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c48f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62128128->62132223] (req 62128128->62132223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.979078:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c48f6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62128128->62132223] (req 62128128->62132223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.979086:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c48f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62128128->62132223] (req 62128128->62132223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.979093:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760772.979098:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c48f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62128128->62132223] (req 62128128->62132223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.979108:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c48f6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62128128->62132223] (req 62128128->62132223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.979190:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c48f6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62128128->62132223] (req 62128128->62132223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760772.979199:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c48f6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62128128->62132223] (req 62128128->62132223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760772.979337:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554706304:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7073us (7306us total) trans 4294982477 rc 0/0 00000100:00100000:3.0:1789760772.979348:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15179 00000100:00100000:3.0:1789760772.979357:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716209498285 time 716217867600 state 0x2 ref 0 00000100:00100000:0.0:1789760772.983434:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:0.0:1789760772.987813:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982478 is committed 00000100:00100000:0.0:1789760772.992978:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:0.0:1789760772.999360:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982479 is committed 00000100:00100000:2.0:1789760773.003140:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760773.003279:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554706688 00000100:00100000:0.0:1789760773.003289:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554706688 00000100:00100000:0.0:1789760773.003362:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716229575850 time 716239793826 state 0x0 ref 1 00000100:00100000:0.0:1789760773.003402:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716240054242 time 716239793826 state 0x1 ref 1 00000100:00100000:0.0:1789760773.003425:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716240054242 time 716249043550 state 0x0 ref 1 00000100:00100000:0.0:1789760773.003436:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554706688:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760773.003482:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62140416, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1da30 00010000:00010000:0.0:1789760773.003492:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c490b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.003518:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760773.003523:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.003529:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c490b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62140416->62144511] (req 62140416->62144511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.003543:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c490b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62140416->62144511] (req 62140416->62144511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789760773.004253:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760773.007971:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982480 is committed 00002000:00010000:0.0:1789760773.008084:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62140416 -> 62144512 00002000:00010000:0.0:1789760773.008089:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121392 -> 121400 00010000:00010000:0.0:1789760773.008102:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c490b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62140416->62144511] (req 62140416->62144511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.008115:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c490b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62140416->62144511] (req 62140416->62144511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.008133:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c490b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62140416->62144511] (req 62140416->62144511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.008141:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760773.008147:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c490b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62140416->62144511] (req 62140416->62144511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.008156:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c490b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62140416->62144511] (req 62140416->62144511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.008176:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c490b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62140416->62144511] (req 62140416->62144511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.008184:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c490b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62140416->62144511] (req 62140416->62144511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.008282:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554706688:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4842us (5161us total) trans 4294982480 rc 0/0 00000100:00100000:0.0:1789760773.008294:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15182 00000100:00100000:0.0:1789760773.008303:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716240054242 time 716249043550 state 0x2 ref 0 00000100:00100000:0.0:1789760773.012271:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760773.012451:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554706816 00000100:00100000:3.0:1789760773.012461:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554706816 00000100:00100000:3.0:1789760773.012525:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716240054242 time 716249043550 state 0x0 ref 1 00000100:00100000:3.0:1789760773.012541:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716249303966 time 716249043550 state 0x1 ref 1 00000100:00100000:3.0:1789760773.012570:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716249303966 time 716258188906 state 0x0 ref 1 00000100:00100000:3.0:1789760773.012584:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e56300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554706816:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760773.012632:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62144512, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1da38 00010000:00010000:3.0:1789760773.012642:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4912 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.012658:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760773.012662:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.012669:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4912 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62144512->62148607] (req 62144512->62148607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.012683:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4912 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62144512->62148607] (req 62144512->62148607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760773.018725:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62144512 -> 62148608 00002000:00010000:3.0:1789760773.018731:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121400 -> 121408 00010000:00010000:3.0:1789760773.018745:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4912 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62144512->62148607] (req 62144512->62148607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.018769:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4912 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62144512->62148607] (req 62144512->62148607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.018779:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4912 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62144512->62148607] (req 62144512->62148607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.018786:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760773.018791:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4912 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62144512->62148607] (req 62144512->62148607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.018800:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4912 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62144512->62148607] (req 62144512->62148607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.018821:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4912 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62144512->62148607] (req 62144512->62148607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.018830:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4912 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62144512->62148607] (req 62144512->62148607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.018924:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e56300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554706816:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6338us (6660us total) trans 4294982481 rc 0/0 00000100:00100000:3.0:1789760773.018936:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15183 00000100:00100000:3.0:1789760773.018945:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716249303966 time 716258188906 state 0x2 ref 0 00000100:00100000:3.0:1789760773.023373:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760773.033896:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982482 is committed 00000100:00100000:2.0:1789760773.039912:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760773.045823:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982483 is committed 00000100:00100000:0.0:1789760773.051339:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760773.051608:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554707200 00000100:00100000:2.0:1789760773.051618:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554707200 00000100:00100000:2.0:1789760773.051673:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716270025289 time 716285835848 state 0x0 ref 1 00000100:00100000:2.0:1789760773.051689:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716286096264 time 716285835848 state 0x1 ref 1 00000100:00100000:2.0:1789760773.051727:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716286096264 time 716297344951 state 0x0 ref 1 00000100:00100000:2.0:1789760773.051740:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554707200:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760773.051910:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62156800, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1da50 00010000:00010000:2.0:1789760773.051919:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4927 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.051934:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760773.051938:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.051944:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4927 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62156800->62160895] (req 62156800->62160895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.051956:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4927 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62156800->62160895] (req 62156800->62160895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.058691:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982484 is committed 00002000:00010000:2.0:1789760773.059478:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62156800 -> 62160896 00002000:00010000:2.0:1789760773.059485:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121424 -> 121432 00010000:00010000:2.0:1789760773.059498:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4927 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62156800->62160895] (req 62156800->62160895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.059575:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4927 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62156800->62160895] (req 62156800->62160895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.059585:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4927 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62156800->62160895] (req 62156800->62160895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.059598:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760773.059670:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4927 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62156800->62160895] (req 62156800->62160895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.059682:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4927 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62156800->62160895] (req 62156800->62160895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.059771:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4927 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62156800->62160895] (req 62156800->62160895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.059780:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4927 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62156800->62160895] (req 62156800->62160895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.060020:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554707200:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8276us (8684us total) trans 4294982484 rc 0/0 00000100:00100000:2.0:1789760773.060032:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15186 00000100:00100000:2.0:1789760773.060042:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716286096264 time 716297344951 state 0x2 ref 0 00000100:00100000:2.0:1789760773.065734:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760773.065900:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554707328 00000100:00100000:3.0:1789760773.065911:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554707328 00000100:00100000:3.0:1789760773.065978:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716286096264 time 716297344951 state 0x0 ref 1 00000100:00100000:3.0:1789760773.065992:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716297605367 time 716297344951 state 0x1 ref 1 00000100:00100000:3.0:1789760773.066024:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716297605367 time 716311642549 state 0x0 ref 1 00000100:00100000:3.0:1789760773.066037:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554707328:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760773.066084:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62160896, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1da58 00010000:00010000:3.0:1789760773.066095:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c492e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.066112:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760773.066117:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.066123:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c492e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62160896->62164991] (req 62160896->62164991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.066143:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c492e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62160896->62164991] (req 62160896->62164991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789760773.072675:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62160896 -> 62164992 00002000:00010000:3.0:1789760773.072684:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121432 -> 121440 00010000:00010000:3.0:1789760773.072699:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c492e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62160896->62164991] (req 62160896->62164991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.072713:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c492e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62160896->62164991] (req 62160896->62164991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.072723:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c492e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62160896->62164991] (req 62160896->62164991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.072730:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760773.072735:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c492e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62160896->62164991] (req 62160896->62164991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.072745:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c492e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62160896->62164991] (req 62160896->62164991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.072782:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c492e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62160896->62164991] (req 62160896->62164991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.072861:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c492e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62160896->62164991] (req 62160896->62164991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.073010:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554707328:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6969us (7280us total) trans 4294982485 rc 0/0 00000100:00100000:3.0:1789760773.073022:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15187 00000100:00100000:3.0:1789760773.073032:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716297605367 time 716311642549 state 0x2 ref 0 00000100:00100000:2.0:1789760773.077087:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554707456 00000100:00100000:2.0:1789760773.077097:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554707456 00000100:00100000:2.0:1789760773.077216:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716297605367 time 716311642549 state 0x0 ref 1 00000100:00100000:2.0:1789760773.077231:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716311902965 time 716311642549 state 0x1 ref 1 00000100:00100000:2.0:1789760773.077298:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716311902965 time 716322915581 state 0x0 ref 1 00000100:00100000:2.0:1789760773.077311:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554707456:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760773.077357:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62164992, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1da60 00010000:00010000:2.0:1789760773.077366:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4935 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.077411:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760773.077416:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.077423:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4935 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62164992->62169087] (req 62164992->62169087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.077437:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4935 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62164992->62169087] (req 62164992->62169087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760773.082223:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62164992 -> 62169088 00002000:00010000:2.0:1789760773.082228:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121440 -> 121448 00010000:00010000:2.0:1789760773.082242:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4935 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62164992->62169087] (req 62164992->62169087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.082255:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4935 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62164992->62169087] (req 62164992->62169087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.082265:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4935 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62164992->62169087] (req 62164992->62169087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.082272:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760773.082277:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4935 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62164992->62169087] (req 62164992->62169087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.082287:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4935 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62164992->62169087] (req 62164992->62169087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.082307:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4935 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62164992->62169087] (req 62164992->62169087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.082315:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4935 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62164992->62169087] (req 62164992->62169087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.082653:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554707456:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5337us (5690us total) trans 4294982486 rc 0/0 00000100:00100000:2.0:1789760773.082666:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15188 00000100:00100000:2.0:1789760773.082677:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716311902965 time 716322915581 state 0x2 ref 0 00000100:00100000:0.0:1789760773.086680:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760773.091070:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982487 is committed 00000100:00100000:0.0:1789760773.096592:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:0.0:1789760773.104947:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982488 is committed 00000100:00100000:2.0:1789760773.110034:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760773.110278:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554707840 00000100:00100000:3.0:1789760773.110289:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554707840 00000100:00100000:3.0:1789760773.110347:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716332792451 time 716343196930 state 0x0 ref 1 00000100:00100000:3.0:1789760773.110364:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716343457346 time 716343196930 state 0x1 ref 1 00000100:00100000:2.0:1789760773.110598:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716343457346 time 716356213758 state 0x0 ref 1 00000100:00100000:2.0:1789760773.110612:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554707840:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760773.110659:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62177280, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1da78 00010000:00010000:2.0:1789760773.110668:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c494a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.110684:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760773.110689:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.110695:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c494a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62177280->62181375] (req 62177280->62181375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.110709:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c494a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62177280->62181375] (req 62177280->62181375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.115085:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982489 is committed 00002000:00010000:2.0:1789760773.115231:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62177280 -> 62181376 00002000:00010000:2.0:1789760773.115236:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121464 -> 121472 00010000:00010000:2.0:1789760773.115249:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c494a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62177280->62181375] (req 62177280->62181375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.115262:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c494a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62177280->62181375] (req 62177280->62181375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.115270:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c494a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62177280->62181375] (req 62177280->62181375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.115289:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760773.115293:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c494a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62177280->62181375] (req 62177280->62181375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.115302:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c494a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62177280->62181375] (req 62177280->62181375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.115320:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c494a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62177280->62181375] (req 62177280->62181375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.115333:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c494a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62177280->62181375] (req 62177280->62181375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.115484:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554707840:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4869us (5454us total) trans 4294982489 rc 0/0 00000100:00100000:2.0:1789760773.115493:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15191 00000100:00100000:2.0:1789760773.115502:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716343457346 time 716356213758 state 0x2 ref 0 00000100:00100000:0.0:1789760773.119143:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760773.119190:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554707968 00000100:00100000:2.0:1789760773.119200:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554707968 00000100:00100000:2.0:1789760773.119254:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716343457346 time 716356213758 state 0x0 ref 1 00000100:00100000:2.0:1789760773.119269:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716356474174 time 716356213758 state 0x1 ref 1 00000100:00100000:2.0:1789760773.119307:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716356474174 time 716364925045 state 0x0 ref 1 00000100:00100000:2.0:1789760773.119319:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554707968:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760773.119366:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62181376, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1da80 00010000:00010000:2.0:1789760773.119393:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4951 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.119409:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760773.119413:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.119419:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4951 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62181376->62185471] (req 62181376->62185471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.119438:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4951 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62181376->62185471] (req 62181376->62185471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.126086:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982490 is committed 00002000:00010000:2.0:1789760773.126843:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62181376 -> 62185472 00002000:00010000:2.0:1789760773.126850:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121472 -> 121480 00010000:00010000:2.0:1789760773.126863:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4951 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62181376->62185471] (req 62181376->62185471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.126877:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4951 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62181376->62185471] (req 62181376->62185471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.126886:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4951 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62181376->62185471] (req 62181376->62185471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.126893:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760773.126898:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4951 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62181376->62185471] (req 62181376->62185471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.126907:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4951 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62181376->62185471] (req 62181376->62185471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.126926:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4951 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62181376->62185471] (req 62181376->62185471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.126934:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4951 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62181376->62185471] (req 62181376->62185471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.127075:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554707968:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7753us (7943us total) trans 4294982490 rc 0/0 00000100:00100000:2.0:1789760773.127086:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15192 00000100:00100000:2.0:1789760773.127104:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716356474174 time 716364925045 state 0x2 ref 0 00000100:00100000:2.0:1789760773.133271:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554708096 00000100:00100000:2.0:1789760773.133281:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554708096 00000100:00100000:2.0:1789760773.133336:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716356474174 time 716364925045 state 0x0 ref 1 00000100:00100000:2.0:1789760773.133352:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716365185461 time 716364925045 state 0x1 ref 1 00000100:00100000:2.0:1789760773.133439:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716365185461 time 716379056114 state 0x0 ref 1 00000100:00100000:2.0:1789760773.133452:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554708096:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760773.133500:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62185472, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1da88 00010000:00010000:2.0:1789760773.133520:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4958 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.133536:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760773.133540:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.133547:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4958 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62185472->62189567] (req 62185472->62189567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.133562:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4958 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62185472->62189567] (req 62185472->62189567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760773.138535:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62185472 -> 62189568 00002000:00010000:2.0:1789760773.138540:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121480 -> 121488 00010000:00010000:2.0:1789760773.138553:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4958 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62185472->62189567] (req 62185472->62189567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.138565:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4958 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62185472->62189567] (req 62185472->62189567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.138574:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4958 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62185472->62189567] (req 62185472->62189567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.138580:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760773.138585:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4958 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62185472->62189567] (req 62185472->62189567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.138595:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4958 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62185472->62189567] (req 62185472->62189567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.138617:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4958 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62185472->62189567] (req 62185472->62189567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.138625:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4958 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62185472->62189567] (req 62185472->62189567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.138706:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554708096:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5252us (5635us total) trans 4294982491 rc 0/0 00000100:00100000:2.0:1789760773.138717:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15193 00000100:00100000:2.0:1789760773.138727:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716365185461 time 716379056114 state 0x2 ref 0 00000100:00100000:0.0:1789760773.147815:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554708224 00000100:00100000:0.0:1789760773.147824:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554708224 00000100:00100000:0.0:1789760773.147878:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716365185461 time 716379056114 state 0x0 ref 1 00000100:00100000:0.0:1789760773.147894:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716379316530 time 716379056114 state 0x1 ref 1 00000100:00100000:0.0:1789760773.147972:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716379316530 time 716393589127 state 0x0 ref 1 00000100:00100000:0.0:1789760773.147987:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554708224:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760773.148038:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62189568, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1da90 00010000:00010000:0.0:1789760773.148047:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c495f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.148065:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760773.148069:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.148077:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c495f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62189568->62193663] (req 62189568->62193663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.148091:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c495f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62189568->62193663] (req 62189568->62193663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.153375:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982492 is committed 00002000:00010000:0.0:1789760773.153589:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62189568 -> 62193664 00002000:00010000:0.0:1789760773.153595:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121488 -> 121496 00010000:00010000:0.0:1789760773.153613:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c495f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62189568->62193663] (req 62189568->62193663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.153744:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c495f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62189568->62193663] (req 62189568->62193663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.153763:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c495f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62189568->62193663] (req 62189568->62193663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.153771:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760773.153776:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c495f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62189568->62193663] (req 62189568->62193663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.153785:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c495f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62189568->62193663] (req 62189568->62193663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.153804:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c495f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62189568->62193663] (req 62189568->62193663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.153817:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c495f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62189568->62193663] (req 62189568->62193663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.153905:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554708224:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5917us (6461us total) trans 4294982492 rc 0/0 00000100:00100000:0.0:1789760773.153915:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15194 00000100:00100000:0.0:1789760773.153925:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716379316530 time 716393589127 state 0x2 ref 0 00000100:00100000:0.0:1789760773.159783:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760773.159920:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554708352 00000100:00100000:2.0:1789760773.159929:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554708352 00000100:00100000:2.0:1789760773.159978:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716379316530 time 716393589127 state 0x0 ref 1 00000100:00100000:2.0:1789760773.159992:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716393849543 time 716393589127 state 0x1 ref 1 00000100:00100000:2.0:1789760773.160056:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716393849543 time 716405674552 state 0x0 ref 1 00000100:00100000:2.0:1789760773.160069:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e54a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554708352:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760773.160113:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62193664, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1da98 00010000:00010000:2.0:1789760773.160122:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4966 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.160147:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760773.160152:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.160158:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4966 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62193664->62197759] (req 62193664->62197759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.160171:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4966 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62193664->62197759] (req 62193664->62197759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760773.167946:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62193664 -> 62197760 00002000:00010000:2.0:1789760773.167956:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121496 -> 121504 00010000:00010000:2.0:1789760773.167971:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4966 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62193664->62197759] (req 62193664->62197759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.167985:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4966 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62193664->62197759] (req 62193664->62197759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.167995:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4966 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62193664->62197759] (req 62193664->62197759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.168003:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760773.168008:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4966 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62193664->62197759] (req 62193664->62197759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.168017:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4966 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62193664->62197759] (req 62193664->62197759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.168035:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4966 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62193664->62197759] (req 62193664->62197759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.168043:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4966 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62193664->62197759] (req 62193664->62197759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.168257:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554708352:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8185us (8478us total) trans 4294982493 rc 0/0 00000100:00100000:2.0:1789760773.168269:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15195 00000100:00100000:2.0:1789760773.168277:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716393849543 time 716405674552 state 0x2 ref 0 00000100:00100000:2.0:1789760773.180253:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554708480 00000100:00100000:2.0:1789760773.180263:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554708480 00000100:00100000:2.0:1789760773.180319:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716393849543 time 716405674552 state 0x0 ref 1 00000100:00100000:2.0:1789760773.180334:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716405934968 time 716405674552 state 0x1 ref 1 00000100:00100000:2.0:1789760773.180372:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716405934968 time 716425989784 state 0x0 ref 1 00000100:00100000:2.0:1789760773.180581:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554708480:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760773.180711:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62197760, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1daa0 00010000:00010000:2.0:1789760773.180721:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c496d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.180740:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760773.180745:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.180762:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c496d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62197760->62201855] (req 62197760->62201855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.180776:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c496d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62197760->62201855] (req 62197760->62201855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760773.188062:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62197760 -> 62201856 00002000:00010000:2.0:1789760773.188070:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121504 -> 121512 00010000:00010000:2.0:1789760773.188082:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c496d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62197760->62201855] (req 62197760->62201855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.188210:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c496d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62197760->62201855] (req 62197760->62201855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.188220:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c496d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62197760->62201855] (req 62197760->62201855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.188228:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760773.188232:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c496d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62197760->62201855] (req 62197760->62201855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.188241:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c496d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62197760->62201855] (req 62197760->62201855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.188260:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c496d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62197760->62201855] (req 62197760->62201855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.188268:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c496d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62197760->62201855] (req 62197760->62201855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.188862:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554708480:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8281us (8816us total) trans 4294982494 rc 0/0 00000100:00100000:2.0:1789760773.188875:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15196 00000100:00100000:2.0:1789760773.188884:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716405934968 time 716425989784 state 0x2 ref 0 00000100:00100000:2.0:1789760773.195865:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554708608 00000100:00100000:2.0:1789760773.195875:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554708608 00000100:00100000:2.0:1789760773.195930:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716405934968 time 716425989784 state 0x0 ref 1 00000100:00100000:2.0:1789760773.195947:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716426250200 time 716425989784 state 0x1 ref 1 00000100:00100000:2.0:1789760773.195986:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716426250200 time 716441603863 state 0x0 ref 1 00000100:00100000:2.0:1789760773.195999:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554708608:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760773.196049:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62201856, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1daa8 00010000:00010000:2.0:1789760773.196058:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4974 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.196075:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760773.196080:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.196092:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4974 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62201856->62205951] (req 62201856->62205951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.196107:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4974 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62201856->62205951] (req 62201856->62205951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760773.206495:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62201856 -> 62205952 00002000:00010000:0.0:1789760773.206501:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121512 -> 121520 00010000:00010000:0.0:1789760773.206514:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4974 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62201856->62205951] (req 62201856->62205951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.206526:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4974 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62201856->62205951] (req 62201856->62205951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.206536:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4974 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62201856->62205951] (req 62201856->62205951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.206543:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760773.206548:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4974 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62201856->62205951] (req 62201856->62205951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.206557:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4974 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62201856->62205951] (req 62201856->62205951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.206577:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4974 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62201856->62205951] (req 62201856->62205951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.206586:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4974 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62201856->62205951] (req 62201856->62205951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.206673:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554708608:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10672us (11060us total) trans 4294982495 rc 0/0 00000100:00100000:0.0:1789760773.206682:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15197 00000100:00100000:0.0:1789760773.206691:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716426250200 time 716441603863 state 0x2 ref 0 00000100:00100000:0.0:1789760773.211959:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760773.220280:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982496 is committed 00000100:00100000:0.0:1789760773.225193:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554708864 00000100:00100000:0.0:1789760773.225204:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554708864 00000100:00100000:0.0:1789760773.225267:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716441864279 time 716457791639 state 0x0 ref 1 00000100:00100000:0.0:1789760773.225282:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716458052055 time 716457791639 state 0x1 ref 1 00000100:00100000:0.0:1789760773.225314:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716458052055 time 716470933272 state 0x0 ref 1 00000100:00100000:0.0:1789760773.225327:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554708864:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760773.225402:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62210048, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1dab8 00010000:00010000:0.0:1789760773.225412:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4982 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.225433:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760773.225437:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.225443:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4982 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62210048->62214143] (req 62210048->62214143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.225456:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4982 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62210048->62214143] (req 62210048->62214143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760773.232719:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62210048 -> 62214144 00002000:00010000:0.0:1789760773.232724:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121528 -> 121536 00010000:00010000:0.0:1789760773.232755:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4982 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62210048->62214143] (req 62210048->62214143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.232768:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4982 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62210048->62214143] (req 62210048->62214143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.232777:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4982 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62210048->62214143] (req 62210048->62214143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.232785:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760773.232790:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4982 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62210048->62214143] (req 62210048->62214143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.232798:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4982 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62210048->62214143] (req 62210048->62214143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.232926:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4982 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62210048->62214143] (req 62210048->62214143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.232934:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4982 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62210048->62214143] (req 62210048->62214143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.233061:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554708864:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7731us (8116us total) trans 4294982497 rc 0/0 00000100:00100000:0.0:1789760773.233070:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15199 00000100:00100000:0.0:1789760773.233079:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716458052055 time 716470933272 state 0x2 ref 0 00000100:00100000:2.0:1789760773.238938:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:2.0:1789760773.249969:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982498 is committed 00000100:00100000:0.0:1789760773.261823:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554709120 00000100:00100000:0.0:1789760773.261833:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554709120 00000100:00100000:0.0:1789760773.261888:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716471193688 time 716484755135 state 0x0 ref 1 00000100:00100000:0.0:1789760773.261903:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716485015551 time 716484755135 state 0x1 ref 1 00000100:00100000:0.0:1789760773.261940:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716485015551 time 716507557610 state 0x0 ref 1 00000100:00100000:0.0:1789760773.261953:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554709120:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760773.262001:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62218240, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1dac8 00010000:00010000:0.0:1789760773.262011:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4990 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.262026:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760773.262032:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.262038:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4990 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62218240->62222335] (req 62218240->62222335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.262052:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4990 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62218240->62222335] (req 62218240->62222335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789760773.274977:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62218240 -> 62222336 00002000:00010000:0.0:1789760773.274986:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121544 -> 121552 00010000:00010000:0.0:1789760773.274999:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4990 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62218240->62222335] (req 62218240->62222335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.275013:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4990 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62218240->62222335] (req 62218240->62222335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.275023:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4990 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62218240->62222335] (req 62218240->62222335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.275030:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760773.275035:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4990 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62218240->62222335] (req 62218240->62222335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.275044:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4990 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62218240->62222335] (req 62218240->62222335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.275063:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4990 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62218240->62222335] (req 62218240->62222335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.275077:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c4990 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62218240->62222335] (req 62218240->62222335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.275650:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554709120:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13693us (14137us total) trans 4294982499 rc 0/0 00000100:00100000:0.0:1789760773.275661:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15201 00000100:00100000:0.0:1789760773.275671:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716485015551 time 716507557610 state 0x2 ref 0 00000100:00100000:0.0:1789760773.282657:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00002000:00010000:0.0:1789760773.294003:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62222336 -> 62226432 00002000:00010000:0.0:1789760773.294009:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121552 -> 121560 00010000:00010000:0.0:1789760773.294021:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4997 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62222336->62226431] (req 62222336->62226431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.294034:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4997 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62222336->62226431] (req 62222336->62226431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.294043:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4997 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62222336->62226431] (req 62222336->62226431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.294052:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760773.294057:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4997 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62222336->62226431] (req 62222336->62226431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.294066:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4997 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62222336->62226431] (req 62222336->62226431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.294086:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4997 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62222336->62226431] (req 62222336->62226431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.294094:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4997 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62222336->62226431] (req 62222336->62226431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.294189:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e56680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554709248:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11207us (11537us total) trans 4294982500 rc 0/0 00000100:00100000:0.0:1789760773.294199:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15202 00000100:00100000:0.0:1789760773.294208:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716507818026 time 716528585754 state 0x2 ref 0 00000100:00100000:2.0:1789760773.297809:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554709376 00000100:00100000:2.0:1789760773.297818:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554709376 00000100:00100000:2.0:1789760773.297872:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716507818026 time 716528585754 state 0x0 ref 1 00000100:00100000:2.0:1789760773.297887:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716528846170 time 716528585754 state 0x1 ref 1 00000100:00100000:2.0:1789760773.297924:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716528846170 time 716543542635 state 0x0 ref 1 00000100:00100000:2.0:1789760773.297937:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554709376:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760773.297984:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62226432, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1dad8 00010000:00010000:2.0:1789760773.297994:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c499e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.298010:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760773.298014:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.298021:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c499e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62226432->62230527] (req 62226432->62230527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.298034:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c499e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62226432->62230527] (req 62226432->62230527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789760773.310330:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62226432 -> 62230528 00002000:00010000:2.0:1789760773.310336:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121560 -> 121568 00010000:00010000:2.0:1789760773.310348:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c499e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62226432->62230527] (req 62226432->62230527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.310361:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c499e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62226432->62230527] (req 62226432->62230527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.310370:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c499e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62226432->62230527] (req 62226432->62230527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.310432:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760773.310437:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c499e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62226432->62230527] (req 62226432->62230527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.310446:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c499e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62226432->62230527] (req 62226432->62230527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.310465:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c499e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62226432->62230527] (req 62226432->62230527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.310472:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c499e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62226432->62230527] (req 62226432->62230527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.310888:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554709376:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12947us (13192us total) trans 4294982501 rc 0/0 00000100:00100000:3.0:1789760773.310899:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15203 00000100:00100000:3.0:1789760773.310908:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716528846170 time 716543542635 state 0x2 ref 0 00000100:00100000:3.0:1789760773.314654:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554709504 00000100:00100000:3.0:1789760773.314664:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554709504 00000100:00100000:3.0:1789760773.314718:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716528846170 time 716543542635 state 0x0 ref 1 00000100:00100000:3.0:1789760773.314733:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716543803051 time 716543542635 state 0x1 ref 1 00000100:00100000:3.0:1789760773.314764:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716543803051 time 716560382743 state 0x0 ref 1 00000100:00100000:3.0:1789760773.314776:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554709504:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760773.314822:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62230528, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1dae0 00010000:00010000:3.0:1789760773.314833:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c49a5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.314848:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760773.314852:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.314859:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c49a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62230528->62234623] (req 62230528->62234623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.314872:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c49a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62230528->62234623] (req 62230528->62234623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.326074:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982502 is committed 00002000:00010000:3.0:1789760773.326260:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62230528 -> 62234624 00002000:00010000:3.0:1789760773.326266:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121568 -> 121576 00010000:00010000:3.0:1789760773.326280:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c49a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62230528->62234623] (req 62230528->62234623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.326294:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c49a5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62230528->62234623] (req 62230528->62234623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.326303:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c49a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62230528->62234623] (req 62230528->62234623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.326311:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760773.326325:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c49a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62230528->62234623] (req 62230528->62234623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.326336:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c49a5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62230528->62234623] (req 62230528->62234623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.326358:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c49a5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62230528->62234623] (req 62230528->62234623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.326366:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c49a5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62230528->62234623] (req 62230528->62234623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.326474:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554709504:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11695us (11996us total) trans 4294982502 rc 0/0 00000100:00100000:3.0:1789760773.326485:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15204 00000100:00100000:3.0:1789760773.326494:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716543803051 time 716560382743 state 0x2 ref 0 00000100:00100000:0.0:1789760773.332652:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760773.343191:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982503 is committed 00000100:00100000:0.0:1789760773.354813:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760773.363174:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982504 is committed 00002000:00010000:1.0:1789760773.366454:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121584 -> 121592 00010000:00010000:1.0:1789760773.366468:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c49b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62238720->62242815] (req 62238720->62242815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.366482:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c49b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62238720->62242815] (req 62238720->62242815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.366492:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c49b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62238720->62242815] (req 62238720->62242815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.366500:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760773.366505:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c49b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62238720->62242815] (req 62238720->62242815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.366514:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c49b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62238720->62242815] (req 62238720->62242815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.366533:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c49b3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62238720->62242815] (req 62238720->62242815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.366541:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c49b3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62238720->62242815] (req 62238720->62242815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.366682:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554709760:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11620us (11873us total) trans 4294982504 rc 0/0 00000100:00100000:1.0:1789760773.366693:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15206 00000100:00100000:1.0:1789760773.366703:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716578788458 time 716600663236 state 0x2 ref 0 00000100:00100000:0.0:1789760773.372726:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760773.372845:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554709888 00000100:00100000:3.0:1789760773.372854:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554709888 00000100:00100000:3.0:1789760773.372909:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716578788458 time 716600663236 state 0x0 ref 1 00000100:00100000:3.0:1789760773.372923:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716600923652 time 716600663236 state 0x1 ref 1 00000100:00100000:3.0:1789760773.372994:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716600923652 time 716618612096 state 0x0 ref 1 00000100:00100000:3.0:1789760773.373006:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e57100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554709888:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760773.373053:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62242816, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1daf8 00010000:00010000:3.0:1789760773.373063:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c49ba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.373085:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760773.373090:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.373096:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c49ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62242816->62246911] (req 62242816->62246911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.373110:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c49ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62242816->62246911] (req 62242816->62246911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.374862:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760773.374897:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760773.374927:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760773.374947:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760773.375016:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760773.375041:0:6521:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554710144 00000100:00100000:2.0:1789760773.375049:0:6521:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554710144 00000100:00100000:2.0:1789760773.375101:0:6521:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554710400 00000100:00100000:2.0:1789760773.375105:0:6521:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554710400 00000100:00100000:2.0:1789760773.375119:0:6521:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554710272 00000100:00100000:2.0:1789760773.375121:0:6521:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554710272 00000100:00100000:2.0:1789760773.375138:0:6521:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554710528 00000100:00100000:2.0:1789760773.375141:0:6521:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554710528 00000100:00100000:2.0:1789760773.375156:0:6521:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 865 00000100:00100000:2.0:1789760773.375165:0:6521:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+8:2413:x1876699554710144:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:1.0:1789760773.375279:0:11089:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 866 00000100:00100000:1.0:1789760773.375289:0:11089:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2413:x1876699554710400:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:3.0:1789760773.375357:0:7470:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 867 00000100:00100000:3.0:1789760773.375366:0:7470:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554710272:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:2.0:1789760773.376733:0:6521:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+8:2413:x1876699554710144:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 1560us (1866us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760773.376744:0:6521:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 865 00000100:00100000:2.0:1789760773.376786:0:6521:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 868 00000100:00100000:2.0:1789760773.376797:0:6521:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2413:x1876699554710528:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:1.0:1789760773.377004:0:11089:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2413:x1876699554710400:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 1715us (2104us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760773.377013:0:11089:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 866 00000100:00100000:3.0:1789760773.377094:0:7470:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554710272:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 1727us (2144us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760773.377102:0:7470:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 867 00000100:00100000:3.0:1789760773.377227:0:6515:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554710016 00000100:00100000:3.0:1789760773.377235:0:6515:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554710016 00000100:00100000:3.0:1789760773.377283:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 468 00000100:00100000:3.0:1789760773.377290:0:6515:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc320b4850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9765eb0b-55b9-40b6-bfdf-ffee156c3226+13:2413:x1876699554710016:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:3.0:1789760773.377319:0:6515:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc320b4850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9765eb0b-55b9-40b6-bfdf-ffee156c3226+13:2413:x1876699554710016:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 29us (2391us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760773.377324:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 468 00000100:00080000:3.0:1789760773.377369:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789760773.377374:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760773.377679:0:6521:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2413:x1876699554710528:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 883us (2660us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760773.377687:0:6521:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 868 00000001:00080000:1.0:1789760773.386870:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982505 is committed 00002000:00010000:0.0:1789760773.387028:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62242816 -> 62246912 00002000:00010000:0.0:1789760773.387032:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121592 -> 121600 00010000:00010000:0.0:1789760773.387045:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c49ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62242816->62246911] (req 62242816->62246911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.387057:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c49ba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62242816->62246911] (req 62242816->62246911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.387065:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c49ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62242816->62246911] (req 62242816->62246911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.387077:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760773.387082:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c49ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62242816->62246911] (req 62242816->62246911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.387091:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c49ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62242816->62246911] (req 62242816->62246911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.387110:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c49ba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62242816->62246911] (req 62242816->62246911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.387249:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c49ba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62242816->62246911] (req 62242816->62246911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.387413:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e57100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554709888:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14372us (14661us total) trans 4294982505 rc 0/0 00000100:00100000:0.0:1789760773.387424:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15207 00000100:00100000:0.0:1789760773.387434:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716600923652 time 716618612096 state 0x2 ref 0 00000100:00100000:2.0:1789760773.392086:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760773.392634:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554710656 00000100:00100000:3.0:1789760773.392644:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554710656 00000100:00100000:3.0:1789760773.392698:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716600923652 time 716618612096 state 0x0 ref 1 00000100:00100000:3.0:1789760773.392715:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716618872512 time 716618612096 state 0x1 ref 1 00000100:00100000:3.0:1789760773.392757:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716618872512 time 716638367224 state 0x0 ref 1 00000100:00100000:3.0:1789760773.392769:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554710656:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760773.392825:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62246912, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1db00 00010000:00010000:3.0:1789760773.392834:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c49c1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.392849:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760773.392853:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.392860:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c49c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62246912->62251007] (req 62246912->62251007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.392874:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c49c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62246912->62251007] (req 62246912->62251007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760773.398627:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982506 is committed 00002000:00010000:3.0:1789760773.398739:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62246912 -> 62251008 00002000:00010000:3.0:1789760773.398744:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121600 -> 121608 00010000:00010000:3.0:1789760773.398770:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c49c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62246912->62251007] (req 62246912->62251007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.398783:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c49c1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62246912->62251007] (req 62246912->62251007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.398793:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c49c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62246912->62251007] (req 62246912->62251007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.398802:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760773.398816:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c49c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62246912->62251007] (req 62246912->62251007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.398825:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c49c1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62246912->62251007] (req 62246912->62251007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.398843:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c49c1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62246912->62251007] (req 62246912->62251007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.398851:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c49c1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62246912->62251007] (req 62246912->62251007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.398941:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554710656:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6170us (6861us total) trans 4294982506 rc 0/0 00000100:00100000:3.0:1789760773.398951:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15208 00000100:00100000:3.0:1789760773.398960:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716618872512 time 716638367224 state 0x2 ref 0 00000100:00100000:1.0:1789760773.403455:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760773.403629:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554710784 00000100:00100000:0.0:1789760773.403639:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554710784 00000100:00100000:0.0:1789760773.403691:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716618872512 time 716638367224 state 0x0 ref 1 00000100:00100000:0.0:1789760773.403707:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716638627640 time 716638367224 state 0x1 ref 1 00000100:00100000:0.0:1789760773.403738:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716638627640 time 716649356757 state 0x0 ref 1 00000100:00100000:0.0:1789760773.403763:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554710784:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760773.403828:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62251008, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1db08 00010000:00010000:0.0:1789760773.403838:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c49c8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.403862:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760773.403867:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.403873:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c49c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62251008->62255103] (req 62251008->62255103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.403896:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c49c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62251008->62255103] (req 62251008->62255103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.409419:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982507 is committed 00002000:00010000:0.0:1789760773.412669:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62251008 -> 62255104 00002000:00010000:0.0:1789760773.412680:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121608 -> 121616 00010000:00010000:0.0:1789760773.412696:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c49c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62251008->62255103] (req 62251008->62255103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.412709:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c49c8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62251008->62255103] (req 62251008->62255103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.412719:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c49c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62251008->62255103] (req 62251008->62255103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.412726:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760773.412733:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c49c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62251008->62255103] (req 62251008->62255103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.412743:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c49c8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62251008->62255103] (req 62251008->62255103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.412773:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c49c8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62251008->62255103] (req 62251008->62255103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.412781:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c49c8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62251008->62255103] (req 62251008->62255103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.413118:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554710784:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9352us (9669us total) trans 4294982507 rc 0/0 00000100:00100000:0.0:1789760773.413473:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15209 00000100:00100000:0.0:1789760773.413483:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716638627640 time 716649356757 state 0x2 ref 0 00000100:00100000:0.0:1789760773.416409:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760773.416493:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554710912 00000100:00100000:1.0:1789760773.416511:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554710912 00000100:00100000:1.0:1789760773.416567:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716638627640 time 716649356757 state 0x0 ref 1 00000100:00100000:1.0:1789760773.416584:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716649617173 time 716649356757 state 0x1 ref 1 00000100:00100000:1.0:1789760773.416806:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716649617173 time 716662422744 state 0x0 ref 1 00000100:00100000:1.0:1789760773.416831:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554710912:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760773.416884:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62255104, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1db10 00010000:00010000:1.0:1789760773.416894:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c49cf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.416912:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760773.416917:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.416924:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c49cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62255104->62259199] (req 62255104->62259199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.416945:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c49cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62255104->62259199] (req 62255104->62259199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.422689:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982508 is committed 00002000:00010000:1.0:1789760773.422941:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62255104 -> 62259200 00002000:00010000:1.0:1789760773.422946:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121616 -> 121624 00010000:00010000:1.0:1789760773.422959:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c49cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62255104->62259199] (req 62255104->62259199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.422972:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c49cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62255104->62259199] (req 62255104->62259199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.422982:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c49cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62255104->62259199] (req 62255104->62259199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.422989:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760773.422995:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c49cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62255104->62259199] (req 62255104->62259199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.423004:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c49cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62255104->62259199] (req 62255104->62259199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.423023:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c49cf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62255104->62259199] (req 62255104->62259199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.423032:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c49cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62255104->62259199] (req 62255104->62259199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.423277:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554710912:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6443us (6903us total) trans 4294982508 rc 0/0 00000100:00100000:1.0:1789760773.423288:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15210 00000100:00100000:1.0:1789760773.423297:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716649617173 time 716662422744 state 0x2 ref 0 00000100:00100000:2.0:1789760773.427464:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760773.427503:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554711040 00000100:00100000:0.0:1789760773.427513:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554711040 00000100:00100000:0.0:1789760773.427565:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716649617173 time 716662422744 state 0x0 ref 1 00000100:00100000:0.0:1789760773.427580:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716662683160 time 716662422744 state 0x1 ref 1 00000100:00100000:0.0:1789760773.428301:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716662683160 time 716673915172 state 0x0 ref 1 00000100:00100000:0.0:1789760773.428318:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554711040:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760773.428368:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62259200, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1db18 00010000:00010000:0.0:1789760773.428398:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c49d6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.428415:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760773.428423:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.428428:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c49d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62259200->62263295] (req 62259200->62263295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.428443:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c49d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62259200->62263295] (req 62259200->62263295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.433898:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982509 is committed 00002000:00010000:0.0:1789760773.434019:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62259200 -> 62263296 00002000:00010000:0.0:1789760773.434025:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121624 -> 121632 00010000:00010000:0.0:1789760773.434038:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c49d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62259200->62263295] (req 62259200->62263295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.434050:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c49d6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62259200->62263295] (req 62259200->62263295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.434060:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c49d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62259200->62263295] (req 62259200->62263295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.434067:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760773.434072:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c49d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62259200->62263295] (req 62259200->62263295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.434086:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c49d6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62259200->62263295] (req 62259200->62263295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.434105:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c49d6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62259200->62263295] (req 62259200->62263295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.434112:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c49d6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62259200->62263295] (req 62259200->62263295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.434263:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554711040:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5943us (6803us total) trans 4294982509 rc 0/0 00000100:00100000:0.0:1789760773.434274:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15211 00000100:00100000:0.0:1789760773.434283:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716662683160 time 716673915172 state 0x2 ref 0 00000100:00100000:0.0:1789760773.440038:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760773.440147:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554711168 00000100:00100000:3.0:1789760773.440157:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554711168 00000100:00100000:3.0:1789760773.440215:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716662683160 time 716673915172 state 0x0 ref 1 00000100:00100000:3.0:1789760773.440231:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716674175588 time 716673915172 state 0x1 ref 1 00000100:00100000:3.0:1789760773.440271:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716674175588 time 716685888915 state 0x0 ref 1 00000100:00100000:3.0:1789760773.440288:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554711168:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760773.440334:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62263296, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1db20 00010000:00010000:3.0:1789760773.440343:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c49dd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.440358:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760773.440362:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.440367:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c49dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62263296->62267391] (req 62263296->62267391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.440643:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c49dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62263296->62267391] (req 62263296->62267391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.451635:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982510 is committed 00002000:00010000:2.0:1789760773.452181:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62263296 -> 62267392 00002000:00010000:2.0:1789760773.452187:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121632 -> 121640 00010000:00010000:2.0:1789760773.452201:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c49dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62263296->62267391] (req 62263296->62267391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.452214:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c49dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62263296->62267391] (req 62263296->62267391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.452223:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c49dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62263296->62267391] (req 62263296->62267391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.452236:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760773.452240:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c49dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62263296->62267391] (req 62263296->62267391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.452249:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c49dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62263296->62267391] (req 62263296->62267391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.452270:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c49dd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62263296->62267391] (req 62263296->62267391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.452278:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c49dd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62263296->62267391] (req 62263296->62267391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.452429:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554711168:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12087us (12344us total) trans 4294982510 rc 0/0 00000100:00100000:2.0:1789760773.452443:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15212 00000100:00100000:2.0:1789760773.452453:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716674175588 time 716685888915 state 0x2 ref 0 00000100:00100000:2.0:1789760773.456769:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760773.457511:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554711296 00000100:00100000:1.0:1789760773.457530:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554711296 00000100:00100000:1.0:1789760773.457616:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716674175588 time 716685888915 state 0x0 ref 1 00000100:00100000:1.0:1789760773.457636:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716686149331 time 716685888915 state 0x1 ref 1 00000100:00100000:1.0:1789760773.457680:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716686149331 time 716703297862 state 0x0 ref 1 00000100:00100000:1.0:1789760773.457697:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554711296:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760773.457779:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62267392, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1db28 00010000:00010000:1.0:1789760773.457791:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c49e4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.457812:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760773.457817:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.457826:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c49e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62267392->62271487] (req 62267392->62271487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.457843:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c49e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62267392->62271487] (req 62267392->62271487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.466232:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982511 is committed 00002000:00010000:0.0:1789760773.471454:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62267392 -> 62271488 00002000:00010000:0.0:1789760773.471464:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121640 -> 121648 00010000:00010000:0.0:1789760773.471478:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c49e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62267392->62271487] (req 62267392->62271487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.471490:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c49e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62267392->62271487] (req 62267392->62271487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.471499:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c49e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62267392->62271487] (req 62267392->62271487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.471508:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760773.471514:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c49e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62267392->62271487] (req 62267392->62271487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.471522:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c49e4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62267392->62271487] (req 62267392->62271487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.471541:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c49e4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62267392->62271487] (req 62267392->62271487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.471548:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c49e4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62267392->62271487] (req 62267392->62271487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.471635:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554711296:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13937us (14870us total) trans 4294982511 rc 0/0 00000100:00100000:0.0:1789760773.471645:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15213 00000100:00100000:0.0:1789760773.471654:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716686149331 time 716703297862 state 0x2 ref 0 00000100:00100000:2.0:1789760773.476491:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760773.476609:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554711424 00000100:00100000:1.0:1789760773.476626:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554711424 00000100:00100000:1.0:1789760773.476709:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716686149331 time 716703297862 state 0x0 ref 1 00000100:00100000:1.0:1789760773.476730:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716703558278 time 716703297862 state 0x1 ref 1 00000100:00100000:1.0:1789760773.476822:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716703558278 time 716722439534 state 0x0 ref 1 00000100:00100000:1.0:1789760773.476837:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554711424:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760773.476896:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62271488, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1db30 00010000:00010000:1.0:1789760773.476906:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c49eb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.476926:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760773.476931:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.476939:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c49eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62271488->62275583] (req 62271488->62275583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.476955:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c49eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62271488->62275583] (req 62271488->62275583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.484641:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982512 is committed 00002000:00010000:1.0:1789760773.484796:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62271488 -> 62275584 00002000:00010000:1.0:1789760773.484802:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121648 -> 121656 00010000:00010000:1.0:1789760773.484819:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c49eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62271488->62275583] (req 62271488->62275583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.484842:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c49eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62271488->62275583] (req 62271488->62275583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.484852:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c49eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62271488->62275583] (req 62271488->62275583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.484859:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760773.484865:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c49eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62271488->62275583] (req 62271488->62275583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.484875:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c49eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62271488->62275583] (req 62271488->62275583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.484899:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c49eb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62271488->62275583] (req 62271488->62275583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.484907:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c49eb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62271488->62275583] (req 62271488->62275583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.485057:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554711424:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8216us (8568us total) trans 4294982512 rc 0/0 00000100:00100000:1.0:1789760773.485069:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15214 00000100:00100000:1.0:1789760773.485081:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716703558278 time 716722439534 state 0x2 ref 0 00000100:00100000:1.0:1789760773.497497:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760773.497875:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554711552 00000100:00100000:0.0:1789760773.497890:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554711552 00000100:00100000:0.0:1789760773.497967:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716703558278 time 716722439534 state 0x0 ref 1 00000100:00100000:0.0:1789760773.497986:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716722699950 time 716722439534 state 0x1 ref 1 00000100:00100000:0.0:1789760773.498036:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716722699950 time 716743653016 state 0x0 ref 1 00000100:00100000:0.0:1789760773.498055:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554711552:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760773.498121:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62275584, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1db38 00010000:00010000:0.0:1789760773.498971:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c49f2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.498993:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760773.498999:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.499006:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c49f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62275584->62279679] (req 62275584->62279679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.499024:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c49f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62275584->62279679] (req 62275584->62279679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.511740:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982513 is committed 00002000:00010000:0.0:1789760773.511990:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62275584 -> 62279680 00002000:00010000:0.0:1789760773.511997:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121656 -> 121664 00010000:00010000:0.0:1789760773.512015:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c49f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62275584->62279679] (req 62275584->62279679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.512030:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c49f2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62275584->62279679] (req 62275584->62279679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.512040:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c49f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62275584->62279679] (req 62275584->62279679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.512048:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760773.512055:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c49f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62275584->62279679] (req 62275584->62279679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.512065:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c49f2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62275584->62279679] (req 62275584->62279679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.512089:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c49f2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62275584->62279679] (req 62275584->62279679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.512098:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c49f2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62275584->62279679] (req 62275584->62279679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.512859:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554711552:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14795us (15366us total) trans 4294982513 rc 0/0 00000100:00100000:0.0:1789760773.512879:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15215 00000100:00100000:0.0:1789760773.512892:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716722699950 time 716743653016 state 0x2 ref 0 00000100:00100000:3.0:1789760773.524023:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760773.524116:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554711680 00000100:00100000:2.0:1789760773.524140:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554711680 00000100:00100000:2.0:1789760773.524232:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716722699950 time 716743653016 state 0x0 ref 1 00000100:00100000:2.0:1789760773.524255:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716743913432 time 716743653016 state 0x1 ref 1 00000100:00100000:2.0:1789760773.524291:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716743913432 time 716769909469 state 0x0 ref 1 00000100:00100000:2.0:1789760773.524306:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554711680:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760773.524372:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62279680, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1db40 00010000:00010000:2.0:1789760773.524405:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c49f9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.524426:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760773.524432:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.524440:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c49f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62279680->62283775] (req 62279680->62283775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.524456:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c49f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62279680->62283775] (req 62279680->62283775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.533535:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982514 is committed 00002000:00010000:2.0:1789760773.533919:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62279680 -> 62283776 00002000:00010000:2.0:1789760773.533926:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121664 -> 121672 00010000:00010000:2.0:1789760773.533944:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c49f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62279680->62283775] (req 62279680->62283775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.533959:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c49f9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62279680->62283775] (req 62279680->62283775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.533969:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c49f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62279680->62283775] (req 62279680->62283775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.533978:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760773.533984:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c49f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62279680->62283775] (req 62279680->62283775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.533995:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c49f9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62279680->62283775] (req 62279680->62283775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.534020:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c49f9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62279680->62283775] (req 62279680->62283775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.534029:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c49f9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62279680->62283775] (req 62279680->62283775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.534957:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554711680:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10640us (10939us total) trans 4294982514 rc 0/0 00000100:00100000:2.0:1789760773.534975:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15216 00000100:00100000:2.0:1789760773.534988:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716743913432 time 716769909469 state 0x2 ref 0 00000100:00100000:1.0:1789760773.540338:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760773.540653:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554711808 00000100:00100000:2.0:1789760773.540729:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554711808 00000100:00100000:3.0:1789760773.540993:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716743913432 time 716769909469 state 0x0 ref 1 00000100:00100000:3.0:1789760773.541012:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716770169885 time 716769909469 state 0x1 ref 1 00000100:00100000:3.0:1789760773.541103:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716770169885 time 716786719496 state 0x0 ref 1 00000100:00100000:3.0:1789760773.541120:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554711808:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760773.541203:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62283776, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1db48 00010000:00010000:3.0:1789760773.541215:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4a00 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.541242:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760773.541247:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.541256:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4a00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62283776->62287871] (req 62283776->62287871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.541272:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4a00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62283776->62287871] (req 62283776->62287871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.550236:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982515 is committed 00002000:00010000:3.0:1789760773.550499:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62283776 -> 62287872 00002000:00010000:3.0:1789760773.550506:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121672 -> 121680 00010000:00010000:3.0:1789760773.550525:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4a00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62283776->62287871] (req 62283776->62287871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.550541:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4a00 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62283776->62287871] (req 62283776->62287871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.550551:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4a00 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62283776->62287871] (req 62283776->62287871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.550560:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760773.550566:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4a00 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62283776->62287871] (req 62283776->62287871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.550577:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4a00 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62283776->62287871] (req 62283776->62287871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.550740:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4a00 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62283776->62287871] (req 62283776->62287871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.550922:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4a00 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62283776->62287871] (req 62283776->62287871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.551055:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554711808:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9931us (10725us total) trans 4294982515 rc 0/0 00000100:00100000:3.0:1789760773.551069:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15217 00000100:00100000:3.0:1789760773.554481:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716770169885 time 716786719496 state 0x2 ref 0 00000100:00100000:0.0:1789760773.555725:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760773.558269:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554711936 00000100:00100000:1.0:1789760773.558291:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554711936 00000100:00100000:1.0:1789760773.562291:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716770169885 time 716786719496 state 0x0 ref 1 00000100:00100000:1.0:1789760773.562318:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716786979912 time 716786719496 state 0x1 ref 1 00000100:00100000:1.0:1789760773.562545:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716786979912 time 716808160618 state 0x0 ref 1 00000100:00100000:1.0:1789760773.562577:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554711936:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760773.562662:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62287872, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1db50 00010000:00010000:1.0:1789760773.562674:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a07 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.562696:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760773.562702:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.562709:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62287872->62291967] (req 62287872->62291967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.562726:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62287872->62291967] (req 62287872->62291967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.572406:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982516 is committed 00002000:00010000:1.0:1789760773.572743:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62287872 -> 62291968 00002000:00010000:1.0:1789760773.572760:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121680 -> 121688 00010000:00010000:1.0:1789760773.572778:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62287872->62291967] (req 62287872->62291967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.572801:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a07 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62287872->62291967] (req 62287872->62291967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.572812:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a07 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62287872->62291967] (req 62287872->62291967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.572820:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760773.572826:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a07 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62287872->62291967] (req 62287872->62291967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.572836:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a07 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62287872->62291967] (req 62287872->62291967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.572979:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a07 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62287872->62291967] (req 62287872->62291967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.572989:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a07 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62287872->62291967] (req 62287872->62291967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.573131:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554711936:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10557us (17398us total) trans 4294982516 rc 0/0 00000100:00100000:1.0:1789760773.573146:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15218 00000100:00100000:1.0:1789760773.573158:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716786979912 time 716808160618 state 0x2 ref 0 00000100:00100000:0.0:1789760773.577773:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760773.577952:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554712064 00000100:00100000:2.0:1789760773.577967:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554712064 00000100:00100000:2.0:1789760773.578046:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716786979912 time 716808160618 state 0x0 ref 1 00000100:00100000:2.0:1789760773.578065:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716808421034 time 716808160618 state 0x1 ref 1 00000100:00100000:2.0:1789760773.578153:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716808421034 time 716823770800 state 0x0 ref 1 00000100:00100000:2.0:1789760773.578169:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554712064:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760773.578237:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62291968, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1db58 00010000:00010000:2.0:1789760773.578248:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4a0e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.578270:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760773.578275:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.578283:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4a0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62291968->62296063] (req 62291968->62296063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.578302:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4a0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62291968->62296063] (req 62291968->62296063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.586074:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982517 is committed 00002000:00010000:2.0:1789760773.586226:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62291968 -> 62296064 00002000:00010000:2.0:1789760773.586232:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121688 -> 121696 00010000:00010000:2.0:1789760773.586251:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4a0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62291968->62296063] (req 62291968->62296063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.586277:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4a0e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62291968->62296063] (req 62291968->62296063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.586287:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4a0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62291968->62296063] (req 62291968->62296063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.586301:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760773.586308:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4a0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62291968->62296063] (req 62291968->62296063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.586318:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4a0e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62291968->62296063] (req 62291968->62296063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.586357:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4a0e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62291968->62296063] (req 62291968->62296063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.586367:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4a0e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62291968->62296063] (req 62291968->62296063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.586556:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554712064:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8383us (8801us total) trans 4294982517 rc 0/0 00000100:00100000:2.0:1789760773.586569:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15219 00000100:00100000:2.0:1789760773.586581:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716808421034 time 716823770800 state 0x2 ref 0 00000100:00100000:2.0:1789760773.593639:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760773.593776:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554712192 00000100:00100000:1.0:1789760773.593791:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554712192 00000100:00100000:1.0:1789760773.593864:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716808421034 time 716823770800 state 0x0 ref 1 00000100:00100000:1.0:1789760773.593884:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716824031216 time 716823770800 state 0x1 ref 1 00000100:00100000:1.0:1789760773.593925:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716824031216 time 716839542936 state 0x0 ref 1 00000100:00100000:1.0:1789760773.593952:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554712192:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760773.594023:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62296064, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1db60 00010000:00010000:1.0:1789760773.594035:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a15 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.594058:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760773.594063:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.594072:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62296064->62300159] (req 62296064->62300159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.594090:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62296064->62300159] (req 62296064->62300159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.606122:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982518 is committed 00002000:00010000:1.0:1789760773.606305:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62296064 -> 62300160 00002000:00010000:1.0:1789760773.606311:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121696 -> 121704 00010000:00010000:1.0:1789760773.606327:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62296064->62300159] (req 62296064->62300159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.606338:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a15 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62296064->62300159] (req 62296064->62300159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.606353:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62296064->62300159] (req 62296064->62300159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.606361:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760773.606366:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62296064->62300159] (req 62296064->62300159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.606410:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a15 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62296064->62300159] (req 62296064->62300159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.606437:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a15 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62296064->62300159] (req 62296064->62300159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.606445:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a15 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62296064->62300159] (req 62296064->62300159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.606561:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554712192:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12608us (12933us total) trans 4294982518 rc 0/0 00000100:00100000:1.0:1789760773.606572:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15220 00000100:00100000:1.0:1789760773.606581:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716824031216 time 716839542936 state 0x2 ref 0 00000100:00100000:0.0:1789760773.612070:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760773.612234:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554712320 00000100:00100000:3.0:1789760773.612247:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554712320 00000100:00100000:3.0:1789760773.612320:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716824031216 time 716839542936 state 0x0 ref 1 00000100:00100000:3.0:1789760773.612340:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716839803352 time 716839542936 state 0x1 ref 1 00000100:00100000:3.0:1789760773.612417:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716839803352 time 716858034657 state 0x0 ref 1 00000100:00100000:3.0:1789760773.612432:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554712320:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760773.612493:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62300160, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1db68 00010000:00010000:3.0:1789760773.612504:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4a1c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.612525:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:3.0:1789760773.612532:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.612540:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4a1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62300160->62304255] (req 62300160->62304255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.612556:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4a1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62300160->62304255] (req 62300160->62304255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.619679:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982519 is committed 00002000:00010000:3.0:1789760773.620161:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62300160 -> 62304256 00002000:00010000:3.0:1789760773.620168:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121704 -> 121712 00010000:00010000:3.0:1789760773.620187:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4a1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62300160->62304255] (req 62300160->62304255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.620204:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4a1c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62300160->62304255] (req 62300160->62304255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.620214:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4a1c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62300160->62304255] (req 62300160->62304255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.620223:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760773.620228:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4a1c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62300160->62304255] (req 62300160->62304255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.620299:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4a1c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62300160->62304255] (req 62300160->62304255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.620325:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4a1c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62300160->62304255] (req 62300160->62304255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.620340:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4a1c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62300160->62304255] (req 62300160->62304255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.620780:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554712320:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8333us (8709us total) trans 4294982519 rc 0/0 00000100:00100000:3.0:1789760773.620795:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15221 00000100:00100000:3.0:1789760773.620807:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716839803352 time 716858034657 state 0x2 ref 0 00000100:00100000:0.0:1789760773.624338:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760773.624484:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554712448 00000100:00100000:1.0:1789760773.624495:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554712448 00000100:00100000:1.0:1789760773.624569:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716839803352 time 716858034657 state 0x0 ref 1 00000100:00100000:1.0:1789760773.624589:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716858295073 time 716858034657 state 0x1 ref 1 00000100:00100000:1.0:1789760773.624623:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716858295073 time 716870241711 state 0x0 ref 1 00000100:00100000:1.0:1789760773.624635:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554712448:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760773.624696:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62304256, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1db70 00010000:00010000:1.0:1789760773.624707:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a23 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.624726:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760773.624731:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.624738:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62304256->62308351] (req 62304256->62308351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.624765:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62304256->62308351] (req 62304256->62308351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.631576:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982520 is committed 00002000:00010000:1.0:1789760773.631726:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62304256 -> 62308352 00002000:00010000:1.0:1789760773.631732:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121712 -> 121720 00010000:00010000:1.0:1789760773.631762:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62304256->62308351] (req 62304256->62308351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.631779:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a23 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62304256->62308351] (req 62304256->62308351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.631790:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a23 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62304256->62308351] (req 62304256->62308351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.631805:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760773.631811:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a23 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62304256->62308351] (req 62304256->62308351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.631821:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a23 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62304256->62308351] (req 62304256->62308351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.631847:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a23 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62304256->62308351] (req 62304256->62308351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.631855:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4a23 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62304256->62308351] (req 62304256->62308351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.631993:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554712448:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7351us (7659us total) trans 4294982520 rc 0/0 00000100:00100000:1.0:1789760773.632006:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15222 00000100:00100000:1.0:1789760773.632019:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716858295073 time 716870241711 state 0x2 ref 0 00000100:00100000:2.0:1789760773.637338:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760773.637647:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554712576 00000100:00100000:0.0:1789760773.637660:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554712576 00000100:00100000:0.0:1789760773.637732:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716858295073 time 716870241711 state 0x0 ref 1 00000100:00100000:0.0:1789760773.637765:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716870502127 time 716870241711 state 0x1 ref 1 00000100:00100000:0.0:1789760773.637801:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716870502127 time 716883419695 state 0x0 ref 1 00000100:00100000:0.0:1789760773.637816:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554712576:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760773.637878:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62308352, mtime: 0x6aad9504, atime: 0x0, ctime: 0x6aad9504, blocks: 0x1db78 00010000:00010000:0.0:1789760773.637890:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a2a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.637910:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760773.637915:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.637922:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62308352->62312447] (req 62308352->62312447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.637937:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62308352->62312447] (req 62308352->62312447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.647242:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982521 is committed 00002000:00010000:0.0:1789760773.647883:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62308352 -> 62312448 00002000:00010000:0.0:1789760773.647889:0:8441:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760772 -> 1789760773 00002000:00010000:0.0:1789760773.647891:0:8441:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760772 -> 1789760773 00002000:00010000:0.0:1789760773.647894:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121720 -> 121728 00010000:00010000:0.0:1789760773.647913:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62308352->62312447] (req 62308352->62312447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.647929:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a2a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62308352->62312447] (req 62308352->62312447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.647940:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a2a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62308352->62312447] (req 62308352->62312447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.647948:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760773.647954:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a2a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62308352->62312447] (req 62308352->62312447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.647964:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a2a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62308352->62312447] (req 62308352->62312447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.647990:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a2a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62308352->62312447] (req 62308352->62312447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.647998:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a2a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62308352->62312447] (req 62308352->62312447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.648757:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554712576:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10520us (11013us total) trans 4294982521 rc 0/0 00000100:00100000:0.0:1789760773.648773:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15223 00000100:00100000:0.0:1789760773.648786:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716870502127 time 716883419695 state 0x2 ref 0 00000100:00100000:0.0:1789760773.652705:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760773.652818:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554712704 00000100:00100000:2.0:1789760773.652830:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554712704 00000100:00100000:2.0:1789760773.652905:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716870502127 time 716883419695 state 0x0 ref 1 00000100:00100000:2.0:1789760773.652925:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716883680111 time 716883419695 state 0x1 ref 1 00000100:00100000:2.0:1789760773.652966:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716883680111 time 716898584588 state 0x0 ref 1 00000100:00100000:2.0:1789760773.652980:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554712704:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760773.653042:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62312448, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1db80 00010000:00010000:2.0:1789760773.653053:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a31 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.653074:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760773.653079:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.653087:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62312448->62316543] (req 62312448->62316543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.653105:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62312448->62316543] (req 62312448->62316543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.661401:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982522 is committed 00002000:00010000:2.0:1789760773.661826:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62312448 -> 62316544 00002000:00010000:2.0:1789760773.661834:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121728 -> 121736 00010000:00010000:2.0:1789760773.661851:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62312448->62316543] (req 62312448->62316543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.661867:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a31 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62312448->62316543] (req 62312448->62316543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.661877:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a31 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62312448->62316543] (req 62312448->62316543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.661887:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760773.661893:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a31 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62312448->62316543] (req 62312448->62316543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.661902:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a31 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62312448->62316543] (req 62312448->62316543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.661927:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a31 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62312448->62316543] (req 62312448->62316543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.661934:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a31 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62312448->62316543] (req 62312448->62316543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.662071:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554712704:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9086us (9373us total) trans 4294982522 rc 0/0 00000100:00100000:2.0:1789760773.662083:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15224 00000100:00100000:2.0:1789760773.662094:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716883680111 time 716898584588 state 0x2 ref 0 00000100:00100000:3.0:1789760773.666918:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760773.668163:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554712832 00000100:00100000:0.0:1789760773.668180:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554712832 00000100:00100000:0.0:1789760773.668268:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716883680111 time 716898584588 state 0x0 ref 1 00000100:00100000:0.0:1789760773.668292:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716898845004 time 716898584588 state 0x1 ref 1 00000100:00100000:0.0:1789760773.668336:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716898845004 time 716913954257 state 0x0 ref 1 00000100:00100000:0.0:1789760773.668352:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554712832:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760773.668452:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62316544, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1db88 00010000:00010000:0.0:1789760773.668465:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a38 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.668486:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760773.668491:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.668499:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62316544->62320639] (req 62316544->62320639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.668518:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62316544->62320639] (req 62316544->62320639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.675741:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982523 is committed 00002000:00010000:0.0:1789760773.675977:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62316544 -> 62320640 00002000:00010000:0.0:1789760773.675984:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121736 -> 121744 00010000:00010000:0.0:1789760773.676003:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62316544->62320639] (req 62316544->62320639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.676018:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a38 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62316544->62320639] (req 62316544->62320639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.676029:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62316544->62320639] (req 62316544->62320639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.676041:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760773.676047:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62316544->62320639] (req 62316544->62320639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.676057:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a38 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62316544->62320639] (req 62316544->62320639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.676084:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a38 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62316544->62320639] (req 62316544->62320639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.676101:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a38 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62316544->62320639] (req 62316544->62320639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.676246:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554712832:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7890us (9335us total) trans 4294982523 rc 0/0 00000100:00100000:0.0:1789760773.676259:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15225 00000100:00100000:0.0:1789760773.676271:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716898845004 time 716913954257 state 0x2 ref 0 00000100:00100000:1.0:1789760773.680880:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760773.680971:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554712960 00000100:00100000:0.0:1789760773.680985:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554712960 00000100:00100000:0.0:1789760773.681059:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716898845004 time 716913954257 state 0x0 ref 1 00000100:00100000:0.0:1789760773.681078:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716914214673 time 716913954257 state 0x1 ref 1 00000100:00100000:0.0:1789760773.681119:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716914214673 time 716926737538 state 0x0 ref 1 00000100:00100000:0.0:1789760773.681138:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554712960:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760773.681206:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62320640, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1db90 00010000:00010000:0.0:1789760773.681217:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4a3f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.681237:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760773.681243:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.681251:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4a3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62320640->62324735] (req 62320640->62324735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.681266:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4a3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62320640->62324735] (req 62320640->62324735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760773.687028:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982524 is committed 00002000:00010000:0.0:1789760773.695473:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62320640 -> 62324736 00002000:00010000:0.0:1789760773.695489:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121744 -> 121752 00010000:00010000:0.0:1789760773.695513:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4a3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62320640->62324735] (req 62320640->62324735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.695529:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4a3f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62320640->62324735] (req 62320640->62324735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.695539:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4a3f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62320640->62324735] (req 62320640->62324735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.695547:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760773.695554:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4a3f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62320640->62324735] (req 62320640->62324735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.695564:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4a3f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62320640->62324735] (req 62320640->62324735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.695606:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4a3f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62320640->62324735] (req 62320640->62324735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.695615:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4a3f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62320640->62324735] (req 62320640->62324735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.696131:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554712960:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14986us (15257us total) trans 4294982524 rc 0/0 00000100:00100000:0.0:1789760773.696146:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15226 00000100:00100000:0.0:1789760773.696160:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716914214673 time 716926737538 state 0x2 ref 0 00000100:00100000:1.0:1789760773.700289:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760773.702297:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554713088 00000100:00100000:2.0:1789760773.702320:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554713088 00000100:00100000:2.0:1789760773.702420:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716914214673 time 716926737538 state 0x0 ref 1 00000100:00100000:2.0:1789760773.702440:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716926997954 time 716926737538 state 0x1 ref 1 00000100:00100000:2.0:1789760773.702477:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716926997954 time 716948094916 state 0x0 ref 1 00000100:00100000:2.0:1789760773.702498:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554713088:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760773.702572:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62324736, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1db98 00010000:00010000:2.0:1789760773.702584:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a46 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.702607:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760773.702613:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.702621:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62324736->62328831] (req 62324736->62328831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.702640:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62324736->62328831] (req 62324736->62328831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.713655:0:6508:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760773.714089:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982525 is committed 00002000:00010000:2.0:1789760773.714237:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62324736 -> 62328832 00002000:00010000:2.0:1789760773.714243:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121752 -> 121760 00010000:00010000:2.0:1789760773.714260:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62324736->62328831] (req 62324736->62328831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.714277:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a46 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62324736->62328831] (req 62324736->62328831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.714286:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a46 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62324736->62328831] (req 62324736->62328831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.714293:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760773.714300:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a46 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62324736->62328831] (req 62324736->62328831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.714311:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a46 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62324736->62328831] (req 62324736->62328831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.714337:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a46 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62324736->62328831] (req 62324736->62328831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.714345:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a46 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62324736->62328831] (req 62324736->62328831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.714533:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554713088:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12029us (14251us total) trans 4294982525 rc 0/0 00000100:00100000:2.0:1789760773.714547:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15227 00000100:00100000:2.0:1789760773.714560:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716926997954 time 716948094916 state 0x2 ref 0 00000100:00100000:3.0:1789760773.720160:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760773.720273:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554713216 00000100:00100000:2.0:1789760773.720285:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554713216 00000100:00100000:2.0:1789760773.720367:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716926997954 time 716948094916 state 0x0 ref 1 00000100:00100000:2.0:1789760773.720405:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716948355332 time 716948094916 state 0x1 ref 1 00000100:00100000:2.0:1789760773.720430:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716948355332 time 716966049183 state 0x0 ref 1 00000100:00100000:2.0:1789760773.720444:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe1e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554713216:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760773.720508:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62328832, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dba0 00010000:00010000:2.0:1789760773.720519:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4a4d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.720538:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:2.0:1789760773.720542:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.720550:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4a4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62328832->62332927] (req 62328832->62332927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.720567:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4a4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62328832->62332927] (req 62328832->62332927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.731822:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982526 is committed 00002000:00010000:2.0:1789760773.732068:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62328832 -> 62332928 00002000:00010000:2.0:1789760773.732074:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121760 -> 121768 00010000:00010000:2.0:1789760773.732104:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4a4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62328832->62332927] (req 62328832->62332927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.732127:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4a4d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62328832->62332927] (req 62328832->62332927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.732138:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4a4d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62328832->62332927] (req 62328832->62332927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.732147:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760773.732153:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4a4d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62328832->62332927] (req 62328832->62332927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.732164:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4a4d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62328832->62332927] (req 62328832->62332927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.732190:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4a4d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62328832->62332927] (req 62328832->62332927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.732199:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4a4d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62328832->62332927] (req 62328832->62332927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.732605:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe1e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554713216:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12155us (12450us total) trans 4294982526 rc 0/0 00000100:00100000:2.0:1789760773.732619:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15228 00000100:00100000:2.0:1789760773.732632:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716948355332 time 716966049183 state 0x2 ref 0 00000100:00100000:1.0:1789760773.739472:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760773.739679:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554713344 00000100:00100000:0.0:1789760773.739692:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554713344 00000100:00100000:0.0:1789760773.739776:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716948355332 time 716966049183 state 0x0 ref 1 00000100:00100000:0.0:1789760773.739795:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716966309599 time 716966049183 state 0x1 ref 1 00000100:00100000:0.0:1789760773.739881:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716966309599 time 716985498801 state 0x0 ref 1 00000100:00100000:0.0:1789760773.739896:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554713344:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760773.739961:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62332928, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dba8 00010000:00010000:0.0:1789760773.739972:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a54 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.739991:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:0.0:1789760773.739996:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.740004:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62332928->62337023] (req 62332928->62337023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.740020:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62332928->62337023] (req 62332928->62337023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.747440:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982527 is committed 00002000:00010000:0.0:1789760773.748313:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62332928 -> 62337024 00002000:00010000:0.0:1789760773.748320:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121768 -> 121776 00010000:00010000:0.0:1789760773.748338:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62332928->62337023] (req 62332928->62337023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.748354:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a54 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62332928->62337023] (req 62332928->62337023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.748364:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a54 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62332928->62337023] (req 62332928->62337023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.748373:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760773.748409:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a54 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62332928->62337023] (req 62332928->62337023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.748428:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a54 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62332928->62337023] (req 62332928->62337023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.748453:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a54 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62332928->62337023] (req 62332928->62337023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.748461:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a54 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62332928->62337023] (req 62332928->62337023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.748590:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554713344:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8689us (9126us total) trans 4294982527 rc 0/0 00000100:00100000:0.0:1789760773.748602:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15229 00000100:00100000:0.0:1789760773.748614:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716966309599 time 716985498801 state 0x2 ref 0 00000100:00100000:0.0:1789760773.752916:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760773.753090:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554713472 00000100:00100000:1.0:1789760773.753104:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554713472 00000100:00100000:1.0:1789760773.753180:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716966309599 time 716985498801 state 0x0 ref 1 00000100:00100000:1.0:1789760773.753201:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716985759217 time 716985498801 state 0x1 ref 1 00000100:00100000:1.0:1789760773.753312:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716985759217 time 716998921252 state 0x0 ref 1 00000100:00100000:1.0:1789760773.753327:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554713472:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760773.754068:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62337024, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dbb0 00010000:00010000:1.0:1789760773.754084:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4a5b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.754106:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=716s v=(0 0) 00010000:00010000:1.0:1789760773.754111:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.754119:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4a5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62337024->62341119] (req 62337024->62341119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.754136:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4a5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62337024->62341119] (req 62337024->62341119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.762593:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982528 is committed 00002000:00010000:1.0:1789760773.762787:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62337024 -> 62341120 00002000:00010000:1.0:1789760773.762794:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121776 -> 121784 00010000:00010000:1.0:1789760773.762812:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4a5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62337024->62341119] (req 62337024->62341119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.762826:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4a5b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62337024->62341119] (req 62337024->62341119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.762836:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4a5b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62337024->62341119] (req 62337024->62341119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.762849:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760773.762855:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4a5b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62337024->62341119] (req 62337024->62341119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.762865:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4a5b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62337024->62341119] (req 62337024->62341119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.763073:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4a5b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62337024->62341119] (req 62337024->62341119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.763083:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4a5b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62337024->62341119] (req 62337024->62341119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.763212:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554713472:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9880us (10304us total) trans 4294982528 rc 0/0 00000100:00100000:1.0:1789760773.763225:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15230 00000100:00100000:1.0:1789760773.763239:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716985759217 time 716998921252 state 0x2 ref 0 00000100:00100000:3.0:1789760773.765945:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760773.766143:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554713600 00000100:00100000:2.0:1789760773.766157:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554713600 00000100:00100000:2.0:1789760773.766238:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716985759217 time 716998921252 state 0x0 ref 1 00000100:00100000:2.0:1789760773.766260:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716999181668 time 716998921252 state 0x1 ref 1 00000100:00100000:2.0:1789760773.766555:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716999181668 time 717011919110 state 0x0 ref 1 00000100:00100000:2.0:1789760773.766578:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554713600:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760773.766650:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62341120, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dbb8 00010000:00010000:2.0:1789760773.766662:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a62 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.766687:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760773.766693:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.766700:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62341120->62345215] (req 62341120->62345215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.766718:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62341120->62345215] (req 62341120->62345215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.774478:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982529 is committed 00002000:00010000:2.0:1789760773.776573:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62341120 -> 62345216 00002000:00010000:2.0:1789760773.776587:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121784 -> 121792 00010000:00010000:2.0:1789760773.776605:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62341120->62345215] (req 62341120->62345215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.776620:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a62 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62341120->62345215] (req 62341120->62345215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.776630:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62341120->62345215] (req 62341120->62345215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.776638:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760773.776644:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62341120->62345215] (req 62341120->62345215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.776655:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a62 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62341120->62345215] (req 62341120->62345215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.776685:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a62 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62341120->62345215] (req 62341120->62345215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.776694:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4a62 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62341120->62345215] (req 62341120->62345215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.777254:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554713600:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10672us (11312us total) trans 4294982529 rc 0/0 00000100:00100000:2.0:1789760773.777269:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15231 00000100:00100000:2.0:1789760773.777281:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716999181668 time 717011919110 state 0x2 ref 0 00000100:00100000:2.0:1789760773.788936:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760773.789186:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554713728 00000100:00100000:1.0:1789760773.789200:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554713728 00000100:00100000:1.0:1789760773.789277:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 716999181668 time 717011919110 state 0x0 ref 1 00000100:00100000:1.0:1789760773.789297:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717012179526 time 717011919110 state 0x1 ref 1 00000100:00100000:1.0:1789760773.789513:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717012179526 time 717035129982 state 0x0 ref 1 00000100:00100000:1.0:1789760773.789532:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554713728:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760773.789598:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62345216, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dbc0 00010000:00010000:1.0:1789760773.789609:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4a69 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.789628:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760773.789633:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.789640:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4a69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62345216->62349311] (req 62345216->62349311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.789657:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4a69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62345216->62349311] (req 62345216->62349311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.800109:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982530 is committed 00002000:00010000:1.0:1789760773.800240:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62345216 -> 62349312 00002000:00010000:1.0:1789760773.800247:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121792 -> 121800 00010000:00010000:1.0:1789760773.800588:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4a69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62345216->62349311] (req 62345216->62349311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.800606:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4a69 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62345216->62349311] (req 62345216->62349311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.800615:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4a69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62345216->62349311] (req 62345216->62349311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.800623:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760773.800630:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4a69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62345216->62349311] (req 62345216->62349311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.800739:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4a69 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62345216->62349311] (req 62345216->62349311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.800841:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4a69 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62345216->62349311] (req 62345216->62349311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.800851:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4a69 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62345216->62349311] (req 62345216->62349311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.801101:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554713728:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11554us (12788us total) trans 4294982530 rc 0/0 00000100:00100000:1.0:1789760773.801116:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15232 00000100:00100000:1.0:1789760773.801129:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717012179526 time 717035129982 state 0x2 ref 0 00000100:00100000:1.0:1789760773.806450:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760773.806571:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554713856 00000100:00100000:0.0:1789760773.806585:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554713856 00000100:00100000:0.0:1789760773.806721:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717012179526 time 717035129982 state 0x0 ref 1 00000100:00100000:0.0:1789760773.806741:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717035390398 time 717035129982 state 0x1 ref 1 00000100:00100000:0.0:1789760773.806888:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717035390398 time 717052504875 state 0x0 ref 1 00000100:00100000:0.0:1789760773.806905:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554713856:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760773.806979:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62349312, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dbc8 00010000:00010000:0.0:1789760773.806991:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a70 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.807009:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760773.807013:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.807019:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62349312->62353407] (req 62349312->62353407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.807032:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62349312->62353407] (req 62349312->62353407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.815136:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982531 is committed 00002000:00010000:3.0:1789760773.815356:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62349312 -> 62353408 00002000:00010000:3.0:1789760773.815363:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121800 -> 121808 00010000:00010000:3.0:1789760773.815420:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62349312->62353407] (req 62349312->62353407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.815437:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a70 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62349312->62353407] (req 62349312->62353407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.815447:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a70 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62349312->62353407] (req 62349312->62353407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.815461:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760773.815467:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a70 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62349312->62353407] (req 62349312->62353407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.815477:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a70 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62349312->62353407] (req 62349312->62353407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.815502:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a70 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62349312->62353407] (req 62349312->62353407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.815510:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4a70 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62349312->62353407] (req 62349312->62353407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.815640:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554713856:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8731us (9199us total) trans 4294982531 rc 0/0 00000100:00100000:3.0:1789760773.815653:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15233 00000100:00100000:3.0:1789760773.815665:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717035390398 time 717052504875 state 0x2 ref 0 00000100:00100000:3.0:1789760773.819885:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760773.820099:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554713984 00000100:00100000:0.0:1789760773.820113:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554713984 00000100:00100000:0.0:1789760773.820205:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717035390398 time 717052504875 state 0x0 ref 1 00000100:00100000:0.0:1789760773.820225:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717052765291 time 717052504875 state 0x1 ref 1 00000100:00100000:0.0:1789760773.820264:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717052765291 time 717065882383 state 0x0 ref 1 00000100:00100000:0.0:1789760773.820278:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554713984:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760773.820362:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62353408, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dbd0 00010000:00010000:0.0:1789760773.820374:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4a77 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.820445:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760773.820451:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.820458:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4a77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62353408->62357503] (req 62353408->62357503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.820476:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4a77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62353408->62357503] (req 62353408->62357503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.829182:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982532 is committed 00002000:00010000:0.0:1789760773.829793:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62353408 -> 62357504 00002000:00010000:0.0:1789760773.829802:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121808 -> 121816 00010000:00010000:0.0:1789760773.829821:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4a77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62353408->62357503] (req 62353408->62357503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.829837:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4a77 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62353408->62357503] (req 62353408->62357503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.829848:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4a77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62353408->62357503] (req 62353408->62357503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.829855:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760773.829861:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4a77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62353408->62357503] (req 62353408->62357503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.829871:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4a77 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62353408->62357503] (req 62353408->62357503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.829903:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4a77 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62353408->62357503] (req 62353408->62357503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.829912:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4a77 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62353408->62357503] (req 62353408->62357503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.830054:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554713984:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9771us (10177us total) trans 4294982532 rc 0/0 00000100:00100000:0.0:1789760773.830066:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15234 00000100:00100000:0.0:1789760773.830078:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717052765291 time 717065882383 state 0x2 ref 0 00000100:00100000:2.0:1789760773.833205:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760773.834155:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554714112 00000100:00100000:0.0:1789760773.834172:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554714112 00000100:00100000:0.0:1789760773.834245:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717052765291 time 717065882383 state 0x0 ref 1 00000100:00100000:0.0:1789760773.834264:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717066142799 time 717065882383 state 0x1 ref 1 00000100:00100000:0.0:1789760773.835044:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717066142799 time 717080655141 state 0x0 ref 1 00000100:00100000:0.0:1789760773.835069:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554714112:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760773.836117:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62357504, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dbd8 00010000:00010000:0.0:1789760773.836142:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4a7e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.836165:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760773.836172:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.836180:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4a7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62357504->62361599] (req 62357504->62361599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.836198:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4a7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62357504->62361599] (req 62357504->62361599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.843843:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982533 is committed 00002000:00010000:0.0:1789760773.844075:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62357504 -> 62361600 00002000:00010000:0.0:1789760773.844082:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121816 -> 121824 00010000:00010000:0.0:1789760773.844101:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4a7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62357504->62361599] (req 62357504->62361599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.844492:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4a7e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62357504->62361599] (req 62357504->62361599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.844736:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4a7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62357504->62361599] (req 62357504->62361599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.844754:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760773.844760:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4a7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62357504->62361599] (req 62357504->62361599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.844771:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4a7e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62357504->62361599] (req 62357504->62361599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.844798:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4a7e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62357504->62361599] (req 62357504->62361599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.844814:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4a7e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62357504->62361599] (req 62357504->62361599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760773.845790:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554714112:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10717us (12593us total) trans 4294982533 rc 0/0 00000100:00100000:0.0:1789760773.845806:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15235 00000100:00100000:0.0:1789760773.845819:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717066142799 time 717080655141 state 0x2 ref 0 00000100:00100000:1.0:1789760773.850164:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760773.850830:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554714240 00000100:00100000:2.0:1789760773.850845:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554714240 00000100:00100000:2.0:1789760773.850924:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717066142799 time 717080655141 state 0x0 ref 1 00000100:00100000:2.0:1789760773.850945:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717080915557 time 717080655141 state 0x1 ref 1 00000100:00100000:2.0:1789760773.851622:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717080915557 time 717097234520 state 0x0 ref 1 00000100:00100000:2.0:1789760773.851643:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554714240:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760773.851728:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62361600, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dbe0 00010000:00010000:2.0:1789760773.851741:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4a85 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.851773:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760773.851778:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.851786:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4a85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62361600->62365695] (req 62361600->62365695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.851804:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4a85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62361600->62365695] (req 62361600->62365695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.859622:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982534 is committed 00002000:00010000:2.0:1789760773.859856:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62361600 -> 62365696 00002000:00010000:2.0:1789760773.859864:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121824 -> 121832 00010000:00010000:2.0:1789760773.859883:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4a85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62361600->62365695] (req 62361600->62365695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.859898:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4a85 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62361600->62365695] (req 62361600->62365695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.859908:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4a85 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62361600->62365695] (req 62361600->62365695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.859917:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760773.859924:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4a85 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62361600->62365695] (req 62361600->62365695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.859941:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4a85 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62361600->62365695] (req 62361600->62365695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.859966:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4a85 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62361600->62365695] (req 62361600->62365695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.859974:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4a85 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62361600->62365695] (req 62361600->62365695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.860250:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554714240:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8606us (10096us total) trans 4294982534 rc 0/0 00000100:00100000:2.0:1789760773.860264:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15236 00000100:00100000:2.0:1789760773.860278:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717080915557 time 717097234520 state 0x2 ref 0 00000100:00100000:3.0:1789760773.868069:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760773.868418:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554714368 00000100:00100000:0.0:1789760773.868433:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554714368 00000100:00100000:0.0:1789760773.868513:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717080915557 time 717097234520 state 0x0 ref 1 00000100:00100000:0.0:1789760773.868534:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717097494936 time 717097234520 state 0x1 ref 1 00000100:00100000:0.0:1789760773.868572:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717097494936 time 717114190440 state 0x0 ref 1 00000100:00100000:0.0:1789760773.868587:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554714368:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760773.868658:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62365696, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dbe8 00010000:00010000:0.0:1789760773.868670:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4a8c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.868691:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760773.868697:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.868705:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4a8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62365696->62369791] (req 62365696->62369791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.868724:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4a8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62365696->62369791] (req 62365696->62369791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.875544:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982535 is committed 00002000:00010000:1.0:1789760773.888322:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62365696 -> 62369792 00002000:00010000:1.0:1789760773.888336:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121832 -> 121840 00010000:00010000:1.0:1789760773.888355:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4a8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62365696->62369791] (req 62365696->62369791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.888370:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4a8c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62365696->62369791] (req 62365696->62369791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.888424:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4a8c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62365696->62369791] (req 62365696->62369791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.888432:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760773.888438:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4a8c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62365696->62369791] (req 62365696->62369791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.888449:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4a8c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62365696->62369791] (req 62365696->62369791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.888476:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4a8c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62365696->62369791] (req 62365696->62369791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.888484:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c4a8c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62365696->62369791] (req 62365696->62369791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.888619:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554714368:12345-192.168.203.22@tcp:4:dd.0 Request processed in 20025us (20558us total) trans 4294982535 rc 0/0 00000100:00100000:1.0:1789760773.888633:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15237 00000100:00100000:1.0:1789760773.888645:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717097494936 time 717114190440 state 0x2 ref 0 00000100:00100000:1.0:1789760773.897497:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760773.897699:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554714496 00000100:00100000:0.0:1789760773.897712:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554714496 00000100:00100000:0.0:1789760773.897798:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717097494936 time 717114190440 state 0x0 ref 1 00000100:00100000:0.0:1789760773.897818:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717114450856 time 717114190440 state 0x1 ref 1 00000100:00100000:0.0:1789760773.897854:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717114450856 time 717143473116 state 0x0 ref 1 00000100:00100000:0.0:1789760773.897867:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554714496:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760773.897931:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62369792, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dbf0 00010000:00010000:0.0:1789760773.897943:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4a93 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.897963:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760773.897968:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760773.897976:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4a93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62369792->62373887] (req 62369792->62373887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760773.897993:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4a93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62369792->62373887] (req 62369792->62373887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.908902:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982536 is committed 00002000:00010000:1.0:1789760773.909051:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62369792 -> 62373888 00002000:00010000:1.0:1789760773.909058:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121840 -> 121848 00010000:00010000:1.0:1789760773.909077:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4a93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62369792->62373887] (req 62369792->62373887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.909093:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4a93 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62369792->62373887] (req 62369792->62373887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.909104:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4a93 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62369792->62373887] (req 62369792->62373887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.909112:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760773.909127:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4a93 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62369792->62373887] (req 62369792->62373887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.909139:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4a93 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62369792->62373887] (req 62369792->62373887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.909165:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4a93 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62369792->62373887] (req 62369792->62373887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.909174:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4a93 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62369792->62373887] (req 62369792->62373887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.909302:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554714496:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11430us (11815us total) trans 4294982536 rc 0/0 00000100:00100000:1.0:1789760773.909315:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15238 00000100:00100000:1.0:1789760773.909326:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717114450856 time 717143473116 state 0x2 ref 0 00000100:00100000:2.0:1789760773.914091:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760773.914478:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554714624 00000100:00100000:3.0:1789760773.914495:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554714624 00000100:00100000:3.0:1789760773.914675:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717114450856 time 717143473116 state 0x0 ref 1 00000100:00100000:3.0:1789760773.914696:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717143733532 time 717143473116 state 0x1 ref 1 00000100:00100000:3.0:1789760773.914737:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717143733532 time 717160354487 state 0x0 ref 1 00000100:00100000:3.0:1789760773.914763:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554714624:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760773.914831:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62373888, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dbf8 00010000:00010000:3.0:1789760773.914842:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4a9a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.914864:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760773.914870:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.914879:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4a9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62373888->62377983] (req 62373888->62377983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.914896:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4a9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62373888->62377983] (req 62373888->62377983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.921035:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982537 is committed 00002000:00010000:3.0:1789760773.921498:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62373888 -> 62377984 00002000:00010000:3.0:1789760773.921505:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121848 -> 121856 00010000:00010000:3.0:1789760773.921523:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4a9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62373888->62377983] (req 62373888->62377983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.921538:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4a9a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62373888->62377983] (req 62373888->62377983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.921548:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4a9a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62373888->62377983] (req 62373888->62377983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.921564:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760773.921570:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4a9a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62373888->62377983] (req 62373888->62377983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.921581:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4a9a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62373888->62377983] (req 62373888->62377983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.921607:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4a9a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62373888->62377983] (req 62373888->62377983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.921616:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4a9a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62373888->62377983] (req 62373888->62377983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.921762:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554714624:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6983us (7668us total) trans 4294982537 rc 0/0 00000100:00100000:3.0:1789760773.921774:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15239 00000100:00100000:3.0:1789760773.921787:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717143733532 time 717160354487 state 0x2 ref 0 00000100:00100000:3.0:1789760773.924920:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760773.925092:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554714752 00000100:00100000:2.0:1789760773.925104:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554714752 00000100:00100000:2.0:1789760773.925186:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717143733532 time 717160354487 state 0x0 ref 1 00000100:00100000:2.0:1789760773.925204:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717160614903 time 717160354487 state 0x1 ref 1 00000100:00100000:2.0:1789760773.925242:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717160614903 time 717170859709 state 0x0 ref 1 00000100:00100000:2.0:1789760773.925257:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554714752:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760773.925320:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62377984, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dc00 00010000:00010000:2.0:1789760773.925332:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4aa1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.925354:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760773.925359:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.925367:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4aa1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62377984->62382079] (req 62377984->62382079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.925796:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4aa1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62377984->62382079] (req 62377984->62382079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760773.931614:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982538 is committed 00002000:00010000:2.0:1789760773.931764:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62377984 -> 62382080 00002000:00010000:2.0:1789760773.931770:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121856 -> 121864 00010000:00010000:2.0:1789760773.931788:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4aa1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62377984->62382079] (req 62377984->62382079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.931802:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4aa1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62377984->62382079] (req 62377984->62382079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.931812:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4aa1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62377984->62382079] (req 62377984->62382079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.931821:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760773.931827:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4aa1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62377984->62382079] (req 62377984->62382079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.931844:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4aa1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62377984->62382079] (req 62377984->62382079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.931868:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4aa1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62377984->62382079] (req 62377984->62382079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.931876:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4aa1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62377984->62382079] (req 62377984->62382079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.932042:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554714752:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6779us (7127us total) trans 4294982538 rc 0/0 00000100:00100000:2.0:1789760773.932053:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15240 00000100:00100000:2.0:1789760773.932065:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717160614903 time 717170859709 state 0x2 ref 0 00000100:00100000:1.0:1789760773.935083:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760773.935249:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554714880 00000100:00100000:3.0:1789760773.935262:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554714880 00000100:00100000:3.0:1789760773.935332:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717160614903 time 717170859709 state 0x0 ref 1 00000100:00100000:3.0:1789760773.935350:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717171120125 time 717170859709 state 0x1 ref 1 00000100:00100000:3.0:1789760773.935444:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717171120125 time 717181061760 state 0x0 ref 1 00000100:00100000:3.0:1789760773.935459:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554714880:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760773.935526:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62382080, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dc08 00010000:00010000:3.0:1789760773.935536:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4aa8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.935557:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760773.935562:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.935570:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4aa8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62382080->62386175] (req 62382080->62386175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.935586:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4aa8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62382080->62386175] (req 62382080->62386175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.945260:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982539 is committed 00002000:00010000:3.0:1789760773.945570:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62382080 -> 62386176 00002000:00010000:3.0:1789760773.945578:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121864 -> 121872 00010000:00010000:3.0:1789760773.945596:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4aa8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62382080->62386175] (req 62382080->62386175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.945611:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4aa8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62382080->62386175] (req 62382080->62386175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.945620:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4aa8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62382080->62386175] (req 62382080->62386175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.945628:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760773.945634:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4aa8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62382080->62386175] (req 62382080->62386175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.945644:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4aa8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62382080->62386175] (req 62382080->62386175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.945669:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4aa8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62382080->62386175] (req 62382080->62386175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.945677:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4aa8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62382080->62386175] (req 62382080->62386175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760773.945820:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554714880:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10355us (10741us total) trans 4294982539 rc 0/0 00000100:00100000:3.0:1789760773.945834:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15241 00000100:00100000:3.0:1789760773.945847:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717171120125 time 717181061760 state 0x2 ref 0 00000100:00100000:1.0:1789760773.950439:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760773.950795:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554715008 00000100:00100000:2.0:1789760773.950810:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554715008 00000100:00100000:2.0:1789760773.950887:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717171120125 time 717181061760 state 0x0 ref 1 00000100:00100000:2.0:1789760773.950909:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717181322176 time 717181061760 state 0x1 ref 1 00000100:00100000:2.0:1789760773.950945:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717181322176 time 717196563099 state 0x0 ref 1 00000100:00100000:2.0:1789760773.950960:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554715008:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760773.951027:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62386176, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dc10 00010000:00010000:2.0:1789760773.951038:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4aaf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.951059:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760773.951065:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.951073:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4aaf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62386176->62390271] (req 62386176->62390271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.951090:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4aaf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62386176->62390271] (req 62386176->62390271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760773.957070:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982540 is committed 00002000:00010000:1.0:1789760773.957259:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62386176 -> 62390272 00002000:00010000:1.0:1789760773.957266:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121872 -> 121880 00010000:00010000:1.0:1789760773.957284:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4aaf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62386176->62390271] (req 62386176->62390271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.957300:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4aaf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62386176->62390271] (req 62386176->62390271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.957310:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4aaf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62386176->62390271] (req 62386176->62390271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.957320:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760773.957326:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4aaf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62386176->62390271] (req 62386176->62390271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.957337:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4aaf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62386176->62390271] (req 62386176->62390271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.957361:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4aaf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62386176->62390271] (req 62386176->62390271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.957370:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4aaf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62386176->62390271] (req 62386176->62390271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.957660:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554715008:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6695us (7228us total) trans 4294982540 rc 0/0 00000100:00100000:1.0:1789760773.957675:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15242 00000100:00100000:1.0:1789760773.957687:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717181322176 time 717196563099 state 0x2 ref 0 00000100:00100000:0.0:1789760773.961153:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760773.961595:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554715136 00000100:00100000:2.0:1789760773.961608:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554715136 00000100:00100000:2.0:1789760773.961674:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717181322176 time 717196563099 state 0x0 ref 1 00000100:00100000:2.0:1789760773.961692:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717196823515 time 717196563099 state 0x1 ref 1 00000100:00100000:2.0:1789760773.961721:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717196823515 time 717207339425 state 0x0 ref 1 00000100:00100000:2.0:1789760773.961732:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554715136:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760773.961800:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62390272, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dc18 00010000:00010000:2.0:1789760773.961810:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4ab6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.961830:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760773.961835:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.961842:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4ab6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62390272->62394367] (req 62390272->62394367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.961858:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4ab6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62390272->62394367] (req 62390272->62394367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.968055:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982541 is committed 00002000:00010000:2.0:1789760773.968469:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62390272 -> 62394368 00002000:00010000:2.0:1789760773.968476:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121880 -> 121888 00010000:00010000:2.0:1789760773.968493:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4ab6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62390272->62394367] (req 62390272->62394367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.968519:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4ab6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62390272->62394367] (req 62390272->62394367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.968530:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4ab6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62390272->62394367] (req 62390272->62394367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.968538:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760773.968544:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4ab6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62390272->62394367] (req 62390272->62394367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.968555:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4ab6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62390272->62394367] (req 62390272->62394367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.968762:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4ab6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62390272->62394367] (req 62390272->62394367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.968773:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4ab6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62390272->62394367] (req 62390272->62394367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.968905:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554715136:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7166us (7757us total) trans 4294982541 rc 0/0 00000100:00100000:2.0:1789760773.968919:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15243 00000100:00100000:2.0:1789760773.968931:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717196823515 time 717207339425 state 0x2 ref 0 00000100:00100000:2.0:1789760773.973421:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760773.973600:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554715264 00000100:00100000:1.0:1789760773.973614:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554715264 00000100:00100000:1.0:1789760773.973688:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717196823515 time 717207339425 state 0x0 ref 1 00000100:00100000:1.0:1789760773.973708:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717207599841 time 717207339425 state 0x1 ref 1 00000100:00100000:1.0:1789760773.973814:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717207599841 time 717219431485 state 0x0 ref 1 00000100:00100000:1.0:1789760773.973840:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554715264:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760773.973906:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62394368, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dc20 00010000:00010000:1.0:1789760773.973917:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c4abd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.973937:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760773.973943:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760773.973951:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c4abd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62394368->62398463] (req 62394368->62398463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.973968:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c4abd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62394368->62398463] (req 62394368->62398463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.979314:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982542 is committed 00002000:00010000:1.0:1789760773.979936:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62394368 -> 62398464 00002000:00010000:1.0:1789760773.979944:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121888 -> 121896 00010000:00010000:1.0:1789760773.979963:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c4abd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62394368->62398463] (req 62394368->62398463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.979977:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c4abd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62394368->62398463] (req 62394368->62398463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.979987:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c4abd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62394368->62398463] (req 62394368->62398463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.979994:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655de00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760773.980000:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c4abd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62394368->62398463] (req 62394368->62398463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.980010:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c4abd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62394368->62398463] (req 62394368->62398463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.980040:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c4abd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62394368->62398463] (req 62394368->62398463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760773.980049:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c4abd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62394368->62398463] (req 62394368->62398463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760773.980323:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554715264:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6489us (6946us total) trans 4294982542 rc 0/0 00000100:00100000:1.0:1789760773.980337:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15244 00000100:00100000:1.0:1789760773.980350:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717207599841 time 717219431485 state 0x2 ref 0 00000100:00100000:1.0:1789760773.984287:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760773.984575:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554715392 00000100:00100000:3.0:1789760773.984588:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554715392 00000100:00100000:3.0:1789760773.984663:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717207599841 time 717219431485 state 0x0 ref 1 00000100:00100000:3.0:1789760773.984682:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717219691901 time 717219431485 state 0x1 ref 1 00000100:00100000:3.0:1789760773.984723:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717219691901 time 717230341256 state 0x0 ref 1 00000100:00100000:3.0:1789760773.984738:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554715392:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760773.984814:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62398464, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dc28 00010000:00010000:3.0:1789760773.984827:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4ac4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.984847:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760773.984852:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760773.984859:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4ac4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62398464->62402559] (req 62398464->62402559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760773.984876:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4ac4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62398464->62402559] (req 62398464->62402559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760773.990928:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982543 is committed 00002000:00010000:2.0:1789760773.991071:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62398464 -> 62402560 00002000:00010000:2.0:1789760773.991077:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121896 -> 121904 00010000:00010000:2.0:1789760773.991094:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4ac4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62398464->62402559] (req 62398464->62402559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.991109:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4ac4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62398464->62402559] (req 62398464->62402559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.991129:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4ac4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62398464->62402559] (req 62398464->62402559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.991138:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760773.991144:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4ac4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62398464->62402559] (req 62398464->62402559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.991154:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4ac4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62398464->62402559] (req 62398464->62402559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.991178:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4ac4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62398464->62402559] (req 62398464->62402559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.991187:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4ac4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62398464->62402559] (req 62398464->62402559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760773.991358:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554715392:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6615us (7081us total) trans 4294982543 rc 0/0 00000100:00100000:2.0:1789760773.991372:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15245 00000100:00100000:2.0:1789760773.991436:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717219691901 time 717230341256 state 0x2 ref 0 00000100:00100000:0.0:1789760773.995321:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760773.995638:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554715520 00000100:00100000:2.0:1789760773.995652:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554715520 00000100:00100000:2.0:1789760773.995724:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717219691901 time 717230341256 state 0x0 ref 1 00000100:00100000:2.0:1789760773.995753:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717230601672 time 717230341256 state 0x1 ref 1 00000100:00100000:2.0:1789760773.995792:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717230601672 time 717241410038 state 0x0 ref 1 00000100:00100000:2.0:1789760773.995807:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554715520:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760773.995868:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62402560, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dc30 00010000:00010000:2.0:1789760773.995879:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4acb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.995900:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760773.995906:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760773.995914:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4acb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62402560->62406655] (req 62402560->62406655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760773.995936:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4acb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62402560->62406655] (req 62402560->62406655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.002961:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982544 is committed 00002000:00010000:2.0:1789760774.003631:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62402560 -> 62406656 00002000:00010000:2.0:1789760774.003638:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121904 -> 121912 00010000:00010000:2.0:1789760774.003654:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4acb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62402560->62406655] (req 62402560->62406655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.003669:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4acb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62402560->62406655] (req 62402560->62406655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.003678:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4acb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62402560->62406655] (req 62402560->62406655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.003685:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760774.003690:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4acb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62402560->62406655] (req 62402560->62406655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.003699:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4acb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62402560->62406655] (req 62402560->62406655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.003722:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4acb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62402560->62406655] (req 62402560->62406655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.003731:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4acb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62402560->62406655] (req 62402560->62406655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.003913:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554715520:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8102us (8598us total) trans 4294982544 rc 0/0 00000100:00100000:2.0:1789760774.003926:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15246 00000100:00100000:2.0:1789760774.003938:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717230601672 time 717241410038 state 0x2 ref 0 00000100:00100000:1.0:1789760774.007331:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760774.007722:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554715648 00000100:00100000:2.0:1789760774.007736:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554715648 00000100:00100000:2.0:1789760774.007958:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717230601672 time 717241410038 state 0x0 ref 1 00000100:00100000:2.0:1789760774.007977:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717241670454 time 717241410038 state 0x1 ref 1 00000100:00100000:2.0:1789760774.008025:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717241670454 time 717253642819 state 0x0 ref 1 00000100:00100000:2.0:1789760774.008044:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554715648:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760774.008104:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62406656, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dc38 00010000:00010000:2.0:1789760774.008436:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4ad2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.008459:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760774.008464:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.008473:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4ad2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62406656->62410751] (req 62406656->62410751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.008492:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4ad2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62406656->62410751] (req 62406656->62410751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.015434:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982545 is committed 00002000:00010000:2.0:1789760774.017873:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62406656 -> 62410752 00002000:00010000:2.0:1789760774.017886:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121912 -> 121920 00010000:00010000:2.0:1789760774.017905:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4ad2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62406656->62410751] (req 62406656->62410751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.017921:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4ad2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62406656->62410751] (req 62406656->62410751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.017932:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4ad2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62406656->62410751] (req 62406656->62410751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.017940:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760774.017946:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4ad2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62406656->62410751] (req 62406656->62410751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.017956:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4ad2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62406656->62410751] (req 62406656->62410751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.017982:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4ad2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62406656->62410751] (req 62406656->62410751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.017990:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4ad2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62406656->62410751] (req 62406656->62410751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.018180:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554715648:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10137us (10926us total) trans 4294982545 rc 0/0 00000100:00100000:2.0:1789760774.018195:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15247 00000100:00100000:2.0:1789760774.018208:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717241670454 time 717253642819 state 0x2 ref 0 00000100:00100000:1.0:1789760774.022169:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760774.022267:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554715776 00000100:00100000:2.0:1789760774.022287:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554715776 00000100:00100000:2.0:1789760774.022359:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717241670454 time 717253642819 state 0x0 ref 1 00000100:00100000:2.0:1789760774.022412:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717253903235 time 717253642819 state 0x1 ref 1 00000100:00100000:2.0:1789760774.022448:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717253903235 time 717268065320 state 0x0 ref 1 00000100:00100000:2.0:1789760774.022462:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554715776:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760774.022817:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62410752, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dc40 00010000:00010000:2.0:1789760774.022830:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4ad9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.022849:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760774.022855:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.022863:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4ad9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62410752->62414847] (req 62410752->62414847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.022880:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4ad9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62410752->62414847] (req 62410752->62414847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.028217:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982546 is committed 00002000:00010000:2.0:1789760774.028620:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62410752 -> 62414848 00002000:00010000:2.0:1789760774.028627:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121920 -> 121928 00010000:00010000:2.0:1789760774.028646:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4ad9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62410752->62414847] (req 62410752->62414847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.028662:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4ad9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62410752->62414847] (req 62410752->62414847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.028673:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4ad9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62410752->62414847] (req 62410752->62414847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.028682:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760774.028687:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4ad9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62410752->62414847] (req 62410752->62414847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.028698:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4ad9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62410752->62414847] (req 62410752->62414847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.028722:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4ad9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62410752->62414847] (req 62410752->62414847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.028731:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4ad9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62410752->62414847] (req 62410752->62414847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.028957:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554715776:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6490us (6795us total) trans 4294982546 rc 0/0 00000100:00100000:2.0:1789760774.028968:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15248 00000100:00100000:2.0:1789760774.028979:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717253903235 time 717268065320 state 0x2 ref 0 00000100:00100000:0.0:1789760774.032455:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760774.033010:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554715904 00000100:00100000:1.0:1789760774.033023:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554715904 00000100:00100000:1.0:1789760774.033097:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717253903235 time 717268065320 state 0x0 ref 1 00000100:00100000:1.0:1789760774.033115:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717268325736 time 717268065320 state 0x1 ref 1 00000100:00100000:1.0:1789760774.033162:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717268325736 time 717278779561 state 0x0 ref 1 00000100:00100000:1.0:1789760774.033175:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554715904:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760774.033236:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62414848, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dc48 00010000:00010000:1.0:1789760774.033247:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4ae0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.033267:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760774.033272:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.033280:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4ae0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62414848->62418943] (req 62414848->62418943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.033296:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4ae0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62414848->62418943] (req 62414848->62418943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.038622:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982547 is committed 00002000:00010000:1.0:1789760774.040391:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62414848 -> 62418944 00002000:00010000:1.0:1789760774.040405:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121928 -> 121936 00010000:00010000:1.0:1789760774.040423:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4ae0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62414848->62418943] (req 62414848->62418943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.040438:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4ae0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62414848->62418943] (req 62414848->62418943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.040449:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4ae0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62414848->62418943] (req 62414848->62418943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.040457:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760774.040463:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4ae0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62414848->62418943] (req 62414848->62418943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.040474:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4ae0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62414848->62418943] (req 62414848->62418943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.040601:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4ae0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62414848->62418943] (req 62414848->62418943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.040611:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4ae0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62414848->62418943] (req 62414848->62418943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.042259:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554715904:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7565us (8295us total) trans 4294982547 rc 0/0 00000100:00100000:1.0:1789760774.042281:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15249 00000100:00100000:1.0:1789760774.042293:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717268325736 time 717278779561 state 0x2 ref 0 00000100:00100000:1.0:1789760774.044908:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760774.044993:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554716032 00000100:00100000:2.0:1789760774.045007:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554716032 00000100:00100000:2.0:1789760774.045078:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717268325736 time 717278779561 state 0x0 ref 1 00000100:00100000:2.0:1789760774.045096:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717279039977 time 717278779561 state 0x1 ref 1 00000100:00100000:1.0:1789760774.045197:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717279039977 time 717290812724 state 0x0 ref 1 00000100:00100000:1.0:1789760774.045212:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554716032:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760774.045269:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62418944, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dc50 00010000:00010000:1.0:1789760774.045278:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4ae7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.045299:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760774.045303:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.045311:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4ae7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62418944->62423039] (req 62418944->62423039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.045327:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4ae7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62418944->62423039] (req 62418944->62423039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.051951:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982548 is committed 00002000:00010000:2.0:1789760774.052168:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62418944 -> 62423040 00002000:00010000:2.0:1789760774.052174:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121936 -> 121944 00010000:00010000:2.0:1789760774.052193:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4ae7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62418944->62423039] (req 62418944->62423039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.052209:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4ae7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62418944->62423039] (req 62418944->62423039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.052220:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4ae7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62418944->62423039] (req 62418944->62423039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.052228:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760774.052234:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4ae7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62418944->62423039] (req 62418944->62423039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.052244:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4ae7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62418944->62423039] (req 62418944->62423039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.052267:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4ae7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62418944->62423039] (req 62418944->62423039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.052281:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4ae7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62418944->62423039] (req 62418944->62423039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.052597:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554716032:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7379us (7694us total) trans 4294982548 rc 0/0 00000100:00100000:2.0:1789760774.052610:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15250 00000100:00100000:2.0:1789760774.052623:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717279039977 time 717290812724 state 0x2 ref 0 00000100:00100000:2.0:1789760774.060079:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760774.060230:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554716160 00000100:00100000:3.0:1789760774.060240:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554716160 00000100:00100000:3.0:1789760774.060356:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717279039977 time 717290812724 state 0x0 ref 1 00000100:00100000:3.0:1789760774.060372:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717291073140 time 717290812724 state 0x1 ref 1 00000100:00100000:3.0:1789760774.060492:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717291073140 time 717306109462 state 0x0 ref 1 00000100:00100000:3.0:1789760774.060506:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554716160:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760774.060550:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62423040, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dc58 00010000:00010000:3.0:1789760774.060558:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4aee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.060585:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760774.060590:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.060596:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4aee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62423040->62427135] (req 62423040->62427135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.060610:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4aee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62423040->62427135] (req 62423040->62427135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.067440:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982549 is committed 00002000:00010000:3.0:1789760774.067999:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62423040 -> 62427136 00002000:00010000:3.0:1789760774.068007:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121944 -> 121952 00010000:00010000:3.0:1789760774.068024:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4aee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62423040->62427135] (req 62423040->62427135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.068040:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4aee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62423040->62427135] (req 62423040->62427135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.068051:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4aee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62423040->62427135] (req 62423040->62427135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.068059:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760774.068064:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4aee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62423040->62427135] (req 62423040->62427135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.068075:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4aee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62423040->62427135] (req 62423040->62427135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.068098:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4aee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62423040->62427135] (req 62423040->62427135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.068105:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4aee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62423040->62427135] (req 62423040->62427135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.068330:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554716160:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7818us (8251us total) trans 4294982549 rc 0/0 00000100:00100000:3.0:1789760774.068343:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15251 00000100:00100000:3.0:1789760774.068356:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717291073140 time 717306109462 state 0x2 ref 0 00000100:00100000:0.0:1789760774.073650:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760774.073887:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554716288 00000100:00100000:1.0:1789760774.073897:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554716288 00000100:00100000:1.0:1789760774.073951:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717291073140 time 717306109462 state 0x0 ref 1 00000100:00100000:1.0:1789760774.073973:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717306369878 time 717306109462 state 0x1 ref 1 00000100:00100000:1.0:1789760774.074007:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717306369878 time 717319624319 state 0x0 ref 1 00000100:00100000:1.0:1789760774.074021:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554716288:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760774.074080:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62427136, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dc60 00010000:00010000:1.0:1789760774.074091:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4af5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.074107:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760774.074285:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.074296:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4af5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62427136->62431231] (req 62427136->62431231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.074312:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4af5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62427136->62431231] (req 62427136->62431231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.082889:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982550 is committed 00002000:00010000:1.0:1789760774.083198:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62427136 -> 62431232 00002000:00010000:1.0:1789760774.083204:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121952 -> 121960 00010000:00010000:1.0:1789760774.083218:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4af5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62427136->62431231] (req 62427136->62431231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.083231:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4af5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62427136->62431231] (req 62427136->62431231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.083241:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4af5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62427136->62431231] (req 62427136->62431231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.083248:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760774.083253:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4af5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62427136->62431231] (req 62427136->62431231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.083267:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4af5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62427136->62431231] (req 62427136->62431231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.083287:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4af5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62427136->62431231] (req 62427136->62431231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.083295:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4af5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62427136->62431231] (req 62427136->62431231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.083651:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554716288:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9627us (10050us total) trans 4294982550 rc 0/0 00000100:00100000:1.0:1789760774.083663:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15252 00000100:00100000:1.0:1789760774.083673:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717306369878 time 717319624319 state 0x2 ref 0 00000100:00100000:1.0:1789760774.091679:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760774.093325:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554716416 00000100:00100000:2.0:1789760774.093338:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554716416 00000100:00100000:0.0:1789760774.094579:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717306369878 time 717319624319 state 0x0 ref 1 00000100:00100000:0.0:1789760774.094596:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717319884735 time 717319624319 state 0x1 ref 1 00000100:00100000:0.0:1789760774.094691:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717319884735 time 717340308230 state 0x0 ref 1 00000100:00100000:0.0:1789760774.094715:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554716416:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760774.094778:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62431232, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dc68 00010000:00010000:0.0:1789760774.094788:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4afc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.094802:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760774.094807:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.094814:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4afc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62431232->62435327] (req 62431232->62435327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.094844:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4afc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62431232->62435327] (req 62431232->62435327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.100015:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982551 is committed 00002000:00010000:0.0:1789760774.100192:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62431232 -> 62435328 00002000:00010000:0.0:1789760774.100197:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121960 -> 121968 00010000:00010000:0.0:1789760774.100209:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4afc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62431232->62435327] (req 62431232->62435327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.100223:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4afc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62431232->62435327] (req 62431232->62435327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.100233:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4afc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62431232->62435327] (req 62431232->62435327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.100241:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760774.100246:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4afc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62431232->62435327] (req 62431232->62435327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.100255:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4afc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62431232->62435327] (req 62431232->62435327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.100275:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4afc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62431232->62435327] (req 62431232->62435327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.100284:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4afc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62431232->62435327] (req 62431232->62435327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.101705:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554716416:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5674us (8707us total) trans 4294982551 rc 0/0 00000100:00100000:0.0:1789760774.101720:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15253 00000100:00100000:0.0:1789760774.101730:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717319884735 time 717340308230 state 0x2 ref 0 00000100:00100000:2.0:1789760774.108652:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760774.108796:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554716544 00000100:00100000:0.0:1789760774.108806:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554716544 00000100:00100000:0.0:1789760774.108862:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717319884735 time 717340308230 state 0x0 ref 1 00000100:00100000:0.0:1789760774.108877:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717340568646 time 717340308230 state 0x1 ref 1 00000100:00100000:0.0:1789760774.108912:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717340568646 time 717354529478 state 0x0 ref 1 00000100:00100000:0.0:1789760774.108925:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554716544:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760774.108973:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62435328, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dc70 00010000:00010000:0.0:1789760774.108983:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b03 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.108999:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760774.109003:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.109009:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62435328->62439423] (req 62435328->62439423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.109024:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62435328->62439423] (req 62435328->62439423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.114012:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982552 is committed 00002000:00010000:0.0:1789760774.114125:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62435328 -> 62439424 00002000:00010000:0.0:1789760774.114130:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121968 -> 121976 00010000:00010000:0.0:1789760774.114143:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62435328->62439423] (req 62435328->62439423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.114156:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b03 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62435328->62439423] (req 62435328->62439423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.114165:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b03 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62435328->62439423] (req 62435328->62439423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.114172:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760774.114177:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b03 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62435328->62439423] (req 62435328->62439423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.114186:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b03 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62435328->62439423] (req 62435328->62439423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.114205:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b03 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62435328->62439423] (req 62435328->62439423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.114212:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b03 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62435328->62439423] (req 62435328->62439423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.114301:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554716544:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5373us (5654us total) trans 4294982552 rc 0/0 00000100:00100000:0.0:1789760774.114312:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15254 00000100:00100000:0.0:1789760774.114322:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717340568646 time 717354529478 state 0x2 ref 0 00000100:00100000:3.0:1789760774.117692:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760774.117852:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554716672 00000100:00100000:2.0:1789760774.117862:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554716672 00000100:00100000:2.0:1789760774.117915:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717340568646 time 717354529478 state 0x0 ref 1 00000100:00100000:2.0:1789760774.117931:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717354789894 time 717354529478 state 0x1 ref 1 00000100:00100000:2.0:1789760774.117957:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717354789894 time 717363575956 state 0x0 ref 1 00000100:00100000:2.0:1789760774.117970:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554716672:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760774.118019:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62439424, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dc78 00010000:00010000:2.0:1789760774.118028:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b0a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.118047:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760774.118052:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.118058:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62439424->62443519] (req 62439424->62443519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.118072:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62439424->62443519] (req 62439424->62443519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.122056:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982553 is committed 00002000:00010000:2.0:1789760774.122185:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62439424 -> 62443520 00002000:00010000:2.0:1789760774.122191:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121976 -> 121984 00010000:00010000:2.0:1789760774.122206:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62439424->62443519] (req 62439424->62443519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.122219:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b0a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62439424->62443519] (req 62439424->62443519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.122228:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b0a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62439424->62443519] (req 62439424->62443519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.122234:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760774.122239:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b0a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62439424->62443519] (req 62439424->62443519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.122247:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b0a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62439424->62443519] (req 62439424->62443519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.122273:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b0a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62439424->62443519] (req 62439424->62443519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.122280:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b0a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62439424->62443519] (req 62439424->62443519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.122445:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554716672:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4407us (4692us total) trans 4294982553 rc 0/0 00000100:00100000:2.0:1789760774.122456:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15255 00000100:00100000:2.0:1789760774.122466:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717354789894 time 717363575956 state 0x2 ref 0 00000100:00100000:1.0:1789760774.130115:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760774.130260:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554716800 00000100:00100000:2.0:1789760774.130270:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554716800 00000100:00100000:2.0:1789760774.130318:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717354789894 time 717363575956 state 0x0 ref 1 00000100:00100000:2.0:1789760774.130333:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717363836372 time 717363575956 state 0x1 ref 1 00000100:00100000:2.0:1789760774.130366:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717363836372 time 717375984690 state 0x0 ref 1 00000100:00100000:2.0:1789760774.130404:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554716800:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760774.130452:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62443520, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dc80 00010000:00010000:2.0:1789760774.130462:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b11 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.130478:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760774.130483:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.130489:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62443520->62447615] (req 62443520->62447615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.130515:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62443520->62447615] (req 62443520->62447615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.140620:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982554 is committed 00002000:00010000:2.0:1789760774.140817:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62443520 -> 62447616 00002000:00010000:2.0:1789760774.140823:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121984 -> 121992 00010000:00010000:2.0:1789760774.140835:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62443520->62447615] (req 62443520->62447615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.140849:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b11 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62443520->62447615] (req 62443520->62447615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.140858:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62443520->62447615] (req 62443520->62447615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.140866:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760774.140871:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62443520->62447615] (req 62443520->62447615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.140880:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b11 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62443520->62447615] (req 62443520->62447615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.140899:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b11 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62443520->62447615] (req 62443520->62447615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.140913:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b11 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62443520->62447615] (req 62443520->62447615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.141017:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554716800:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10635us (10905us total) trans 4294982554 rc 0/0 00000100:00100000:2.0:1789760774.141028:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15256 00000100:00100000:2.0:1789760774.141040:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717363836372 time 717375984690 state 0x2 ref 0 00000100:00100000:1.0:1789760774.146006:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760774.146973:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554716928 00000100:00100000:0.0:1789760774.146985:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554716928 00000100:00100000:0.0:1789760774.147047:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717363836372 time 717375984690 state 0x0 ref 1 00000100:00100000:0.0:1789760774.147062:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717376245106 time 717375984690 state 0x1 ref 1 00000100:00100000:0.0:1789760774.147096:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717376245106 time 717392714969 state 0x0 ref 1 00000100:00100000:0.0:1789760774.147109:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554716928:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760774.147155:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62447616, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dc88 00010000:00010000:0.0:1789760774.147164:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b18 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.147180:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760774.147185:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.147191:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62447616->62451711] (req 62447616->62451711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.147204:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62447616->62451711] (req 62447616->62451711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.153091:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982555 is committed 00002000:00010000:0.0:1789760774.155282:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62447616 -> 62451712 00002000:00010000:0.0:1789760774.155290:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 121992 -> 122000 00010000:00010000:0.0:1789760774.155303:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62447616->62451711] (req 62447616->62451711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.155316:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b18 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62447616->62451711] (req 62447616->62451711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.155326:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b18 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62447616->62451711] (req 62447616->62451711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.155333:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760774.155338:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b18 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62447616->62451711] (req 62447616->62451711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.155348:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b18 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62447616->62451711] (req 62447616->62451711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.155365:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b18 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62447616->62451711] (req 62447616->62451711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.155373:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b18 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62447616->62451711] (req 62447616->62451711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.155649:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554716928:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8536us (9646us total) trans 4294982555 rc 0/0 00000100:00100000:0.0:1789760774.155659:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15257 00000100:00100000:0.0:1789760774.155669:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717376245106 time 717392714969 state 0x2 ref 0 00000100:00100000:3.0:1789760774.164751:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760774.165002:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554717056 00000100:00100000:0.0:1789760774.165011:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554717056 00000100:00100000:0.0:1789760774.165064:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717376245106 time 717392714969 state 0x0 ref 1 00000100:00100000:0.0:1789760774.165081:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717392975385 time 717392714969 state 0x1 ref 1 00000100:00100000:0.0:1789760774.165263:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717392975385 time 717410879826 state 0x0 ref 1 00000100:00100000:0.0:1789760774.165278:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554717056:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760774.165327:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62451712, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dc90 00010000:00010000:0.0:1789760774.165337:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b1f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.165353:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760774.165357:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.165363:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62451712->62455807] (req 62451712->62455807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.165399:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62451712->62455807] (req 62451712->62455807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.171100:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982556 is committed 00002000:00010000:2.0:1789760774.171816:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62451712 -> 62455808 00002000:00010000:2.0:1789760774.171822:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122000 -> 122008 00010000:00010000:2.0:1789760774.171839:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62451712->62455807] (req 62451712->62455807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.171850:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b1f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62451712->62455807] (req 62451712->62455807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.171857:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62451712->62455807] (req 62451712->62455807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.171863:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760774.171867:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62451712->62455807] (req 62451712->62455807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.171876:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b1f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62451712->62455807] (req 62451712->62455807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.171895:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b1f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62451712->62455807] (req 62451712->62455807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.171902:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4b1f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62451712->62455807] (req 62451712->62455807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.172033:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554717056:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6754us (7296us total) trans 4294982556 rc 0/0 00000100:00100000:2.0:1789760774.172044:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15258 00000100:00100000:2.0:1789760774.172053:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717392975385 time 717410879826 state 0x2 ref 0 00000100:00100000:1.0:1789760774.181244:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760774.182557:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554717184 00000100:00100000:2.0:1789760774.182571:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554717184 00000100:00100000:2.0:1789760774.182635:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717392975385 time 717410879826 state 0x0 ref 1 00000100:00100000:2.0:1789760774.182653:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717411140242 time 717410879826 state 0x1 ref 1 00000100:00100000:2.0:1789760774.182854:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717411140242 time 717428470831 state 0x0 ref 1 00000100:00100000:2.0:1789760774.182868:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554717184:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760774.182931:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62455808, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dc98 00010000:00010000:2.0:1789760774.182941:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b26 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.182958:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760774.182964:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.182971:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62455808->62459903] (req 62455808->62459903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.182986:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62455808->62459903] (req 62455808->62459903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.187213:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982557 is committed 00002000:00010000:3.0:1789760774.197235:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62455808 -> 62459904 00002000:00010000:3.0:1789760774.197244:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122008 -> 122016 00010000:00010000:3.0:1789760774.197257:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62455808->62459903] (req 62455808->62459903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.197268:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b26 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62455808->62459903] (req 62455808->62459903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.197277:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62455808->62459903] (req 62455808->62459903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.197284:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760774.197288:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62455808->62459903] (req 62455808->62459903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.197297:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b26 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62455808->62459903] (req 62455808->62459903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.197323:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b26 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62455808->62459903] (req 62455808->62459903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.197331:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4b26 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62455808->62459903] (req 62455808->62459903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.198333:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554717184:12345-192.168.203.22@tcp:4:dd.0 Request processed in 15458us (17089us total) trans 4294982557 rc 0/0 00000100:00100000:3.0:1789760774.198346:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15259 00000100:00100000:3.0:1789760774.198355:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717411140242 time 717428470831 state 0x2 ref 0 00000100:00100000:1.0:1789760774.208243:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760774.208306:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554717312 00000100:00100000:2.0:1789760774.208316:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554717312 00000100:00100000:2.0:1789760774.208368:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717411140242 time 717428470831 state 0x0 ref 1 00000100:00100000:2.0:1789760774.208532:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717428731247 time 717428470831 state 0x1 ref 1 00000100:00100000:2.0:1789760774.208609:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717428731247 time 717454226509 state 0x0 ref 1 00000100:00100000:2.0:1789760774.208623:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554717312:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760774.208664:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62459904, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dca0 00010000:00010000:2.0:1789760774.208671:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4b2d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.208684:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760774.208688:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.208701:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4b2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62459904->62463999] (req 62459904->62463999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.208713:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4b2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62459904->62463999] (req 62459904->62463999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.213017:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982558 is committed 00002000:00010000:2.0:1789760774.213221:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62459904 -> 62464000 00002000:00010000:2.0:1789760774.213227:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122016 -> 122024 00010000:00010000:2.0:1789760774.213241:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4b2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62459904->62463999] (req 62459904->62463999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.213253:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4b2d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62459904->62463999] (req 62459904->62463999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.213261:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4b2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62459904->62463999] (req 62459904->62463999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.213268:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760774.213272:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4b2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62459904->62463999] (req 62459904->62463999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.213281:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4b2d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62459904->62463999] (req 62459904->62463999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.213301:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4b2d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62459904->62463999] (req 62459904->62463999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.213309:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4b2d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62459904->62463999] (req 62459904->62463999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.213482:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554717312:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4856us (5242us total) trans 4294982558 rc 0/0 00000100:00100000:2.0:1789760774.213495:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15260 00000100:00100000:2.0:1789760774.213504:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717428731247 time 717454226509 state 0x2 ref 0 00000100:00100000:0.0:1789760774.216880:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760774.217054:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554717440 00000100:00100000:1.0:1789760774.217072:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554717440 00000100:00100000:1.0:1789760774.217124:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717428731247 time 717454226509 state 0x0 ref 1 00000100:00100000:1.0:1789760774.217143:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717454486925 time 717454226509 state 0x1 ref 1 00000100:00100000:1.0:1789760774.217178:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717454486925 time 717462796603 state 0x0 ref 1 00000100:00100000:1.0:1789760774.217190:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554717440:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760774.217236:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62464000, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dca8 00010000:00010000:1.0:1789760774.217245:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4b34 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.217261:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760774.217266:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.217272:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4b34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62464000->62468095] (req 62464000->62468095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.217285:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4b34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62464000->62468095] (req 62464000->62468095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.221733:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982559 is committed 00002000:00010000:1.0:1789760774.222027:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62464000 -> 62468096 00002000:00010000:1.0:1789760774.222032:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122024 -> 122032 00010000:00010000:1.0:1789760774.222045:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4b34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62464000->62468095] (req 62464000->62468095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.222372:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4b34 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62464000->62468095] (req 62464000->62468095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.222463:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4b34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62464000->62468095] (req 62464000->62468095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.222469:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760774.222473:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4b34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62464000->62468095] (req 62464000->62468095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.222480:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4b34 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62464000->62468095] (req 62464000->62468095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.222495:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4b34 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62464000->62468095] (req 62464000->62468095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.222548:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4b34 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62464000->62468095] (req 62464000->62468095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.222626:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554717440:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5429us (5748us total) trans 4294982559 rc 0/0 00000100:00100000:1.0:1789760774.222636:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15261 00000100:00100000:1.0:1789760774.222646:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717454486925 time 717462796603 state 0x2 ref 0 00000100:00100000:1.0:1789760774.227079:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760774.227159:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554717568 00000100:00100000:0.0:1789760774.227169:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554717568 00000100:00100000:0.0:1789760774.227224:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717454486925 time 717462796603 state 0x0 ref 1 00000100:00100000:0.0:1789760774.227239:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717463057019 time 717462796603 state 0x1 ref 1 00000100:00100000:0.0:1789760774.227272:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717463057019 time 717472890380 state 0x0 ref 1 00000100:00100000:0.0:1789760774.227284:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554717568:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760774.227338:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62468096, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dcb0 00010000:00010000:0.0:1789760774.227347:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4b3b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.227368:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760774.227373:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.227411:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4b3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62468096->62472191] (req 62468096->62472191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.227426:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4b3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62468096->62472191] (req 62468096->62472191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.232205:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982560 is committed 00002000:00010000:2.0:1789760774.232848:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62468096 -> 62472192 00002000:00010000:2.0:1789760774.232855:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122032 -> 122040 00010000:00010000:2.0:1789760774.232876:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4b3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62468096->62472191] (req 62468096->62472191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.232891:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4b3b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62468096->62472191] (req 62468096->62472191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.232900:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4b3b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62468096->62472191] (req 62468096->62472191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.232908:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760774.232913:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4b3b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62468096->62472191] (req 62468096->62472191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.232986:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4b3b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62468096->62472191] (req 62468096->62472191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.233007:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4b3b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62468096->62472191] (req 62468096->62472191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.233015:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4b3b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62468096->62472191] (req 62468096->62472191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.233107:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554717568:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5820us (6033us total) trans 4294982560 rc 0/0 00000100:00100000:2.0:1789760774.233118:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15262 00000100:00100000:2.0:1789760774.233128:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717463057019 time 717472890380 state 0x2 ref 0 00000100:00100000:2.0:1789760774.240698:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760774.241110:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554717696 00000100:00100000:3.0:1789760774.241131:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554717696 00000100:00100000:3.0:1789760774.241185:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717463057019 time 717472890380 state 0x0 ref 1 00000100:00100000:3.0:1789760774.241201:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717473150796 time 717472890380 state 0x1 ref 1 00000100:00100000:3.0:1789760774.241228:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717473150796 time 717486847029 state 0x0 ref 1 00000100:00100000:3.0:1789760774.241240:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554717696:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760774.241284:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62472192, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dcb8 00010000:00010000:3.0:1789760774.241295:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4b42 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.241310:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760774.241315:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.241322:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4b42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62472192->62476287] (req 62472192->62476287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.241334:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4b42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62472192->62476287] (req 62472192->62476287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.245504:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982561 is committed 00002000:00010000:3.0:1789760774.245619:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62472192 -> 62476288 00002000:00010000:3.0:1789760774.245625:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122040 -> 122048 00010000:00010000:3.0:1789760774.245649:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4b42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62472192->62476287] (req 62472192->62476287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.245661:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4b42 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62472192->62476287] (req 62472192->62476287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.245669:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4b42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62472192->62476287] (req 62472192->62476287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.245676:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760774.245680:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4b42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62472192->62476287] (req 62472192->62476287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.245688:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4b42 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62472192->62476287] (req 62472192->62476287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.245707:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4b42 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62472192->62476287] (req 62472192->62476287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.245721:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4b42 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62472192->62476287] (req 62472192->62476287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.245826:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554717696:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4584us (5135us total) trans 4294982561 rc 0/0 00000100:00100000:3.0:1789760774.245837:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15263 00000100:00100000:3.0:1789760774.245847:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717473150796 time 717486847029 state 0x2 ref 0 00000100:00100000:0.0:1789760774.250283:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760774.251060:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554717824 00000100:00100000:2.0:1789760774.251071:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554717824 00000100:00100000:2.0:1789760774.251130:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717473150796 time 717486847029 state 0x0 ref 1 00000100:00100000:2.0:1789760774.251148:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717487107445 time 717486847029 state 0x1 ref 1 00000100:00100000:2.0:1789760774.251173:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717487107445 time 717496791609 state 0x0 ref 1 00000100:00100000:2.0:1789760774.251187:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554717824:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760774.251235:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62476288, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dcc0 00010000:00010000:2.0:1789760774.251244:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4b49 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.251260:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760774.251264:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.251270:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4b49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62476288->62480383] (req 62476288->62480383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.251283:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4b49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62476288->62480383] (req 62476288->62480383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.256200:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982562 is committed 00002000:00010000:2.0:1789760774.256289:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62476288 -> 62480384 00002000:00010000:2.0:1789760774.256293:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122048 -> 122056 00010000:00010000:2.0:1789760774.256305:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4b49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62476288->62480383] (req 62476288->62480383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.256318:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4b49 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62476288->62480383] (req 62476288->62480383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.256327:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4b49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62476288->62480383] (req 62476288->62480383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.256334:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760774.256339:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4b49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62476288->62480383] (req 62476288->62480383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.256349:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4b49 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62476288->62480383] (req 62476288->62480383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.256370:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4b49 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62476288->62480383] (req 62476288->62480383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.256397:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4b49 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62476288->62480383] (req 62476288->62480383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.256530:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554717824:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5342us (6251us total) trans 4294982562 rc 0/0 00000100:00100000:2.0:1789760774.256541:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15264 00000100:00100000:2.0:1789760774.256550:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717487107445 time 717496791609 state 0x2 ref 0 00000100:00100000:1.0:1789760774.260226:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760774.260292:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554717952 00000100:00100000:0.0:1789760774.260301:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554717952 00000100:00100000:0.0:1789760774.260950:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717487107445 time 717496791609 state 0x0 ref 1 00000100:00100000:0.0:1789760774.260969:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717497052025 time 717496791609 state 0x1 ref 1 00000100:00100000:0.0:1789760774.261002:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717497052025 time 717506620599 state 0x0 ref 1 00000100:00100000:0.0:1789760774.261016:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554717952:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760774.261062:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62480384, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dcc8 00010000:00010000:0.0:1789760774.261071:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4b50 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.261086:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760774.261091:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.261097:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4b50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62480384->62484479] (req 62480384->62484479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.261109:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4b50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62480384->62484479] (req 62480384->62484479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.266997:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982563 is committed 00002000:00010000:0.0:1789760774.267175:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62480384 -> 62484480 00002000:00010000:0.0:1789760774.267180:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122056 -> 122064 00010000:00010000:0.0:1789760774.267190:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4b50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62480384->62484479] (req 62480384->62484479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.267202:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4b50 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62480384->62484479] (req 62480384->62484479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.267217:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4b50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62480384->62484479] (req 62480384->62484479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.267224:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760774.267228:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4b50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62480384->62484479] (req 62480384->62484479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.267236:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4b50 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62480384->62484479] (req 62480384->62484479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.267255:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4b50 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62480384->62484479] (req 62480384->62484479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.267263:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4b50 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62480384->62484479] (req 62480384->62484479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.267365:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554717952:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6347us (7140us total) trans 4294982563 rc 0/0 00000100:00100000:0.0:1789760774.267375:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15265 00000100:00100000:0.0:1789760774.267418:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717497052025 time 717506620599 state 0x2 ref 0 00000100:00100000:1.0:1789760774.275585:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760774.275859:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554718080 00000100:00100000:0.0:1789760774.275869:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554718080 00000100:00100000:0.0:1789760774.275920:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717497052025 time 717506620599 state 0x0 ref 1 00000100:00100000:0.0:1789760774.275934:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717506881015 time 717506620599 state 0x1 ref 1 00000100:00100000:0.0:1789760774.276284:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717506881015 time 717521901265 state 0x0 ref 1 00000100:00100000:0.0:1789760774.276303:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d81c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554718080:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760774.276354:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62484480, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dcd0 00010000:00010000:0.0:1789760774.276364:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4b57 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.276408:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760774.276413:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.276420:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4b57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62484480->62488575] (req 62484480->62488575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.276435:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4b57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62484480->62488575] (req 62484480->62488575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.282238:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982564 is committed 00002000:00010000:1.0:1789760774.282443:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62484480 -> 62488576 00002000:00010000:1.0:1789760774.282449:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122064 -> 122072 00010000:00010000:1.0:1789760774.282463:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4b57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62484480->62488575] (req 62484480->62488575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.282481:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4b57 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62484480->62488575] (req 62484480->62488575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.282491:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4b57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62484480->62488575] (req 62484480->62488575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.282498:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760774.282503:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4b57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62484480->62488575] (req 62484480->62488575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.282513:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4b57 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62484480->62488575] (req 62484480->62488575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.282533:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4b57 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62484480->62488575] (req 62484480->62488575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.282541:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4b57 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62484480->62488575] (req 62484480->62488575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.282714:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d81c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554718080:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6410us (7131us total) trans 4294982564 rc 0/0 00000100:00100000:1.0:1789760774.282726:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15266 00000100:00100000:1.0:1789760774.282735:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717506881015 time 717521901265 state 0x2 ref 0 00000100:00100000:3.0:1789760774.286330:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760774.286632:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554718208 00000100:00100000:0.0:1789760774.286649:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554718208 00000100:00100000:0.0:1789760774.286703:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717506881015 time 717521901265 state 0x0 ref 1 00000100:00100000:0.0:1789760774.286730:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717522161681 time 717521901265 state 0x1 ref 1 00000100:00100000:0.0:1789760774.286766:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717522161681 time 717532385174 state 0x0 ref 1 00000100:00100000:0.0:1789760774.286778:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554718208:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760774.286829:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62488576, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dcd8 00010000:00010000:0.0:1789760774.286838:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b5e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.286858:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760774.286862:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.286867:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62488576->62492671] (req 62488576->62492671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.286880:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62488576->62492671] (req 62488576->62492671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.291632:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982565 is committed 00002000:00010000:0.0:1789760774.292047:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62488576 -> 62492672 00002000:00010000:0.0:1789760774.292053:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122072 -> 122080 00010000:00010000:0.0:1789760774.292071:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62488576->62492671] (req 62488576->62492671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.292083:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b5e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62488576->62492671] (req 62488576->62492671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.292092:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62488576->62492671] (req 62488576->62492671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.292099:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760774.292105:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62488576->62492671] (req 62488576->62492671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.292164:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b5e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62488576->62492671] (req 62488576->62492671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.292191:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b5e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62488576->62492671] (req 62488576->62492671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.292197:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b5e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62488576->62492671] (req 62488576->62492671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.292337:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554718208:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5555us (6010us total) trans 4294982565 rc 0/0 00000100:00100000:0.0:1789760774.292348:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15267 00000100:00100000:0.0:1789760774.292358:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717522161681 time 717532385174 state 0x2 ref 0 00000100:00100000:2.0:1789760774.296104:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760774.296304:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554718336 00000100:00100000:1.0:1789760774.296316:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554718336 00000100:00100000:1.0:1789760774.296370:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717522161681 time 717532385174 state 0x0 ref 1 00000100:00100000:1.0:1789760774.296588:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717532645590 time 717532385174 state 0x1 ref 1 00000100:00100000:1.0:1789760774.296673:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717532645590 time 717542289822 state 0x0 ref 1 00000100:00100000:1.0:1789760774.296688:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554718336:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760774.296736:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62492672, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dce0 00010000:00010000:1.0:1789760774.296755:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4b65 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.296771:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760774.296776:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.296782:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4b65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62492672->62496767] (req 62492672->62496767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.296805:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4b65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62492672->62496767] (req 62492672->62496767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.300730:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982566 is committed 00002000:00010000:1.0:1789760774.301335:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62492672 -> 62496768 00002000:00010000:1.0:1789760774.301342:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122080 -> 122088 00010000:00010000:1.0:1789760774.301356:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4b65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62492672->62496767] (req 62492672->62496767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.301369:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4b65 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62492672->62496767] (req 62492672->62496767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.301474:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4b65 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62492672->62496767] (req 62492672->62496767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.301482:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760774.301487:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4b65 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62492672->62496767] (req 62492672->62496767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.301562:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4b65 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62492672->62496767] (req 62492672->62496767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.301584:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4b65 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62492672->62496767] (req 62492672->62496767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.301592:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4b65 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62492672->62496767] (req 62492672->62496767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.302022:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554718336:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5332us (5922us total) trans 4294982566 rc 0/0 00000100:00100000:1.0:1789760774.302033:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15268 00000100:00100000:1.0:1789760774.302043:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717532645590 time 717542289822 state 0x2 ref 0 00000100:00100000:1.0:1789760774.309968:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760774.310113:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554718464 00000100:00100000:0.0:1789760774.310294:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554718464 00000100:00100000:0.0:1789760774.310353:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717532645590 time 717542289822 state 0x0 ref 1 00000100:00100000:0.0:1789760774.310368:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717542550238 time 717542289822 state 0x1 ref 1 00000100:00100000:0.0:1789760774.310480:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717542550238 time 717556097709 state 0x0 ref 1 00000100:00100000:0.0:1789760774.310503:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d81880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554718464:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760774.310551:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62496768, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dce8 00010000:00010000:0.0:1789760774.310561:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b6c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.310576:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760774.310581:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.310588:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62496768->62500863] (req 62496768->62500863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.310601:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62496768->62500863] (req 62496768->62500863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.316978:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982567 is committed 00002000:00010000:0.0:1789760774.317229:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62496768 -> 62500864 00002000:00010000:0.0:1789760774.317234:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122088 -> 122096 00010000:00010000:0.0:1789760774.317249:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62496768->62500863] (req 62496768->62500863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.317262:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b6c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62496768->62500863] (req 62496768->62500863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.317271:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b6c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62496768->62500863] (req 62496768->62500863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.317278:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760774.317284:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b6c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62496768->62500863] (req 62496768->62500863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.317293:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b6c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62496768->62500863] (req 62496768->62500863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.317312:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b6c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62496768->62500863] (req 62496768->62500863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.317325:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b6c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62496768->62500863] (req 62496768->62500863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.317668:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d81880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554718464:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7171us (7704us total) trans 4294982567 rc 0/0 00000100:00100000:0.0:1789760774.317681:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15269 00000100:00100000:0.0:1789760774.317689:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717542550238 time 717556097709 state 0x2 ref 0 00000100:00100000:3.0:1789760774.323043:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760774.323213:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554718592 00000100:00100000:1.0:1789760774.323222:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554718592 00000100:00100000:1.0:1789760774.323278:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717542550238 time 717556097709 state 0x0 ref 1 00000100:00100000:1.0:1789760774.323296:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717556358125 time 717556097709 state 0x1 ref 1 00000100:00100000:1.0:1789760774.323329:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717556358125 time 717568947433 state 0x0 ref 1 00000100:00100000:1.0:1789760774.323342:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554718592:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760774.323592:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62500864, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dcf0 00010000:00010000:1.0:1789760774.323602:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b73 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.323619:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760774.323623:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.323630:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62500864->62504959] (req 62500864->62504959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.323642:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62500864->62504959] (req 62500864->62504959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.329005:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982568 is committed 00002000:00010000:1.0:1789760774.329144:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62500864 -> 62504960 00002000:00010000:1.0:1789760774.329149:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122096 -> 122104 00010000:00010000:1.0:1789760774.329164:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62500864->62504959] (req 62500864->62504959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.329176:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b73 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62500864->62504959] (req 62500864->62504959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.329185:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b73 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62500864->62504959] (req 62500864->62504959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.329193:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760774.329198:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b73 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62500864->62504959] (req 62500864->62504959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.329206:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b73 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62500864->62504959] (req 62500864->62504959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.329225:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b73 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62500864->62504959] (req 62500864->62504959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.329232:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b73 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62500864->62504959] (req 62500864->62504959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.329371:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554718592:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6024us (6327us total) trans 4294982568 rc 0/0 00000100:00100000:1.0:1789760774.329433:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15270 00000100:00100000:1.0:1789760774.329443:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717556358125 time 717568947433 state 0x2 ref 0 00000100:00100000:1.0:1789760774.337028:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760774.337123:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554718720 00000100:00100000:3.0:1789760774.337133:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554718720 00000100:00100000:3.0:1789760774.337186:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717556358125 time 717568947433 state 0x0 ref 1 00000100:00100000:3.0:1789760774.337202:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717569207849 time 717568947433 state 0x1 ref 1 00000100:00100000:3.0:1789760774.337237:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717569207849 time 717582855116 state 0x0 ref 1 00000100:00100000:3.0:1789760774.337250:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554718720:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760774.337300:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62504960, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dcf8 00010000:00010000:3.0:1789760774.337310:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4b7a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.337326:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760774.337331:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.337338:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4b7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62504960->62509055] (req 62504960->62509055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.337351:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4b7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62504960->62509055] (req 62504960->62509055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.345814:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982569 is committed 00002000:00010000:0.0:1789760774.346101:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62504960 -> 62509056 00002000:00010000:0.0:1789760774.346107:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122104 -> 122112 00010000:00010000:0.0:1789760774.346193:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4b7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62504960->62509055] (req 62504960->62509055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.346206:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4b7a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62504960->62509055] (req 62504960->62509055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.346214:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4b7a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62504960->62509055] (req 62504960->62509055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.346222:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760774.346226:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4b7a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62504960->62509055] (req 62504960->62509055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.346234:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4b7a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62504960->62509055] (req 62504960->62509055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.346258:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4b7a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62504960->62509055] (req 62504960->62509055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.346265:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4b7a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62504960->62509055] (req 62504960->62509055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.346429:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554718720:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9175us (9403us total) trans 4294982569 rc 0/0 00000100:00100000:0.0:1789760774.346441:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15271 00000100:00100000:0.0:1789760774.346451:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717569207849 time 717582855116 state 0x2 ref 0 00000100:00100000:2.0:1789760774.351936:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760774.352676:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554718848 00000100:00100000:1.0:1789760774.352686:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554718848 00000100:00100000:1.0:1789760774.352751:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717569207849 time 717582855116 state 0x0 ref 1 00000100:00100000:1.0:1789760774.352767:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717583115532 time 717582855116 state 0x1 ref 1 00000100:00100000:1.0:1789760774.352800:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717583115532 time 717598419092 state 0x0 ref 1 00000100:00100000:1.0:1789760774.352814:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554718848:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760774.352869:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62509056, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dd00 00010000:00010000:1.0:1789760774.352878:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b81 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.352895:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760774.352899:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.352905:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62509056->62513151] (req 62509056->62513151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.352919:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62509056->62513151] (req 62509056->62513151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.359019:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982570 is committed 00002000:00010000:1.0:1789760774.359197:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62509056 -> 62513152 00002000:00010000:1.0:1789760774.359203:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122112 -> 122120 00010000:00010000:1.0:1789760774.359218:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62509056->62513151] (req 62509056->62513151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.359230:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b81 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62509056->62513151] (req 62509056->62513151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.359239:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62509056->62513151] (req 62509056->62513151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.359247:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760774.359252:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62509056->62513151] (req 62509056->62513151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.359261:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b81 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62509056->62513151] (req 62509056->62513151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.359339:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b81 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62509056->62513151] (req 62509056->62513151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.359348:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b81 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62509056->62513151] (req 62509056->62513151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.359584:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554718848:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6767us (7652us total) trans 4294982570 rc 0/0 00000100:00100000:1.0:1789760774.359595:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15272 00000100:00100000:1.0:1789760774.359604:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717583115532 time 717598419092 state 0x2 ref 0 00000100:00100000:3.0:1789760774.365418:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760774.365519:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554718976 00000100:00100000:0.0:1789760774.365529:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554718976 00000100:00100000:0.0:1789760774.365583:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717583115532 time 717598419092 state 0x0 ref 1 00000100:00100000:0.0:1789760774.365598:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717598679508 time 717598419092 state 0x1 ref 1 00000100:00100000:0.0:1789760774.365631:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717598679508 time 717611249440 state 0x0 ref 1 00000100:00100000:0.0:1789760774.365644:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554718976:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760774.365698:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62513152, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dd08 00010000:00010000:0.0:1789760774.365707:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b88 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.365727:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760774.365732:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.365748:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62513152->62517247] (req 62513152->62517247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.365762:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62513152->62517247] (req 62513152->62517247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.370359:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982571 is committed 00002000:00010000:0.0:1789760774.375223:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62513152 -> 62517248 00002000:00010000:0.0:1789760774.375233:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122120 -> 122128 00010000:00010000:0.0:1789760774.375248:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62513152->62517247] (req 62513152->62517247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.375263:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b88 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62513152->62517247] (req 62513152->62517247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.375272:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62513152->62517247] (req 62513152->62517247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.375279:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760774.375284:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62513152->62517247] (req 62513152->62517247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.375293:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b88 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62513152->62517247] (req 62513152->62517247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.375490:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b88 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62513152->62517247] (req 62513152->62517247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.376292:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4b88 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62513152->62517247] (req 62513152->62517247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.376917:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554718976:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11266us (11538us total) trans 4294982571 rc 0/0 00000100:00100000:0.0:1789760774.376931:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15273 00000100:00100000:0.0:1789760774.376943:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717598679508 time 717611249440 state 0x2 ref 0 00000100:00100000:3.0:1789760774.383366:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760774.383719:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554719104 00000100:00100000:1.0:1789760774.383729:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554719104 00000100:00100000:1.0:1789760774.383794:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717598679508 time 717611249440 state 0x0 ref 1 00000100:00100000:1.0:1789760774.383810:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717611509856 time 717611249440 state 0x1 ref 1 00000100:00100000:1.0:1789760774.383854:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717611509856 time 717629471840 state 0x0 ref 1 00000100:00100000:1.0:1789760774.383866:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554719104:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760774.383918:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62517248, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dd10 00010000:00010000:1.0:1789760774.383928:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b8f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.383943:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760774.383947:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.383953:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62517248->62521343] (req 62517248->62521343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.383966:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62517248->62521343] (req 62517248->62521343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.389004:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982572 is committed 00002000:00010000:1.0:1789760774.389195:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62517248 -> 62521344 00002000:00010000:1.0:1789760774.389200:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122128 -> 122136 00010000:00010000:1.0:1789760774.389214:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62517248->62521343] (req 62517248->62521343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.389226:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b8f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62517248->62521343] (req 62517248->62521343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.389235:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b8f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62517248->62521343] (req 62517248->62521343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.389242:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760774.389247:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b8f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62517248->62521343] (req 62517248->62521343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.389256:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b8f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62517248->62521343] (req 62517248->62521343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.389276:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b8f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62517248->62521343] (req 62517248->62521343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.389284:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4b8f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62517248->62521343] (req 62517248->62521343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.389913:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554719104:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6043us (6550us total) trans 4294982572 rc 0/0 00000100:00100000:1.0:1789760774.389925:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15274 00000100:00100000:1.0:1789760774.390365:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717611509856 time 717629471840 state 0x2 ref 0 00000100:00100000:2.0:1789760774.395940:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760774.397247:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554719232 00000100:00100000:3.0:1789760774.397258:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554719232 00000100:00100000:3.0:1789760774.397550:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717611509856 time 717629471840 state 0x0 ref 1 00000100:00100000:3.0:1789760774.397565:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717629732256 time 717629471840 state 0x1 ref 1 00000100:00100000:3.0:1789760774.397723:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717629732256 time 717643339586 state 0x0 ref 1 00000100:00100000:3.0:1789760774.397737:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554719232:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760774.397791:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62521344, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dd18 00010000:00010000:3.0:1789760774.397800:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c4b96 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.397816:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760774.397820:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.397925:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c4b96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62521344->62525439] (req 62521344->62525439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.397944:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c4b96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62521344->62525439] (req 62521344->62525439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.402259:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982573 is committed 00002000:00010000:0.0:1789760774.402519:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62521344 -> 62525440 00002000:00010000:0.0:1789760774.402525:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122136 -> 122144 00010000:00010000:0.0:1789760774.402538:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c4b96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62521344->62525439] (req 62521344->62525439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.402550:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c4b96 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62521344->62525439] (req 62521344->62525439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.402558:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c4b96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62521344->62525439] (req 62521344->62525439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.402565:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760774.402570:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c4b96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62521344->62525439] (req 62521344->62525439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.402584:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c4b96 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62521344->62525439] (req 62521344->62525439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.402603:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c4b96 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62521344->62525439] (req 62521344->62525439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.402611:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c4b96 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62521344->62525439] (req 62521344->62525439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.402747:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554719232:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4999us (6802us total) trans 4294982573 rc 0/0 00000100:00100000:0.0:1789760774.402758:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15275 00000100:00100000:0.0:1789760774.402767:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717629732256 time 717643339586 state 0x2 ref 0 00000100:00100000:3.0:1789760774.408041:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760774.409192:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554719360 00000100:00100000:2.0:1789760774.409204:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554719360 00000100:00100000:2.0:1789760774.409257:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717629732256 time 717643339586 state 0x0 ref 1 00000100:00100000:2.0:1789760774.409271:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717643600002 time 717643339586 state 0x1 ref 1 00000100:00100000:2.0:1789760774.409360:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717643600002 time 717654977719 state 0x0 ref 1 00000100:00100000:2.0:1789760774.409375:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554719360:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760774.409748:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62525440, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dd20 00010000:00010000:2.0:1789760774.409759:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4b9d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.409776:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760774.409781:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.409787:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4b9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62525440->62529535] (req 62525440->62529535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.410183:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4b9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62525440->62529535] (req 62525440->62529535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.420205:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982574 is committed 00002000:00010000:2.0:1789760774.420343:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62525440 -> 62529536 00002000:00010000:2.0:1789760774.420349:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122144 -> 122152 00010000:00010000:2.0:1789760774.420362:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4b9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62525440->62529535] (req 62525440->62529535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.420375:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4b9d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62525440->62529535] (req 62525440->62529535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.420490:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4b9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62525440->62529535] (req 62525440->62529535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.420497:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760774.420503:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4b9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62525440->62529535] (req 62525440->62529535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.421707:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4b9d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62525440->62529535] (req 62525440->62529535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.421730:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4b9d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62525440->62529535] (req 62525440->62529535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.421753:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4b9d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62525440->62529535] (req 62525440->62529535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.422229:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554719360:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12850us (14189us total) trans 4294982574 rc 0/0 00000100:00100000:2.0:1789760774.422241:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15276 00000100:00100000:2.0:1789760774.422250:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717643600002 time 717654977719 state 0x2 ref 0 00000100:00100000:3.0:1789760774.428811:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760774.429065:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554719488 00000100:00100000:0.0:1789760774.429075:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554719488 00000100:00100000:0.0:1789760774.429126:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717643600002 time 717654977719 state 0x0 ref 1 00000100:00100000:0.0:1789760774.429141:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717655238135 time 717654977719 state 0x1 ref 1 00000100:00100000:0.0:1789760774.429170:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717655238135 time 717674789686 state 0x0 ref 1 00000100:00100000:0.0:1789760774.429181:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554719488:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760774.429223:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62529536, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dd28 00010000:00010000:0.0:1789760774.429231:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ba4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.429246:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760774.429251:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.429257:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ba4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62529536->62533631] (req 62529536->62533631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.429270:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ba4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62529536->62533631] (req 62529536->62533631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.439008:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982575 is committed 00002000:00010000:1.0:1789760774.439173:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62529536 -> 62533632 00002000:00010000:1.0:1789760774.439179:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122152 -> 122160 00010000:00010000:1.0:1789760774.439192:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ba4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62529536->62533631] (req 62529536->62533631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.439204:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ba4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62529536->62533631] (req 62529536->62533631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.439212:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ba4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62529536->62533631] (req 62529536->62533631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.439219:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760774.439224:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ba4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62529536->62533631] (req 62529536->62533631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.439234:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ba4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62529536->62533631] (req 62529536->62533631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.439262:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ba4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62529536->62533631] (req 62529536->62533631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.439272:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ba4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62529536->62533631] (req 62529536->62533631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.439584:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554719488:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10398us (10778us total) trans 4294982575 rc 0/0 00000100:00100000:1.0:1789760774.439595:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15277 00000100:00100000:1.0:1789760774.439604:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717655238135 time 717674789686 state 0x2 ref 0 00000100:00100000:1.0:1789760774.444630:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760774.445003:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554719616 00000100:00100000:2.0:1789760774.445013:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554719616 00000100:00100000:2.0:1789760774.445067:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717655238135 time 717674789686 state 0x0 ref 1 00000100:00100000:2.0:1789760774.445082:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717675050102 time 717674789686 state 0x1 ref 1 00000100:00100000:2.0:1789760774.445147:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717675050102 time 717690765531 state 0x0 ref 1 00000100:00100000:2.0:1789760774.445159:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554719616:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760774.445201:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62533632, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dd30 00010000:00010000:2.0:1789760774.445210:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4bab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.445226:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760774.445230:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.445236:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4bab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62533632->62537727] (req 62533632->62537727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.445264:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4bab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62533632->62537727] (req 62533632->62537727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760774.453083:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982576 is committed 00002000:00010000:2.0:1789760774.453943:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62533632 -> 62537728 00002000:00010000:2.0:1789760774.453950:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122160 -> 122168 00010000:00010000:2.0:1789760774.453963:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4bab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62533632->62537727] (req 62533632->62537727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.453976:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4bab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62533632->62537727] (req 62533632->62537727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.453987:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4bab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62533632->62537727] (req 62533632->62537727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.453995:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760774.454000:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4bab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62533632->62537727] (req 62533632->62537727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.454010:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4bab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62533632->62537727] (req 62533632->62537727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.454028:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4bab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62533632->62537727] (req 62533632->62537727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.454035:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4bab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62533632->62537727] (req 62533632->62537727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.454235:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554719616:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9071us (9620us total) trans 4294982576 rc 0/0 00000100:00100000:2.0:1789760774.454260:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15278 00000100:00100000:2.0:1789760774.454269:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717675050102 time 717690765531 state 0x2 ref 0 00000100:00100000:3.0:1789760774.458031:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760774.459038:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554719744 00000100:00100000:0.0:1789760774.459049:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554719744 00000100:00100000:0.0:1789760774.459105:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717675050102 time 717690765531 state 0x0 ref 1 00000100:00100000:0.0:1789760774.459132:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717691025947 time 717690765531 state 0x1 ref 1 00000100:00100000:0.0:1789760774.459153:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717691025947 time 717704772683 state 0x0 ref 1 00000100:00100000:0.0:1789760774.459165:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554719744:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760774.459215:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62537728, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dd38 00010000:00010000:0.0:1789760774.459223:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4bb2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.459346:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760774.459352:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.459359:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4bb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62537728->62541823] (req 62537728->62541823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.459375:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4bb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62537728->62541823] (req 62537728->62541823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.465666:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982577 is committed 00002000:00010000:0.0:1789760774.465785:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62537728 -> 62541824 00002000:00010000:0.0:1789760774.465790:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122168 -> 122176 00010000:00010000:0.0:1789760774.465803:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4bb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62537728->62541823] (req 62537728->62541823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.465814:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4bb2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62537728->62541823] (req 62537728->62541823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.465822:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4bb2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62537728->62541823] (req 62537728->62541823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.465828:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760774.465832:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4bb2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62537728->62541823] (req 62537728->62541823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.465841:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4bb2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62537728->62541823] (req 62537728->62541823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.465859:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4bb2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62537728->62541823] (req 62537728->62541823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.465866:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4bb2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62537728->62541823] (req 62537728->62541823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.465956:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554719744:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6788us (7929us total) trans 4294982577 rc 0/0 00000100:00100000:0.0:1789760774.465967:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15279 00000100:00100000:0.0:1789760774.465978:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717691025947 time 717704772683 state 0x2 ref 0 00000100:00100000:1.0:1789760774.470065:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760774.470236:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554719872 00000100:00100000:2.0:1789760774.470342:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554719872 00000100:00100000:2.0:1789760774.470571:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717691025947 time 717704772683 state 0x0 ref 1 00000100:00100000:2.0:1789760774.470588:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717705033099 time 717704772683 state 0x1 ref 1 00000100:00100000:2.0:1789760774.470656:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717705033099 time 717716273093 state 0x0 ref 1 00000100:00100000:2.0:1789760774.470670:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d82680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554719872:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760774.470717:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62541824, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dd40 00010000:00010000:2.0:1789760774.470727:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4bb9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.470753:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760774.470757:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.470764:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4bb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62541824->62545919] (req 62541824->62545919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.470778:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4bb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62541824->62545919] (req 62541824->62545919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.475446:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982578 is committed 00002000:00010000:2.0:1789760774.475569:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62541824 -> 62545920 00002000:00010000:2.0:1789760774.475574:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122176 -> 122184 00010000:00010000:2.0:1789760774.475586:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4bb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62541824->62545919] (req 62541824->62545919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.475599:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4bb9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62541824->62545919] (req 62541824->62545919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.475608:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4bb9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62541824->62545919] (req 62541824->62545919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.475627:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760774.475632:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4bb9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62541824->62545919] (req 62541824->62545919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.475640:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4bb9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62541824->62545919] (req 62541824->62545919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.475659:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4bb9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62541824->62545919] (req 62541824->62545919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.475667:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4bb9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62541824->62545919] (req 62541824->62545919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.475763:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d82680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554719872:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5092us (5702us total) trans 4294982578 rc 0/0 00000100:00100000:2.0:1789760774.475775:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15280 00000100:00100000:2.0:1789760774.475785:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717705033099 time 717716273093 state 0x2 ref 0 00000100:00100000:1.0:1789760774.481631:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760774.484548:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554720000 00000100:00100000:0.0:1789760774.484566:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554720000 00000100:00100000:0.0:1789760774.484733:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717705033099 time 717716273093 state 0x0 ref 1 00000100:00100000:0.0:1789760774.484759:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717716533509 time 717716273093 state 0x1 ref 1 00000100:00100000:0.0:1789760774.484794:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717716533509 time 717730412842 state 0x0 ref 1 00000100:00100000:0.0:1789760774.484807:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d81500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554720000:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760774.484852:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62545920, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dd48 00010000:00010000:0.0:1789760774.484861:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4bc0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.484877:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760774.484881:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.484887:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4bc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62545920->62550015] (req 62545920->62550015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.484901:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4bc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62545920->62550015] (req 62545920->62550015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.489830:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982579 is committed 00002000:00010000:0.0:1789760774.490244:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62545920 -> 62550016 00002000:00010000:0.0:1789760774.490251:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122184 -> 122192 00010000:00010000:0.0:1789760774.490273:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4bc0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62545920->62550015] (req 62545920->62550015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.490285:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4bc0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62545920->62550015] (req 62545920->62550015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.490294:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4bc0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62545920->62550015] (req 62545920->62550015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.490374:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760774.490402:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4bc0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62545920->62550015] (req 62545920->62550015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.490412:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4bc0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62545920->62550015] (req 62545920->62550015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.490432:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4bc0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62545920->62550015] (req 62545920->62550015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.490440:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4bc0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62545920->62550015] (req 62545920->62550015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.490678:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d81500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554720000:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5803us (8987us total) trans 4294982579 rc 0/0 00000100:00100000:0.0:1789760774.490690:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15281 00000100:00100000:0.0:1789760774.490700:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717716533509 time 717730412842 state 0x2 ref 0 00000100:00100000:3.0:1789760774.498614:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760774.498701:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554720128 00000100:00100000:1.0:1789760774.498711:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554720128 00000100:00100000:1.0:1789760774.498778:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717716533509 time 717730412842 state 0x0 ref 1 00000100:00100000:1.0:1789760774.498792:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717730673258 time 717730412842 state 0x1 ref 1 00000100:00100000:1.0:1789760774.498832:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717730673258 time 717744450292 state 0x0 ref 1 00000100:00100000:1.0:1789760774.498848:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb065a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554720128:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760774.498892:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62550016, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dd50 00010000:00010000:1.0:1789760774.498900:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bc7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.498915:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760774.498920:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.498927:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62550016->62554111] (req 62550016->62554111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.498941:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62550016->62554111] (req 62550016->62554111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.505099:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982580 is committed 00002000:00010000:1.0:1789760774.505271:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62550016 -> 62554112 00002000:00010000:1.0:1789760774.505277:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122192 -> 122200 00010000:00010000:1.0:1789760774.505290:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bc7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62550016->62554111] (req 62550016->62554111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.505433:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bc7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62550016->62554111] (req 62550016->62554111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.505443:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bc7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62550016->62554111] (req 62550016->62554111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.505451:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760774.505456:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bc7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62550016->62554111] (req 62550016->62554111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.505465:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bc7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62550016->62554111] (req 62550016->62554111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.505483:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bc7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62550016->62554111] (req 62550016->62554111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.505491:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bc7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62550016->62554111] (req 62550016->62554111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.505767:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554720128:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6916us (7158us total) trans 4294982580 rc 0/0 00000100:00100000:1.0:1789760774.505778:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15282 00000100:00100000:1.0:1789760774.505788:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717730673258 time 717744450292 state 0x2 ref 0 00000100:00100000:1.0:1789760774.513195:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760774.513296:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554720256 00000100:00100000:3.0:1789760774.513305:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554720256 00000100:00100000:3.0:1789760774.513356:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717730673258 time 717744450292 state 0x0 ref 1 00000100:00100000:3.0:1789760774.513371:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717744710708 time 717744450292 state 0x1 ref 1 00000100:00100000:3.0:1789760774.513435:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717744710708 time 717759052641 state 0x0 ref 1 00000100:00100000:3.0:1789760774.513448:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554720256:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760774.513512:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62554112, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dd58 00010000:00010000:3.0:1789760774.513522:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4bce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.513538:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760774.513543:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.513549:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4bce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62554112->62558207] (req 62554112->62558207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.513563:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4bce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62554112->62558207] (req 62554112->62558207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.518969:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982581 is committed 00002000:00010000:3.0:1789760774.520237:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62554112 -> 62558208 00002000:00010000:3.0:1789760774.520244:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122200 -> 122208 00010000:00010000:3.0:1789760774.520257:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4bce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62554112->62558207] (req 62554112->62558207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.520270:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4bce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62554112->62558207] (req 62554112->62558207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.520280:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4bce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62554112->62558207] (req 62554112->62558207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.520288:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760774.520293:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4bce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62554112->62558207] (req 62554112->62558207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.520304:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4bce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62554112->62558207] (req 62554112->62558207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.520322:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4bce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62554112->62558207] (req 62554112->62558207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.520335:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4bce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62554112->62558207] (req 62554112->62558207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.520515:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554720256:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7064us (7324us total) trans 4294982581 rc 0/0 00000100:00100000:3.0:1789760774.520528:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15283 00000100:00100000:3.0:1789760774.520538:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717744710708 time 717759052641 state 0x2 ref 0 00000100:00100000:2.0:1789760774.525005:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760774.525261:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554720384 00000100:00100000:1.0:1789760774.525273:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554720384 00000100:00100000:1.0:1789760774.525508:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717744710708 time 717759052641 state 0x0 ref 1 00000100:00100000:1.0:1789760774.525524:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717759313057 time 717759052641 state 0x1 ref 1 00000100:00100000:1.0:1789760774.525556:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717759313057 time 717771175061 state 0x0 ref 1 00000100:00100000:1.0:1789760774.525569:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554720384:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760774.525616:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62558208, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dd60 00010000:00010000:1.0:1789760774.525625:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bd5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.525639:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760774.525643:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.525650:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bd5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62558208->62562303] (req 62558208->62562303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.525663:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bd5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62558208->62562303] (req 62558208->62562303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.531917:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982582 is committed 00002000:00010000:3.0:1789760774.532033:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62558208 -> 62562304 00002000:00010000:3.0:1789760774.532039:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122208 -> 122216 00010000:00010000:3.0:1789760774.532052:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bd5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62558208->62562303] (req 62558208->62562303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.532065:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bd5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62558208->62562303] (req 62558208->62562303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.532075:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bd5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62558208->62562303] (req 62558208->62562303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.532082:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760774.532087:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bd5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62558208->62562303] (req 62558208->62562303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.532097:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bd5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62558208->62562303] (req 62558208->62562303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.532122:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bd5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62558208->62562303] (req 62558208->62562303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.532130:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bd5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62558208->62562303] (req 62558208->62562303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.532283:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554720384:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6710us (7282us total) trans 4294982582 rc 0/0 00000100:00100000:3.0:1789760774.532295:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15284 00000100:00100000:3.0:1789760774.532305:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717759313057 time 717771175061 state 0x2 ref 0 00000100:00100000:0.0:1789760774.539436:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760774.539505:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554720512 00000100:00100000:1.0:1789760774.539515:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554720512 00000100:00100000:1.0:1789760774.539568:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717759313057 time 717771175061 state 0x0 ref 1 00000100:00100000:1.0:1789760774.539583:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717771435477 time 717771175061 state 0x1 ref 1 00000100:00100000:1.0:1789760774.539670:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717771435477 time 717785285679 state 0x0 ref 1 00000100:00100000:1.0:1789760774.539684:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe1ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554720512:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760774.539730:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62562304, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dd68 00010000:00010000:1.0:1789760774.539751:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bdc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.539767:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760774.539772:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.539778:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bdc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62562304->62566399] (req 62562304->62566399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.539792:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bdc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62562304->62566399] (req 62562304->62566399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.545625:0:3647:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789760774.545646:0:3647:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb05d83100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3647:1876699573914368:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789760774.545714:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760774.545788:0:11526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573914368 00000100:00100000:1.0:1789760774.545795:0:11526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573914368 00000100:00100000:1.0:1789760774.545847:0:11526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 356 00000100:00100000:1.0:1789760774.545855:0:11526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d80e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3647:x1876699573914368:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1789760774.545877:0:11526:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484945600 pending 28672 free 3861176320 avail 3628105728 00000100:00100000:1.0:1789760774.545929:0:11526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d80e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3647:x1876699573914368:12345-0@lo:13:osp-pre-0-1.0 Request processed in 73us (214us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760774.545938:0:11526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 356 00000100:00080000:1.0:1789760774.545962:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760774.546011:0:3647:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb05d83100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3647:1876699573914368:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789760774.546628:0:3646:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789760774.546641:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03e5bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573914240:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760774.546679:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760774.546717:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573914240 00000100:00100000:3.0:1789760774.546722:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573914240 00000100:00100000:3.0:1789760774.546767:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 357 00000100:00100000:3.0:1789760774.546774:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb065a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573914240:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789760774.546787:0:8440:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484945600 pending 28672 free 3861176320 avail 3628105728 00000100:00100000:3.0:1789760774.546819:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573914240:12345-0@lo:13:osp-pre-0-0.0 Request processed in 46us (140us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760774.546826:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 357 00000100:00100000:3.0:1789760774.546858:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03e5bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573914240:0@lo:13:osp-pre-0-0.0 00000100:00080000:1.0:1789760774.546935:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760774.547835:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982583 is committed 00002000:00010000:1.0:1789760774.547939:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62562304 -> 62566400 00002000:00010000:1.0:1789760774.547943:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122216 -> 122224 00010000:00010000:1.0:1789760774.547955:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bdc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62562304->62566399] (req 62562304->62566399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.547967:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bdc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62562304->62566399] (req 62562304->62566399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.547978:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bdc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62562304->62566399] (req 62562304->62566399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.547986:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760774.547991:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bdc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62562304->62566399] (req 62562304->62566399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.548023:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bdc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62562304->62566399] (req 62562304->62566399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.548044:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bdc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62562304->62566399] (req 62562304->62566399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.548051:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4bdc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62562304->62566399] (req 62562304->62566399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.548135:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe1ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554720512:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8450us (8757us total) trans 4294982583 rc 0/0 00000100:00100000:1.0:1789760774.548144:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15285 00000100:00100000:1.0:1789760774.548154:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717771435477 time 717785285679 state 0x2 ref 0 00000100:00100000:1.0:1789760774.551942:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760774.552027:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554720640 00000100:00100000:2.0:1789760774.552037:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554720640 00000100:00100000:2.0:1789760774.552089:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717771435477 time 717785285679 state 0x0 ref 1 00000100:00100000:2.0:1789760774.552105:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717785546095 time 717785285679 state 0x1 ref 1 00000100:00100000:2.0:1789760774.552226:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717785546095 time 717797843066 state 0x0 ref 1 00000100:00100000:2.0:1789760774.552241:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554720640:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760774.552286:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62566400, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dd70 00010000:00010000:2.0:1789760774.552295:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4be3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.552311:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760774.552316:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.552322:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4be3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62566400->62570495] (req 62566400->62570495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.552335:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4be3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62566400->62570495] (req 62566400->62570495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.557260:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982584 is committed 00002000:00010000:2.0:1789760774.558088:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62566400 -> 62570496 00002000:00010000:2.0:1789760774.558097:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122224 -> 122232 00010000:00010000:2.0:1789760774.558111:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4be3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62566400->62570495] (req 62566400->62570495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.558291:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4be3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62566400->62570495] (req 62566400->62570495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.558301:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4be3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62566400->62570495] (req 62566400->62570495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.558308:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760774.558313:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4be3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62566400->62570495] (req 62566400->62570495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.558322:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4be3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62566400->62570495] (req 62566400->62570495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.558346:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4be3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62566400->62570495] (req 62566400->62570495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.558355:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4be3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62566400->62570495] (req 62566400->62570495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.558758:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554720640:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6515us (6820us total) trans 4294982584 rc 0/0 00000100:00100000:2.0:1789760774.558770:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15286 00000100:00100000:2.0:1789760774.558780:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717785546095 time 717797843066 state 0x2 ref 0 00000100:00100000:2.0:1789760774.563023:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760774.563089:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554720768 00000100:00100000:3.0:1789760774.563100:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554720768 00000100:00100000:3.0:1789760774.563166:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717785546095 time 717797843066 state 0x0 ref 1 00000100:00100000:3.0:1789760774.563182:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717798103482 time 717797843066 state 0x1 ref 1 00000100:00100000:3.0:1789760774.563359:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717798103482 time 717808975925 state 0x0 ref 1 00000100:00100000:3.0:1789760774.563374:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554720768:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760774.563468:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62570496, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dd78 00010000:00010000:3.0:1789760774.563477:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4bea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.563603:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:3.0:1789760774.563610:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.563616:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4bea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62570496->62574591] (req 62570496->62574591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.563630:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4bea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62570496->62574591] (req 62570496->62574591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.569205:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982585 is committed 00002000:00010000:3.0:1789760774.569335:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62570496 -> 62574592 00002000:00010000:3.0:1789760774.569341:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122232 -> 122240 00010000:00010000:3.0:1789760774.569355:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4bea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62570496->62574591] (req 62570496->62574591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.569366:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4bea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62570496->62574591] (req 62570496->62574591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.569407:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4bea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62570496->62574591] (req 62570496->62574591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.569415:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760774.569842:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4bea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62570496->62574591] (req 62570496->62574591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.569854:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4bea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62570496->62574591] (req 62570496->62574591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.569873:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4bea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62570496->62574591] (req 62570496->62574591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.569882:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4bea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62570496->62574591] (req 62570496->62574591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.569980:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554720768:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6606us (6962us total) trans 4294982585 rc 0/0 00000100:00100000:3.0:1789760774.569990:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15287 00000100:00100000:3.0:1789760774.569999:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717798103482 time 717808975925 state 0x2 ref 0 00000100:00100000:0.0:1789760774.573197:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760774.573293:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554720896 00000100:00100000:3.0:1789760774.573304:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554720896 00000100:00100000:3.0:1789760774.573359:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717798103482 time 717808975925 state 0x0 ref 1 00000100:00100000:3.0:1789760774.573375:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717809236341 time 717808975925 state 0x1 ref 1 00000100:00100000:3.0:1789760774.573437:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717809236341 time 717819055341 state 0x0 ref 1 00000100:00100000:3.0:1789760774.573449:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554720896:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760774.574052:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62574592, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dd80 00010000:00010000:0.0:1789760774.574064:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4bf1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.574081:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760774.574085:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.574092:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4bf1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62574592->62578687] (req 62574592->62578687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.574106:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4bf1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62574592->62578687] (req 62574592->62578687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.578957:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982586 is committed 00002000:00010000:0.0:1789760774.579065:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62574592 -> 62578688 00002000:00010000:0.0:1789760774.579070:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122240 -> 122248 00010000:00010000:0.0:1789760774.579082:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4bf1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62574592->62578687] (req 62574592->62578687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.579096:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4bf1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62574592->62578687] (req 62574592->62578687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.579110:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4bf1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62574592->62578687] (req 62574592->62578687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.579129:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760774.579134:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4bf1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62574592->62578687] (req 62574592->62578687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.579144:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4bf1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62574592->62578687] (req 62574592->62578687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.579163:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4bf1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62574592->62578687] (req 62574592->62578687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.579172:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4bf1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62574592->62578687] (req 62574592->62578687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.579265:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554720896:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5813us (6073us total) trans 4294982586 rc 0/0 00000100:00100000:0.0:1789760774.579276:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15288 00000100:00100000:0.0:1789760774.579286:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717809236341 time 717819055341 state 0x2 ref 0 00000100:00100000:1.0:1789760774.584744:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760774.584830:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554721024 00000100:00100000:0.0:1789760774.584839:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554721024 00000100:00100000:0.0:1789760774.584893:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717809236341 time 717819055341 state 0x0 ref 1 00000100:00100000:0.0:1789760774.584907:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717819315757 time 717819055341 state 0x1 ref 1 00000100:00100000:0.0:1789760774.584939:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717819315757 time 717830556793 state 0x0 ref 1 00000100:00100000:0.0:1789760774.584951:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d81c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554721024:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760774.584993:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62578688, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dd88 00010000:00010000:0.0:1789760774.585005:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4bf8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.585021:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760774.585025:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.585033:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4bf8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62578688->62582783] (req 62578688->62582783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.585046:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4bf8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62578688->62582783] (req 62578688->62582783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.588735:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982587 is committed 00002000:00010000:0.0:1789760774.588861:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62578688 -> 62582784 00002000:00010000:0.0:1789760774.588867:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122248 -> 122256 00010000:00010000:0.0:1789760774.588880:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4bf8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62578688->62582783] (req 62578688->62582783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.588893:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4bf8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62578688->62582783] (req 62578688->62582783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.588902:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4bf8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62578688->62582783] (req 62578688->62582783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.588915:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760774.588920:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4bf8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62578688->62582783] (req 62578688->62582783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.588929:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4bf8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62578688->62582783] (req 62578688->62582783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.588948:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4bf8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62578688->62582783] (req 62578688->62582783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.588956:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4bf8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62578688->62582783] (req 62578688->62582783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.589046:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d81c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554721024:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4092us (4315us total) trans 4294982587 rc 0/0 00000100:00100000:0.0:1789760774.589057:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15289 00000100:00100000:0.0:1789760774.589066:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717819315757 time 717830556793 state 0x2 ref 0 00000100:00100000:3.0:1789760774.593262:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760774.593429:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554721152 00000100:00100000:1.0:1789760774.593438:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554721152 00000100:00100000:1.0:1789760774.593490:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717819315757 time 717830556793 state 0x0 ref 1 00000100:00100000:1.0:1789760774.593515:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717830817209 time 717830556793 state 0x1 ref 1 00000100:00100000:1.0:1789760774.593541:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717830817209 time 717839159429 state 0x0 ref 1 00000100:00100000:1.0:1789760774.593554:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb065a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554721152:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760774.593601:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62582784, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dd90 00010000:00010000:1.0:1789760774.593610:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4bff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.593625:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760774.593629:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.593636:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4bff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62582784->62586879] (req 62582784->62586879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.593650:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4bff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62582784->62586879] (req 62582784->62586879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.599205:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982588 is committed 00002000:00010000:1.0:1789760774.599586:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62582784 -> 62586880 00002000:00010000:1.0:1789760774.599592:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122256 -> 122264 00010000:00010000:1.0:1789760774.599606:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4bff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62582784->62586879] (req 62582784->62586879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.599619:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4bff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62582784->62586879] (req 62582784->62586879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.599634:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4bff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62582784->62586879] (req 62582784->62586879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.599642:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760774.599648:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4bff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62582784->62586879] (req 62582784->62586879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.599657:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4bff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62582784->62586879] (req 62582784->62586879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.599675:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4bff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62582784->62586879] (req 62582784->62586879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.599683:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4bff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62582784->62586879] (req 62582784->62586879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.599824:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554721152:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6267us (6566us total) trans 4294982588 rc 0/0 00000100:00100000:1.0:1789760774.599835:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15290 00000100:00100000:1.0:1789760774.599844:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717830817209 time 717839159429 state 0x2 ref 0 00000100:00100000:3.0:1789760774.604260:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760774.604822:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554721280 00000100:00100000:0.0:1789760774.604832:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554721280 00000100:00100000:1.0:1789760774.605705:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717830817209 time 717839159429 state 0x0 ref 1 00000100:00100000:1.0:1789760774.605723:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717839419845 time 717839159429 state 0x1 ref 1 00000100:00100000:1.0:1789760774.605800:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717839419845 time 717851418348 state 0x0 ref 1 00000100:00100000:1.0:1789760774.605815:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb065a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554721280:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760774.605859:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62586880, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dd98 00010000:00010000:1.0:1789760774.605868:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c06 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.605883:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760774.605887:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.605893:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62586880->62590975] (req 62586880->62590975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.605907:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62586880->62590975] (req 62586880->62590975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.614046:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982589 is committed 00002000:00010000:1.0:1789760774.614296:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62586880 -> 62590976 00002000:00010000:1.0:1789760774.614303:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122264 -> 122272 00010000:00010000:1.0:1789760774.614315:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62586880->62590975] (req 62586880->62590975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.614328:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c06 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62586880->62590975] (req 62586880->62590975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.614337:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c06 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62586880->62590975] (req 62586880->62590975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.614348:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760774.614353:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c06 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62586880->62590975] (req 62586880->62590975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.614362:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c06 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62586880->62590975] (req 62586880->62590975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.614780:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c06 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62586880->62590975] (req 62586880->62590975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.614797:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c06 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62586880->62590975] (req 62586880->62590975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.614894:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554721280:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9078us (10640us total) trans 4294982589 rc 0/0 00000100:00100000:1.0:1789760774.614906:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15291 00000100:00100000:1.0:1789760774.614915:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717839419845 time 717851418348 state 0x2 ref 0 00000100:00100000:3.0:1789760774.620828:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760774.620910:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554721408 00000100:00100000:0.0:1789760774.620921:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554721408 00000100:00100000:0.0:1789760774.620972:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717839419845 time 717851418348 state 0x0 ref 1 00000100:00100000:0.0:1789760774.620987:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717851678764 time 717851418348 state 0x1 ref 1 00000100:00100000:0.0:1789760774.621023:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717851678764 time 717866641579 state 0x0 ref 1 00000100:00100000:0.0:1789760774.621036:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb065a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554721408:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760774.621083:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62590976, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dda0 00010000:00010000:0.0:1789760774.621092:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4c0d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.621107:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760774.621112:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.621119:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4c0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62590976->62595071] (req 62590976->62595071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.621134:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4c0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62590976->62595071] (req 62590976->62595071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.625846:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982590 is committed 00002000:00010000:0.0:1789760774.627924:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62590976 -> 62595072 00002000:00010000:0.0:1789760774.627932:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122272 -> 122280 00010000:00010000:0.0:1789760774.627945:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4c0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62590976->62595071] (req 62590976->62595071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.627959:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4c0d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62590976->62595071] (req 62590976->62595071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.627968:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4c0d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62590976->62595071] (req 62590976->62595071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.627982:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760774.627987:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4c0d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62590976->62595071] (req 62590976->62595071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.627997:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4c0d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62590976->62595071] (req 62590976->62595071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.628017:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4c0d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62590976->62595071] (req 62590976->62595071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.628026:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4c0d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62590976->62595071] (req 62590976->62595071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.628173:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554721408:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7135us (7351us total) trans 4294982590 rc 0/0 00000100:00100000:0.0:1789760774.628185:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15292 00000100:00100000:0.0:1789760774.628194:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717851678764 time 717866641579 state 0x2 ref 0 00000100:00100000:2.0:1789760774.631955:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760774.634301:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554721536 00000100:00100000:0.0:1789760774.634317:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554721536 00000100:00100000:1.0:1789760774.634482:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717851678764 time 717866641579 state 0x0 ref 1 00000100:00100000:1.0:1789760774.634509:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717866901995 time 717866641579 state 0x1 ref 1 00000100:00100000:1.0:1789760774.634542:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717866901995 time 717880160755 state 0x0 ref 1 00000100:00100000:1.0:1789760774.634554:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554721536:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760774.634600:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62595072, mtime: 0x6aad9505, atime: 0x0, ctime: 0x6aad9505, blocks: 0x1dda8 00010000:00010000:1.0:1789760774.634608:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c14 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.634622:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760774.634627:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.634632:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62595072->62599167] (req 62595072->62599167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.634644:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62595072->62599167] (req 62595072->62599167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.641098:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982591 is committed 00002000:00010000:1.0:1789760774.641286:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62595072 -> 62599168 00002000:00010000:1.0:1789760774.641349:0:8442:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760773 -> 1789760774 00002000:00010000:1.0:1789760774.641352:0:8442:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760773 -> 1789760774 00002000:00010000:1.0:1789760774.641354:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122280 -> 122288 00010000:00010000:1.0:1789760774.641368:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62595072->62599167] (req 62595072->62599167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.641419:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c14 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62595072->62599167] (req 62595072->62599167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.641427:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c14 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62595072->62599167] (req 62595072->62599167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.641762:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760774.641768:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c14 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62595072->62599167] (req 62595072->62599167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.641778:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c14 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62595072->62599167] (req 62595072->62599167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.641799:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c14 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62595072->62599167] (req 62595072->62599167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.641806:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c14 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62595072->62599167] (req 62595072->62599167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.641894:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554721536:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7338us (9945us total) trans 4294982591 rc 0/0 00000100:00100000:1.0:1789760774.641905:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15293 00000100:00100000:1.0:1789760774.641914:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717866901995 time 717880160755 state 0x2 ref 0 00000100:00100000:3.0:1789760774.647300:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760774.647664:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554721664 00000100:00100000:1.0:1789760774.647733:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554721664 00000100:00100000:1.0:1789760774.647880:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717866901995 time 717880160755 state 0x0 ref 1 00000100:00100000:1.0:1789760774.647895:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717880421171 time 717880160755 state 0x1 ref 1 00000100:00100000:0.0:1789760774.648211:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717880421171 time 717893826206 state 0x0 ref 1 00000100:00100000:0.0:1789760774.648228:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb065a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554721664:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760774.648272:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62599168, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1ddb0 00010000:00010000:0.0:1789760774.648282:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4c1b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.648298:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760774.648302:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.648309:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4c1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62599168->62603263] (req 62599168->62603263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.648322:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4c1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62599168->62603263] (req 62599168->62603263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.654099:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982592 is committed 00002000:00010000:1.0:1789760774.654262:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62599168 -> 62603264 00002000:00010000:1.0:1789760774.654267:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122288 -> 122296 00010000:00010000:1.0:1789760774.654280:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4c1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62599168->62603263] (req 62599168->62603263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.654297:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4c1b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62599168->62603263] (req 62599168->62603263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.654306:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4c1b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62599168->62603263] (req 62599168->62603263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.654314:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760774.654319:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4c1b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62599168->62603263] (req 62599168->62603263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.654417:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4c1b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62599168->62603263] (req 62599168->62603263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.654437:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4c1b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62599168->62603263] (req 62599168->62603263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.654445:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4c1b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62599168->62603263] (req 62599168->62603263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.654533:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554721664:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6306us (7237us total) trans 4294982592 rc 0/0 00000100:00100000:1.0:1789760774.654544:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15294 00000100:00100000:1.0:1789760774.654553:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717880421171 time 717893826206 state 0x2 ref 0 00000100:00100000:3.0:1789760774.661097:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760774.661334:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554721792 00000100:00100000:1.0:1789760774.661345:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554721792 00000100:00100000:1.0:1789760774.661412:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717880421171 time 717893826206 state 0x0 ref 1 00000100:00100000:1.0:1789760774.661426:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717894086622 time 717893826206 state 0x1 ref 1 00000100:00100000:1.0:1789760774.661464:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717894086622 time 717907082861 state 0x0 ref 1 00000100:00100000:1.0:1789760774.661478:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb065a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554721792:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760774.661522:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62603264, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1ddb8 00010000:00010000:1.0:1789760774.661532:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c22 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.661548:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760774.661552:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.661559:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62603264->62607359] (req 62603264->62607359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.661573:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62603264->62607359] (req 62603264->62607359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.669438:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982593 is committed 00002000:00010000:1.0:1789760774.670093:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62603264 -> 62607360 00002000:00010000:1.0:1789760774.670100:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122296 -> 122304 00010000:00010000:1.0:1789760774.670172:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62603264->62607359] (req 62603264->62607359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.670187:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c22 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62603264->62607359] (req 62603264->62607359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.670201:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c22 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62603264->62607359] (req 62603264->62607359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.670208:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760774.670219:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c22 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62603264->62607359] (req 62603264->62607359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.670229:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c22 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62603264->62607359] (req 62603264->62607359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.670250:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c22 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62603264->62607359] (req 62603264->62607359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.670258:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c22 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62603264->62607359] (req 62603264->62607359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.670497:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554721792:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8904us (9291us total) trans 4294982593 rc 0/0 00000100:00100000:1.0:1789760774.670508:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15295 00000100:00100000:1.0:1789760774.670518:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717894086622 time 717907082861 state 0x2 ref 0 00000100:00100000:2.0:1789760774.674712:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760774.674793:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554721920 00000100:00100000:1.0:1789760774.674802:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554721920 00000100:00100000:1.0:1789760774.674855:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717894086622 time 717907082861 state 0x0 ref 1 00000100:00100000:1.0:1789760774.674869:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717907343277 time 717907082861 state 0x1 ref 1 00000100:00100000:1.0:1789760774.674902:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717907343277 time 717920521116 state 0x0 ref 1 00000100:00100000:1.0:1789760774.674913:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554721920:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760774.674956:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62607360, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1ddc0 00010000:00010000:1.0:1789760774.674966:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c29 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.674982:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760774.674986:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.674992:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62607360->62611455] (req 62607360->62611455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.675005:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62607360->62611455] (req 62607360->62611455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.687012:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982594 is committed 00002000:00010000:1.0:1789760774.687164:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62607360 -> 62611456 00002000:00010000:1.0:1789760774.687170:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122304 -> 122312 00010000:00010000:1.0:1789760774.687183:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62607360->62611455] (req 62607360->62611455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.687197:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c29 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62607360->62611455] (req 62607360->62611455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.687205:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c29 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62607360->62611455] (req 62607360->62611455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.687218:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760774.687223:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c29 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62607360->62611455] (req 62607360->62611455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.687231:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c29 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62607360->62611455] (req 62607360->62611455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.687250:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c29 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62607360->62611455] (req 62607360->62611455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.687258:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c29 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62607360->62611455] (req 62607360->62611455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.687414:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554721920:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12466us (12685us total) trans 4294982594 rc 0/0 00000100:00100000:1.0:1789760774.687424:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15296 00000100:00100000:1.0:1789760774.687433:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717907343277 time 717920521116 state 0x2 ref 0 00000100:00100000:3.0:1789760774.692991:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760774.695218:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554722048 00000100:00100000:2.0:1789760774.695233:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554722048 00000100:00100000:2.0:1789760774.695295:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717907343277 time 717920521116 state 0x0 ref 1 00000100:00100000:2.0:1789760774.695310:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717920781532 time 717920521116 state 0x1 ref 1 00000100:00100000:2.0:1789760774.695345:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717920781532 time 717940963197 state 0x0 ref 1 00000100:00100000:2.0:1789760774.695357:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb065a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554722048:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760774.695439:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62611456, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1ddc8 00010000:00010000:2.0:1789760774.695448:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4c30 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.695462:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:2.0:1789760774.695467:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.695473:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4c30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62611456->62615551] (req 62611456->62615551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.695486:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4c30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62611456->62615551] (req 62611456->62615551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.702564:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982595 is committed 00002000:00010000:2.0:1789760774.702686:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62611456 -> 62615552 00002000:00010000:2.0:1789760774.702692:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122312 -> 122320 00010000:00010000:2.0:1789760774.703112:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4c30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62611456->62615551] (req 62611456->62615551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.703135:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4c30 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62611456->62615551] (req 62611456->62615551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.703144:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4c30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62611456->62615551] (req 62611456->62615551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.703151:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760774.703156:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4c30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62611456->62615551] (req 62611456->62615551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.703164:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4c30 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62611456->62615551] (req 62611456->62615551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.703184:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4c30 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62611456->62615551] (req 62611456->62615551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.703192:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4c30 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62611456->62615551] (req 62611456->62615551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.703301:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554722048:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7941us (10319us total) trans 4294982595 rc 0/0 00000100:00100000:2.0:1789760774.703311:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15297 00000100:00100000:2.0:1789760774.703320:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717920781532 time 717940963197 state 0x2 ref 0 00000100:00100000:3.0:1789760774.709560:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760774.710517:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554722176 00000100:00100000:0.0:1789760774.710527:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554722176 00000100:00100000:0.0:1789760774.710579:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717920781532 time 717940963197 state 0x0 ref 1 00000100:00100000:0.0:1789760774.710594:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717941223613 time 717940963197 state 0x1 ref 1 00000100:00100000:0.0:1789760774.710683:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717941223613 time 717956299961 state 0x0 ref 1 00000100:00100000:0.0:1789760774.710697:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb065a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554722176:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760774.710759:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62615552, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1ddd0 00010000:00010000:0.0:1789760774.710768:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4c37 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.710789:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:0.0:1789760774.710793:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.710799:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4c37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62615552->62619647] (req 62615552->62619647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.710813:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4c37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62615552->62619647] (req 62615552->62619647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.718979:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982596 is committed 00002000:00010000:0.0:1789760774.719108:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62615552 -> 62619648 00002000:00010000:0.0:1789760774.719123:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122320 -> 122328 00010000:00010000:0.0:1789760774.719142:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4c37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62615552->62619647] (req 62615552->62619647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.719157:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4c37 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62615552->62619647] (req 62615552->62619647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.719166:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4c37 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62615552->62619647] (req 62615552->62619647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.719174:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760774.719179:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4c37 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62615552->62619647] (req 62615552->62619647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.719189:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4c37 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62615552->62619647] (req 62615552->62619647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.719213:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4c37 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62615552->62619647] (req 62615552->62619647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.719221:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4c37 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62615552->62619647] (req 62615552->62619647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.719323:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554722176:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8623us (9765us total) trans 4294982596 rc 0/0 00000100:00100000:0.0:1789760774.719335:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15298 00000100:00100000:0.0:1789760774.719346:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717941223613 time 717956299961 state 0x2 ref 0 00000100:00100000:0.0:1789760774.726219:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760774.726335:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554722304 00000100:00100000:1.0:1789760774.726344:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554722304 00000100:00100000:1.0:1789760774.726421:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717941223613 time 717956299961 state 0x0 ref 1 00000100:00100000:1.0:1789760774.726436:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717956560377 time 717956299961 state 0x1 ref 1 00000100:00100000:1.0:1789760774.726467:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717956560377 time 717972085908 state 0x0 ref 1 00000100:00100000:1.0:1789760774.726482:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554722304:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760774.726524:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62619648, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1ddd8 00010000:00010000:1.0:1789760774.726533:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c3e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.726548:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760774.726552:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.726559:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62619648->62623743] (req 62619648->62623743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.726573:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62619648->62623743] (req 62619648->62623743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.736207:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982597 is committed 00002000:00010000:1.0:1789760774.736697:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62619648 -> 62623744 00002000:00010000:1.0:1789760774.736702:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122328 -> 122336 00010000:00010000:1.0:1789760774.736714:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62619648->62623743] (req 62619648->62623743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.736724:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c3e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62619648->62623743] (req 62619648->62623743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.736730:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62619648->62623743] (req 62619648->62623743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.736734:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760774.736750:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62619648->62623743] (req 62619648->62623743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.736756:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c3e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62619648->62623743] (req 62619648->62623743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.736826:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c3e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62619648->62623743] (req 62619648->62623743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.736835:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c3e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62619648->62623743] (req 62619648->62623743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.736983:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554722304:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10497us (10768us total) trans 4294982597 rc 0/0 00000100:00100000:1.0:1789760774.736994:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15299 00000100:00100000:1.0:1789760774.737003:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717956560377 time 717972085908 state 0x2 ref 0 00010000:00010000:0.0:1789760774.737817:0:8453:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760774.745910:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760774.748239:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554722432 00000100:00100000:1.0:1789760774.748253:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554722432 00000100:00100000:1.0:1789760774.748319:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717956560377 time 717972085908 state 0x0 ref 1 00000100:00100000:1.0:1789760774.748334:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717972346324 time 717972085908 state 0x1 ref 1 00000100:00100000:1.0:1789760774.748447:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717972346324 time 717994065740 state 0x0 ref 1 00000100:00100000:1.0:1789760774.748462:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554722432:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760774.748519:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62623744, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1dde0 00010000:00010000:1.0:1789760774.748529:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c45 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.748546:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=717s v=(0 0) 00010000:00010000:1.0:1789760774.748551:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.748557:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62623744->62627839] (req 62623744->62627839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.748571:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62623744->62627839] (req 62623744->62627839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.755650:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982598 is committed 00002000:00010000:1.0:1789760774.755761:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62623744 -> 62627840 00002000:00010000:1.0:1789760774.755766:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122336 -> 122344 00010000:00010000:1.0:1789760774.755945:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62623744->62627839] (req 62623744->62627839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.755961:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c45 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62623744->62627839] (req 62623744->62627839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.755970:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c45 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62623744->62627839] (req 62623744->62627839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.755977:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760774.755982:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c45 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62623744->62627839] (req 62623744->62627839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.755990:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c45 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62623744->62627839] (req 62623744->62627839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.756012:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c45 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62623744->62627839] (req 62623744->62627839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.756018:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c45 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62623744->62627839] (req 62623744->62627839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.756323:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554722432:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7858us (10417us total) trans 4294982598 rc 0/0 00000100:00100000:1.0:1789760774.756333:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15300 00000100:00100000:1.0:1789760774.756342:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717972346324 time 717994065740 state 0x2 ref 0 00000100:00100000:3.0:1789760774.765680:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760774.766021:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554722560 00000100:00100000:2.0:1789760774.766031:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554722560 00000100:00100000:2.0:1789760774.766096:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717972346324 time 717994065740 state 0x0 ref 1 00000100:00100000:2.0:1789760774.766112:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717994326156 time 717994065740 state 0x1 ref 1 00000100:00100000:2.0:1789760774.766150:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717994326156 time 718011768615 state 0x0 ref 1 00000100:00100000:2.0:1789760774.766162:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554722560:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760774.766207:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62627840, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1dde8 00010000:00010000:2.0:1789760774.766218:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4c4c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.766235:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760774.766240:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760774.766246:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4c4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62627840->62631935] (req 62627840->62631935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.766260:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4c4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62627840->62631935] (req 62627840->62631935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760774.772766:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982599 is committed 00002000:00010000:2.0:1789760774.773409:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62627840 -> 62631936 00002000:00010000:2.0:1789760774.773416:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122344 -> 122352 00010000:00010000:2.0:1789760774.773430:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4c4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62627840->62631935] (req 62627840->62631935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.773442:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4c4c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62627840->62631935] (req 62627840->62631935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.773452:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4c4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62627840->62631935] (req 62627840->62631935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.773459:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760774.773464:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4c4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62627840->62631935] (req 62627840->62631935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.773474:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4c4c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62627840->62631935] (req 62627840->62631935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.773504:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4c4c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62627840->62631935] (req 62627840->62631935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760774.773512:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4c4c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62627840->62631935] (req 62627840->62631935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760774.773606:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554722560:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7441us (7932us total) trans 4294982599 rc 0/0 00000100:00100000:2.0:1789760774.773617:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15301 00000100:00100000:2.0:1789760774.773626:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717994326156 time 718011768615 state 0x2 ref 0 00000100:00100000:0.0:1789760774.779010:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760774.780627:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554722688 00000100:00100000:1.0:1789760774.780643:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554722688 00000100:00100000:1.0:1789760774.780703:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 717994326156 time 718011768615 state 0x0 ref 1 00000100:00100000:1.0:1789760774.780719:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718012029031 time 718011768615 state 0x1 ref 1 00000100:00100000:1.0:1789760774.780761:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718012029031 time 718026379025 state 0x0 ref 1 00000100:00100000:1.0:1789760774.781008:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554722688:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760774.781194:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62631936, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1ddf0 00010000:00010000:1.0:1789760774.781204:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c53 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.781220:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760774.781225:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.781231:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62631936->62636031] (req 62631936->62636031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.781244:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62631936->62636031] (req 62631936->62636031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.793650:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982600 is committed 00002000:00010000:1.0:1789760774.801211:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62631936 -> 62636032 00002000:00010000:1.0:1789760774.801221:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122352 -> 122360 00010000:00010000:1.0:1789760774.801235:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62631936->62636031] (req 62631936->62636031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.801249:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c53 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62631936->62636031] (req 62631936->62636031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.801258:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c53 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62631936->62636031] (req 62631936->62636031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.801265:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760774.801270:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c53 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62631936->62636031] (req 62631936->62636031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.801277:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c53 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62631936->62636031] (req 62631936->62636031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.801296:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c53 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62631936->62636031] (req 62631936->62636031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.801309:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c53 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62631936->62636031] (req 62631936->62636031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.802193:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554722688:12345-192.168.203.22@tcp:4:dd.0 Request processed in 21411us (23183us total) trans 4294982600 rc 0/0 00000100:00100000:1.0:1789760774.802206:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15302 00000100:00100000:1.0:1789760774.802215:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718012029031 time 718026379025 state 0x2 ref 0 00000100:00100000:3.0:1789760774.815058:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760774.815134:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554722816 00000100:00100000:1.0:1789760774.815143:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554722816 00000100:00100000:1.0:1789760774.815193:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718012029031 time 718026379025 state 0x0 ref 1 00000100:00100000:1.0:1789760774.815209:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718026639441 time 718026379025 state 0x1 ref 1 00000100:00100000:1.0:1789760774.815240:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718026639441 time 718060858520 state 0x0 ref 1 00000100:00100000:1.0:1789760774.815251:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554722816:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760774.815295:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62636032, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1ddf8 00010000:00010000:1.0:1789760774.815304:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c5a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.815320:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760774.815324:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.815330:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62636032->62640127] (req 62636032->62640127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.815350:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62636032->62640127] (req 62636032->62640127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.829212:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982601 is committed 00002000:00010000:3.0:1789760774.829326:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62636032 -> 62640128 00002000:00010000:3.0:1789760774.829330:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122360 -> 122368 00010000:00010000:3.0:1789760774.829344:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62636032->62640127] (req 62636032->62640127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.829356:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c5a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62636032->62640127] (req 62636032->62640127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.829366:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c5a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62636032->62640127] (req 62636032->62640127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.829373:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760774.829404:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c5a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62636032->62640127] (req 62636032->62640127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.829413:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c5a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62636032->62640127] (req 62636032->62640127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.829433:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c5a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62636032->62640127] (req 62636032->62640127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.829442:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4c5a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62636032->62640127] (req 62636032->62640127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.829540:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554722816:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14285us (14497us total) trans 4294982601 rc 0/0 00000100:00100000:3.0:1789760774.829551:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15303 00000100:00100000:3.0:1789760774.829560:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718026639441 time 718060858520 state 0x2 ref 0 00000100:00100000:1.0:1789760774.837090:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760774.837368:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554722944 00000100:00100000:3.0:1789760774.837409:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554722944 00000100:00100000:3.0:1789760774.837465:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718026639441 time 718060858520 state 0x0 ref 1 00000100:00100000:3.0:1789760774.837480:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718061118936 time 718060858520 state 0x1 ref 1 00000100:00100000:3.0:1789760774.837550:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718061118936 time 718083167959 state 0x0 ref 1 00000100:00100000:3.0:1789760774.837564:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d81880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554722944:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760774.837610:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62640128, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1de00 00010000:00010000:3.0:1789760774.837619:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4c61 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.837635:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760774.837640:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.837646:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4c61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62640128->62644223] (req 62640128->62644223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.837660:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4c61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62640128->62644223] (req 62640128->62644223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.845419:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982602 is committed 00002000:00010000:3.0:1789760774.845683:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62640128 -> 62644224 00002000:00010000:3.0:1789760774.845688:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122368 -> 122376 00010000:00010000:3.0:1789760774.845885:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4c61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62640128->62644223] (req 62640128->62644223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.845899:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4c61 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62640128->62644223] (req 62640128->62644223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.845915:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4c61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62640128->62644223] (req 62640128->62644223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.845922:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760774.845927:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4c61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62640128->62644223] (req 62640128->62644223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.845937:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4c61 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62640128->62644223] (req 62640128->62644223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.845958:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4c61 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62640128->62644223] (req 62640128->62644223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.845966:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4c61 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62640128->62644223] (req 62640128->62644223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.846192:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d81880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554722944:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8625us (9172us total) trans 4294982602 rc 0/0 00000100:00100000:3.0:1789760774.846203:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15304 00000100:00100000:3.0:1789760774.846214:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718061118936 time 718083167959 state 0x2 ref 0 00000100:00100000:0.0:1789760774.850557:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760774.850623:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554723072 00000100:00100000:1.0:1789760774.850634:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554723072 00000100:00100000:1.0:1789760774.850687:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718061118936 time 718083167959 state 0x0 ref 1 00000100:00100000:1.0:1789760774.850702:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718083428375 time 718083167959 state 0x1 ref 1 00000100:00100000:1.0:1789760774.850744:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718083428375 time 718096352435 state 0x0 ref 1 00000100:00100000:1.0:1789760774.850759:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554723072:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760774.850806:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62644224, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1de08 00010000:00010000:1.0:1789760774.850815:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c68 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.850830:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760774.850835:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.850841:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62644224->62648319] (req 62644224->62648319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.850854:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62644224->62648319] (req 62644224->62648319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760774.858782:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982603 is committed 00002000:00010000:1.0:1789760774.859230:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62644224 -> 62648320 00002000:00010000:1.0:1789760774.859235:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122376 -> 122384 00010000:00010000:1.0:1789760774.859249:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62644224->62648319] (req 62644224->62648319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.859262:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c68 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62644224->62648319] (req 62644224->62648319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.859272:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c68 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62644224->62648319] (req 62644224->62648319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.859279:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760774.859283:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c68 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62644224->62648319] (req 62644224->62648319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.859293:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c68 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62644224->62648319] (req 62644224->62648319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.859312:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c68 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62644224->62648319] (req 62644224->62648319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.859321:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c68 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62644224->62648319] (req 62644224->62648319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.859529:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554723072:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8769us (8975us total) trans 4294982603 rc 0/0 00000100:00100000:1.0:1789760774.859540:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15305 00000100:00100000:1.0:1789760774.859549:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718083428375 time 718096352435 state 0x2 ref 0 00000100:00100000:2.0:1789760774.865511:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760774.865581:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554723200 00000100:00100000:0.0:1789760774.865591:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554723200 00000100:00100000:0.0:1789760774.865641:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718083428375 time 718096352435 state 0x0 ref 1 00000100:00100000:0.0:1789760774.865656:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718096612851 time 718096352435 state 0x1 ref 1 00000100:00100000:0.0:1789760774.865685:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718096612851 time 718111303668 state 0x0 ref 1 00000100:00100000:0.0:1789760774.865705:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554723200:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760774.865759:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62648320, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1de10 00010000:00010000:0.0:1789760774.865769:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4c6f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.865789:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760774.865793:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.865800:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4c6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62648320->62652415] (req 62648320->62652415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.865813:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4c6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62648320->62652415] (req 62648320->62652415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.880785:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982604 is committed 00002000:00010000:0.0:1789760774.881034:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62648320 -> 62652416 00002000:00010000:0.0:1789760774.881040:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122384 -> 122392 00010000:00010000:0.0:1789760774.881053:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4c6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62648320->62652415] (req 62648320->62652415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.881065:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4c6f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62648320->62652415] (req 62648320->62652415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.881074:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4c6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62648320->62652415] (req 62648320->62652415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.881082:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760774.881087:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4c6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62648320->62652415] (req 62648320->62652415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.881096:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4c6f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62648320->62652415] (req 62648320->62652415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.881131:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4c6f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62648320->62652415] (req 62648320->62652415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.881139:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4c6f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62648320->62652415] (req 62648320->62652415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.881233:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554723200:12345-192.168.203.22@tcp:4:dd.0 Request processed in 15533us (15726us total) trans 4294982604 rc 0/0 00000100:00100000:0.0:1789760774.881247:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15306 00000100:00100000:0.0:1789760774.881255:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718096612851 time 718111303668 state 0x2 ref 0 00000100:00100000:3.0:1789760774.886355:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760774.886488:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554723328 00000100:00100000:0.0:1789760774.886509:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554723328 00000100:00100000:0.0:1789760774.886566:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718096612851 time 718111303668 state 0x0 ref 1 00000100:00100000:0.0:1789760774.886580:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718111564084 time 718111303668 state 0x1 ref 1 00000100:00100000:0.0:1789760774.886602:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718111564084 time 718132220427 state 0x0 ref 1 00000100:00100000:0.0:1789760774.886613:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554723328:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760774.886660:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62652416, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1de18 00010000:00010000:0.0:1789760774.886669:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4c76 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.886684:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760774.886690:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.886696:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4c76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62652416->62656511] (req 62652416->62656511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.886710:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4c76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62652416->62656511] (req 62652416->62656511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.894615:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982605 is committed 00002000:00010000:3.0:1789760774.894745:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62652416 -> 62656512 00002000:00010000:3.0:1789760774.894751:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122392 -> 122400 00010000:00010000:3.0:1789760774.894764:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4c76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62652416->62656511] (req 62652416->62656511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.894777:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4c76 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62652416->62656511] (req 62652416->62656511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.894787:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4c76 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62652416->62656511] (req 62652416->62656511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.894804:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760774.894809:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4c76 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62652416->62656511] (req 62652416->62656511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.894819:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4c76 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62652416->62656511] (req 62652416->62656511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.894838:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4c76 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62652416->62656511] (req 62652416->62656511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.894846:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4c76 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62652416->62656511] (req 62652416->62656511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.894941:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554723328:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8325us (8591us total) trans 4294982605 rc 0/0 00000100:00100000:3.0:1789760774.894951:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15307 00000100:00100000:3.0:1789760774.894961:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718111564084 time 718132220427 state 0x2 ref 0 00000100:00100000:2.0:1789760774.902282:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760774.902412:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554723456 00000100:00100000:3.0:1789760774.902421:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554723456 00000100:00100000:3.0:1789760774.902474:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718111564084 time 718132220427 state 0x0 ref 1 00000100:00100000:3.0:1789760774.902489:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718132480843 time 718132220427 state 0x1 ref 1 00000100:00100000:3.0:1789760774.902512:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718132480843 time 718148131028 state 0x0 ref 1 00000100:00100000:3.0:1789760774.902523:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554723456:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760774.902568:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62656512, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1de20 00010000:00010000:3.0:1789760774.902576:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4c7d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.902591:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760774.902595:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.902601:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4c7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62656512->62660607] (req 62656512->62660607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.902615:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4c7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62656512->62660607] (req 62656512->62660607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.910767:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982606 is committed 00002000:00010000:1.0:1789760774.913211:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62656512 -> 62660608 00002000:00010000:1.0:1789760774.913221:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122400 -> 122408 00010000:00010000:1.0:1789760774.913232:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4c7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62656512->62660607] (req 62656512->62660607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.913244:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4c7d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62656512->62660607] (req 62656512->62660607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.913253:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4c7d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62656512->62660607] (req 62656512->62660607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.913260:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760774.913266:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4c7d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62656512->62660607] (req 62656512->62660607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.913275:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4c7d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62656512->62660607] (req 62656512->62660607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.913293:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4c7d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62656512->62660607] (req 62656512->62660607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.913301:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c4c7d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62656512->62660607] (req 62656512->62660607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.913515:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554723456:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10988us (11236us total) trans 4294982606 rc 0/0 00000100:00100000:1.0:1789760774.913526:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15308 00000100:00100000:1.0:1789760774.913535:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718132480843 time 718148131028 state 0x2 ref 0 00000100:00100000:2.0:1789760774.921342:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760774.922278:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554723584 00000100:00100000:1.0:1789760774.922290:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554723584 00000100:00100000:1.0:1789760774.922343:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718132480843 time 718148131028 state 0x0 ref 1 00000100:00100000:1.0:1789760774.922357:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718148391444 time 718148131028 state 0x1 ref 1 00000100:00100000:3.0:1789760774.922483:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718148391444 time 718168098920 state 0x0 ref 1 00000100:00100000:3.0:1789760774.922500:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554723584:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760774.922548:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62660608, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1de28 00010000:00010000:3.0:1789760774.922557:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4c84 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.922577:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760774.922582:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760774.922588:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4c84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62660608->62664703] (req 62660608->62664703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.922603:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4c84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62660608->62664703] (req 62660608->62664703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.934779:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982607 is committed 00002000:00010000:3.0:1789760774.934927:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62660608 -> 62664704 00002000:00010000:3.0:1789760774.934932:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122408 -> 122416 00010000:00010000:3.0:1789760774.934944:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4c84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62660608->62664703] (req 62660608->62664703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.934955:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4c84 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62660608->62664703] (req 62660608->62664703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.934964:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4c84 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62660608->62664703] (req 62660608->62664703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.934972:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760774.934977:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4c84 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62660608->62664703] (req 62660608->62664703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.934986:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4c84 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62660608->62664703] (req 62660608->62664703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.935010:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4c84 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62660608->62664703] (req 62660608->62664703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760774.935017:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4c84 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62660608->62664703] (req 62660608->62664703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760774.935103:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554723584:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12604us (13776us total) trans 4294982607 rc 0/0 00000100:00100000:3.0:1789760774.935113:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15309 00000100:00100000:3.0:1789760774.935122:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718148391444 time 718168098920 state 0x2 ref 0 00000100:00100000:3.0:1789760774.938451:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760774.938763:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554723712 00000100:00100000:1.0:1789760774.938772:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554723712 00000100:00100000:1.0:1789760774.938838:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718148391444 time 718168098920 state 0x0 ref 1 00000100:00100000:1.0:1789760774.938854:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718168359336 time 718168098920 state 0x1 ref 1 00000100:00100000:1.0:1789760774.938888:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718168359336 time 718184506426 state 0x0 ref 1 00000100:00100000:1.0:1789760774.938901:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554723712:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760774.938950:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62664704, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1de30 00010000:00010000:1.0:1789760774.938959:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c8b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.938973:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760774.938978:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.938985:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62664704->62668799] (req 62664704->62668799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.938999:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62664704->62668799] (req 62664704->62668799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.950426:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982608 is committed 00002000:00010000:0.0:1789760774.958360:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62664704 -> 62668800 00002000:00010000:0.0:1789760774.958371:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122416 -> 122424 00010000:00010000:0.0:1789760774.958426:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62664704->62668799] (req 62664704->62668799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.958440:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c8b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62664704->62668799] (req 62664704->62668799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.958448:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62664704->62668799] (req 62664704->62668799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.958456:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760774.958461:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62664704->62668799] (req 62664704->62668799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.958470:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c8b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62664704->62668799] (req 62664704->62668799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.958491:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c8b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62664704->62668799] (req 62664704->62668799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.958504:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c4c8b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62664704->62668799] (req 62664704->62668799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760774.958606:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554723712:12345-192.168.203.22@tcp:4:dd.0 Request processed in 19702us (20159us total) trans 4294982608 rc 0/0 00000100:00100000:0.0:1789760774.958616:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15310 00000100:00100000:0.0:1789760774.958633:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718168359336 time 718184506426 state 0x2 ref 0 00000100:00100000:2.0:1789760774.964279:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760774.964414:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554723840 00000100:00100000:1.0:1789760774.964424:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554723840 00000100:00100000:1.0:1789760774.964478:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718168359336 time 718184506426 state 0x0 ref 1 00000100:00100000:1.0:1789760774.964607:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718184766842 time 718184506426 state 0x1 ref 1 00000100:00100000:1.0:1789760774.964639:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718184766842 time 718210257681 state 0x0 ref 1 00000100:00100000:1.0:1789760774.964651:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554723840:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760774.964701:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62668800, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1de38 00010000:00010000:1.0:1789760774.964710:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4c92 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.964727:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760774.964731:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.964747:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4c92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62668800->62672895] (req 62668800->62672895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.964761:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4c92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62668800->62672895] (req 62668800->62672895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.974002:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982609 is committed 00002000:00010000:1.0:1789760774.974133:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62668800 -> 62672896 00002000:00010000:1.0:1789760774.974138:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122424 -> 122432 00010000:00010000:1.0:1789760774.974151:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4c92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62668800->62672895] (req 62668800->62672895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.974163:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4c92 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62668800->62672895] (req 62668800->62672895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.974172:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4c92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62668800->62672895] (req 62668800->62672895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.974179:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760774.974184:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4c92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62668800->62672895] (req 62668800->62672895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.974195:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4c92 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62668800->62672895] (req 62668800->62672895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.974213:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4c92 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62668800->62672895] (req 62668800->62672895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.974221:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4c92 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62668800->62672895] (req 62668800->62672895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.974360:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554723840:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9705us (10087us total) trans 4294982609 rc 0/0 00000100:00100000:1.0:1789760774.974372:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15311 00000100:00100000:1.0:1789760774.974435:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718184766842 time 718210257681 state 0x2 ref 0 00000100:00100000:2.0:1789760774.979269:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760774.979350:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554723968 00000100:00100000:1.0:1789760774.979360:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554723968 00000100:00100000:1.0:1789760774.979530:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718184766842 time 718210257681 state 0x0 ref 1 00000100:00100000:1.0:1789760774.979545:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718210518097 time 718210257681 state 0x1 ref 1 00000100:00100000:1.0:1789760774.979568:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718210518097 time 718225186680 state 0x0 ref 1 00000100:00100000:1.0:1789760774.979580:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554723968:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760774.979628:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62672896, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1de40 00010000:00010000:1.0:1789760774.979636:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4c99 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.979651:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760774.979656:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760774.979662:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4c99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62672896->62676991] (req 62672896->62676991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.979675:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4c99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62672896->62676991] (req 62672896->62676991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760774.987048:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982610 is committed 00002000:00010000:1.0:1789760774.988146:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62672896 -> 62676992 00002000:00010000:1.0:1789760774.988153:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122432 -> 122440 00010000:00010000:1.0:1789760774.988167:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4c99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62672896->62676991] (req 62672896->62676991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.988179:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4c99 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62672896->62676991] (req 62672896->62676991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.988187:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4c99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62672896->62676991] (req 62672896->62676991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.988194:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760774.988198:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4c99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62672896->62676991] (req 62672896->62676991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.988207:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4c99 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62672896->62676991] (req 62672896->62676991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.988225:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4c99 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62672896->62676991] (req 62672896->62676991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760774.988232:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4c99 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62672896->62676991] (req 62672896->62676991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760774.988324:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554723968:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8741us (9059us total) trans 4294982610 rc 0/0 00000100:00100000:1.0:1789760774.988334:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15312 00000100:00100000:1.0:1789760774.988343:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718210518097 time 718225186680 state 0x2 ref 0 00000100:00100000:3.0:1789760774.994925:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760774.995581:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554724096 00000100:00100000:0.0:1789760774.995591:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554724096 00000100:00100000:0.0:1789760774.995647:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718210518097 time 718225186680 state 0x0 ref 1 00000100:00100000:0.0:1789760774.995663:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718225447096 time 718225186680 state 0x1 ref 1 00000100:00100000:0.0:1789760774.995698:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718225447096 time 718241316615 state 0x0 ref 1 00000100:00100000:0.0:1789760774.995710:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554724096:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760774.995766:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62676992, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1de48 00010000:00010000:0.0:1789760774.995775:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4ca0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.996212:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760774.996219:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760774.996226:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4ca0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62676992->62681087] (req 62676992->62681087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760774.996246:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4ca0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62676992->62681087] (req 62676992->62681087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.003513:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982611 is committed 00002000:00010000:0.0:1789760775.004534:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62676992 -> 62681088 00002000:00010000:0.0:1789760775.004541:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122440 -> 122448 00010000:00010000:0.0:1789760775.004555:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4ca0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62676992->62681087] (req 62676992->62681087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.004576:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4ca0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62676992->62681087] (req 62676992->62681087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.004585:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4ca0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62676992->62681087] (req 62676992->62681087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.004593:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760775.004598:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4ca0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62676992->62681087] (req 62676992->62681087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.004607:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4ca0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62676992->62681087] (req 62676992->62681087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.004626:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4ca0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62676992->62681087] (req 62676992->62681087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.004634:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4ca0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62676992->62681087] (req 62676992->62681087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.004727:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554724096:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9014us (9807us total) trans 4294982611 rc 0/0 00000100:00100000:0.0:1789760775.004750:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15313 00000100:00100000:0.0:1789760775.004759:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718225447096 time 718241316615 state 0x2 ref 0 00000100:00100000:2.0:1789760775.016821:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760775.016904:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554724224 00000100:00100000:0.0:1789760775.016914:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554724224 00000100:00100000:0.0:1789760775.016990:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718225447096 time 718241316615 state 0x0 ref 1 00000100:00100000:0.0:1789760775.017005:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718241577031 time 718241316615 state 0x1 ref 1 00000100:00100000:0.0:1789760775.017039:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718241577031 time 718262657722 state 0x0 ref 1 00000100:00100000:0.0:1789760775.017057:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554724224:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760775.017102:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62681088, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1de50 00010000:00010000:0.0:1789760775.017111:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4ca7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.017127:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760775.017131:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.017136:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4ca7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62681088->62685183] (req 62681088->62685183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.017150:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4ca7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62681088->62685183] (req 62681088->62685183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760775.017587:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789760775.023158:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982612 is committed 00002000:00010000:0.0:1789760775.023353:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62681088 -> 62685184 00002000:00010000:0.0:1789760775.023359:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122448 -> 122456 00010000:00010000:0.0:1789760775.023372:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4ca7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62681088->62685183] (req 62681088->62685183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.023423:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4ca7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62681088->62685183] (req 62681088->62685183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.023431:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4ca7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62681088->62685183] (req 62681088->62685183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.023437:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760775.023442:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4ca7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62681088->62685183] (req 62681088->62685183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.023451:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4ca7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62681088->62685183] (req 62681088->62685183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.023471:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4ca7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62681088->62685183] (req 62681088->62685183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.023479:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4ca7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62681088->62685183] (req 62681088->62685183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.023570:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554724224:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6512us (6754us total) trans 4294982612 rc 0/0 00000100:00100000:0.0:1789760775.023583:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15314 00000100:00100000:0.0:1789760775.023591:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718241577031 time 718262657722 state 0x2 ref 0 00000100:00100000:2.0:1789760775.027020:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760775.027126:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554724352 00000100:00100000:1.0:1789760775.027136:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554724352 00000100:00100000:1.0:1789760775.027198:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718241577031 time 718262657722 state 0x0 ref 1 00000100:00100000:1.0:1789760775.027213:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718262918138 time 718262657722 state 0x1 ref 1 00000100:00100000:1.0:1789760775.027244:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718262918138 time 718272862025 state 0x0 ref 1 00000100:00100000:1.0:1789760775.027256:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554724352:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760775.027305:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62685184, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1de58 00010000:00010000:1.0:1789760775.027315:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4cae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.027330:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760775.027334:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.027340:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4cae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62685184->62689279] (req 62685184->62689279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.027352:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4cae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62685184->62689279] (req 62685184->62689279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.033422:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982613 is committed 00002000:00010000:1.0:1789760775.033597:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62685184 -> 62689280 00002000:00010000:1.0:1789760775.033602:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122456 -> 122464 00010000:00010000:1.0:1789760775.033615:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4cae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62685184->62689279] (req 62685184->62689279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.033633:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4cae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62685184->62689279] (req 62685184->62689279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.033642:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4cae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62685184->62689279] (req 62685184->62689279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.033649:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760775.033654:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4cae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62685184->62689279] (req 62685184->62689279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.033664:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4cae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62685184->62689279] (req 62685184->62689279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.033685:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4cae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62685184->62689279] (req 62685184->62689279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.033691:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c4cae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62685184->62689279] (req 62685184->62689279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.033916:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554724352:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6656us (6899us total) trans 4294982613 rc 0/0 00000100:00100000:1.0:1789760775.033927:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15315 00000100:00100000:1.0:1789760775.033936:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718262918138 time 718272862025 state 0x2 ref 0 00000100:00100000:3.0:1789760775.043076:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760775.043799:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554724480 00000100:00100000:2.0:1789760775.043811:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554724480 00000100:00100000:2.0:1789760775.043867:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718262918138 time 718272862025 state 0x0 ref 1 00000100:00100000:2.0:1789760775.043883:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718273122441 time 718272862025 state 0x1 ref 1 00000100:00100000:2.0:1789760775.043948:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718273122441 time 718289566825 state 0x0 ref 1 00000100:00100000:2.0:1789760775.043961:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554724480:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760775.044009:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62689280, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1de60 00010000:00010000:2.0:1789760775.044019:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4cb5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.044035:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760775.044040:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.044046:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4cb5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62689280->62693375] (req 62689280->62693375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.044059:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4cb5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62689280->62693375] (req 62689280->62693375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.052645:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982614 is committed 00002000:00010000:1.0:1789760775.052811:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62689280 -> 62693376 00002000:00010000:1.0:1789760775.052818:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122464 -> 122472 00010000:00010000:1.0:1789760775.052831:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4cb5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62689280->62693375] (req 62689280->62693375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.052844:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4cb5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62689280->62693375] (req 62689280->62693375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.052852:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4cb5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62689280->62693375] (req 62689280->62693375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.052859:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760775.052864:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4cb5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62689280->62693375] (req 62689280->62693375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.052872:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4cb5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62689280->62693375] (req 62689280->62693375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.052892:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4cb5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62689280->62693375] (req 62689280->62693375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.052898:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4cb5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62689280->62693375] (req 62689280->62693375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.052996:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554724480:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9032us (9924us total) trans 4294982614 rc 0/0 00000100:00100000:1.0:1789760775.053006:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15316 00000100:00100000:1.0:1789760775.053016:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718273122441 time 718289566825 state 0x2 ref 0 00000100:00100000:2.0:1789760775.059534:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760775.059629:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554724608 00000100:00100000:0.0:1789760775.059639:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554724608 00000100:00100000:0.0:1789760775.059696:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718273122441 time 718289566825 state 0x0 ref 1 00000100:00100000:0.0:1789760775.059711:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718289827241 time 718289566825 state 0x1 ref 1 00000100:00100000:0.0:1789760775.059754:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718289827241 time 718305372759 state 0x0 ref 1 00000100:00100000:0.0:1789760775.059774:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e54e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554724608:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760775.059822:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62693376, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1de68 00010000:00010000:0.0:1789760775.059832:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4cbc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.059846:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760775.059850:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.059857:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4cbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62693376->62697471] (req 62693376->62697471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.059870:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4cbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62693376->62697471] (req 62693376->62697471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.065892:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982615 is committed 00002000:00010000:3.0:1789760775.066687:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62693376 -> 62697472 00002000:00010000:3.0:1789760775.066692:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122472 -> 122480 00010000:00010000:3.0:1789760775.066706:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4cbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62693376->62697471] (req 62693376->62697471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.066718:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4cbc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62693376->62697471] (req 62693376->62697471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.066726:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4cbc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62693376->62697471] (req 62693376->62697471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.066748:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760775.066752:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4cbc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62693376->62697471] (req 62693376->62697471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.066760:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4cbc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62693376->62697471] (req 62693376->62697471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.066779:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4cbc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62693376->62697471] (req 62693376->62697471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.066787:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c4cbc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62693376->62697471] (req 62693376->62697471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.066923:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554724608:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7154us (7394us total) trans 4294982615 rc 0/0 00000100:00100000:3.0:1789760775.066934:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15317 00000100:00100000:3.0:1789760775.066944:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718289827241 time 718305372759 state 0x2 ref 0 00000100:00100000:2.0:1789760775.071747:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760775.071875:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554724736 00000100:00100000:0.0:1789760775.071885:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554724736 00000100:00100000:0.0:1789760775.071936:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718289827241 time 718305372759 state 0x0 ref 1 00000100:00100000:0.0:1789760775.071952:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718305633175 time 718305372759 state 0x1 ref 1 00000100:00100000:0.0:1789760775.071987:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718305633175 time 718317605465 state 0x0 ref 1 00000100:00100000:0.0:1789760775.072000:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554724736:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760775.072047:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62697472, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1de70 00010000:00010000:0.0:1789760775.072057:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4cc3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.072073:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760775.072077:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.072084:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4cc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62697472->62701567] (req 62697472->62701567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.072098:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4cc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62697472->62701567] (req 62697472->62701567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.078667:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982616 is committed 00002000:00010000:2.0:1789760775.078894:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62697472 -> 62701568 00002000:00010000:2.0:1789760775.078899:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122480 -> 122488 00010000:00010000:2.0:1789760775.078913:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4cc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62697472->62701567] (req 62697472->62701567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.078926:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4cc3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62697472->62701567] (req 62697472->62701567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.078936:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4cc3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62697472->62701567] (req 62697472->62701567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.078943:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760775.078949:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4cc3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62697472->62701567] (req 62697472->62701567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.078958:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4cc3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62697472->62701567] (req 62697472->62701567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.078978:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4cc3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62697472->62701567] (req 62697472->62701567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.078985:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c4cc3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62697472->62701567] (req 62697472->62701567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.079175:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554724736:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7113us (7382us total) trans 4294982616 rc 0/0 00000100:00100000:2.0:1789760775.079187:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15318 00000100:00100000:2.0:1789760775.079196:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718305633175 time 718317605465 state 0x2 ref 0 00000100:00100000:1.0:1789760775.087658:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760775.087830:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554724864 00000100:00100000:2.0:1789760775.087840:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554724864 00000100:00100000:2.0:1789760775.087910:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718305633175 time 718317605465 state 0x0 ref 1 00000100:00100000:2.0:1789760775.087925:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718317865881 time 718317605465 state 0x1 ref 1 00000100:00100000:2.0:1789760775.087959:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718317865881 time 718333578040 state 0x0 ref 1 00000100:00100000:2.0:1789760775.087973:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554724864:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760775.088026:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62701568, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1de78 00010000:00010000:2.0:1789760775.088037:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4cca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.088057:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760775.088063:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.088071:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4cca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62701568->62705663] (req 62701568->62705663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.088089:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4cca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62701568->62705663] (req 62701568->62705663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.094246:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982617 is committed 00002000:00010000:2.0:1789760775.094434:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62701568 -> 62705664 00002000:00010000:2.0:1789760775.094439:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122488 -> 122496 00010000:00010000:2.0:1789760775.094452:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4cca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62701568->62705663] (req 62701568->62705663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.095185:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4cca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62701568->62705663] (req 62701568->62705663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.095199:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4cca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62701568->62705663] (req 62701568->62705663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.095217:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760775.095222:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4cca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62701568->62705663] (req 62701568->62705663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.095232:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4cca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62701568->62705663] (req 62701568->62705663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.095253:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4cca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62701568->62705663] (req 62701568->62705663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.095260:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4cca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62701568->62705663] (req 62701568->62705663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.095477:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554724864:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7497us (7821us total) trans 4294982617 rc 0/0 00000100:00100000:2.0:1789760775.095488:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15319 00000100:00100000:2.0:1789760775.095497:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718317865881 time 718333578040 state 0x2 ref 0 00000100:00100000:0.0:1789760775.101494:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760775.101605:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554724992 00000100:00100000:1.0:1789760775.101615:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554724992 00000100:00100000:1.0:1789760775.101666:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718317865881 time 718333578040 state 0x0 ref 1 00000100:00100000:1.0:1789760775.101683:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718333838456 time 718333578040 state 0x1 ref 1 00000100:00100000:1.0:1789760775.101724:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718333838456 time 718347342387 state 0x0 ref 1 00000100:00100000:1.0:1789760775.101746:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554724992:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760775.101791:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62705664, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1de80 00010000:00010000:1.0:1789760775.101800:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4cd1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.101815:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760775.101819:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.101826:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4cd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62705664->62709759] (req 62705664->62709759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.101838:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4cd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62705664->62709759] (req 62705664->62709759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.105727:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982618 is committed 00002000:00010000:1.0:1789760775.105857:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62705664 -> 62709760 00002000:00010000:1.0:1789760775.105862:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122496 -> 122504 00010000:00010000:1.0:1789760775.105876:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4cd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62705664->62709759] (req 62705664->62709759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.105889:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4cd1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62705664->62709759] (req 62705664->62709759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.105899:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4cd1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62705664->62709759] (req 62705664->62709759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.105906:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760775.105912:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4cd1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62705664->62709759] (req 62705664->62709759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.105923:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4cd1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62705664->62709759] (req 62705664->62709759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.105943:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4cd1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62705664->62709759] (req 62705664->62709759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.105952:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4cd1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62705664->62709759] (req 62705664->62709759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.106099:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554724992:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4358us (4612us total) trans 4294982618 rc 0/0 00000100:00100000:1.0:1789760775.106117:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15320 00000100:00100000:1.0:1789760775.106127:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718333838456 time 718347342387 state 0x2 ref 0 00000100:00100000:3.0:1789760775.112472:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760775.112526:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554725120 00000100:00100000:0.0:1789760775.112536:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554725120 00000100:00100000:0.0:1789760775.112590:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718333838456 time 718347342387 state 0x0 ref 1 00000100:00100000:0.0:1789760775.112606:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718347602803 time 718347342387 state 0x1 ref 1 00000100:00100000:0.0:1789760775.112644:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718347602803 time 718358262259 state 0x0 ref 1 00000100:00100000:0.0:1789760775.112656:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554725120:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760775.112703:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62709760, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1de88 00010000:00010000:0.0:1789760775.112712:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cd8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.112727:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760775.112740:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.112747:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62709760->62713855] (req 62709760->62713855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.112760:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62709760->62713855] (req 62709760->62713855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.117444:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982619 is committed 00002000:00010000:0.0:1789760775.117567:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62709760 -> 62713856 00002000:00010000:0.0:1789760775.117572:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122504 -> 122512 00010000:00010000:0.0:1789760775.117586:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62709760->62713855] (req 62709760->62713855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.117599:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cd8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62709760->62713855] (req 62709760->62713855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.117608:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cd8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62709760->62713855] (req 62709760->62713855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.117621:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760775.117627:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cd8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62709760->62713855] (req 62709760->62713855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.117636:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cd8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62709760->62713855] (req 62709760->62713855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.117656:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cd8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62709760->62713855] (req 62709760->62713855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.117664:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cd8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62709760->62713855] (req 62709760->62713855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.117782:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554725120:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5123us (5317us total) trans 4294982619 rc 0/0 00000100:00100000:0.0:1789760775.117792:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15321 00000100:00100000:0.0:1789760775.117802:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718347602803 time 718358262259 state 0x2 ref 0 00000100:00100000:3.0:1789760775.124015:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760775.124845:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554725248 00000100:00100000:2.0:1789760775.124856:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554725248 00000100:00100000:2.0:1789760775.124929:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718347602803 time 718358262259 state 0x0 ref 1 00000100:00100000:2.0:1789760775.124946:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718358522675 time 718358262259 state 0x1 ref 1 00000100:00100000:2.0:1789760775.124982:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718358522675 time 718370601044 state 0x0 ref 1 00000100:00100000:2.0:1789760775.124995:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554725248:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760775.125044:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62713856, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1de90 00010000:00010000:2.0:1789760775.125054:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4cdf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.125077:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760775.125082:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.125088:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4cdf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62713856->62717951] (req 62713856->62717951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.125102:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4cdf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62713856->62717951] (req 62713856->62717951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.129988:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982620 is committed 00002000:00010000:2.0:1789760775.131693:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62713856 -> 62717952 00002000:00010000:2.0:1789760775.131702:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122512 -> 122520 00010000:00010000:2.0:1789760775.131714:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4cdf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62713856->62717951] (req 62713856->62717951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.131729:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4cdf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62713856->62717951] (req 62713856->62717951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.131747:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4cdf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62713856->62717951] (req 62713856->62717951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.131755:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760775.131759:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4cdf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62713856->62717951] (req 62713856->62717951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.131768:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4cdf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62713856->62717951] (req 62713856->62717951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.131795:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4cdf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62713856->62717951] (req 62713856->62717951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.131803:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4cdf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62713856->62717951] (req 62713856->62717951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.131910:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554725248:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6913us (7901us total) trans 4294982620 rc 0/0 00000100:00100000:2.0:1789760775.131921:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15322 00000100:00100000:2.0:1789760775.131939:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718358522675 time 718370601044 state 0x2 ref 0 00000100:00100000:2.0:1789760775.148324:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760775.149323:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554725376 00000100:00100000:1.0:1789760775.149333:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554725376 00000100:00100000:1.0:1789760775.149434:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718358522675 time 718370601044 state 0x0 ref 1 00000100:00100000:1.0:1789760775.149449:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718370861460 time 718370601044 state 0x1 ref 1 00000100:00100000:1.0:1789760775.149483:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718370861460 time 718395102009 state 0x0 ref 1 00000100:00100000:1.0:1789760775.149497:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e56d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554725376:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760775.149543:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62717952, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1de98 00010000:00010000:1.0:1789760775.149553:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ce6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.149569:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760775.149574:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.149580:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ce6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62717952->62722047] (req 62717952->62722047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.149594:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ce6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62717952->62722047] (req 62717952->62722047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.158001:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982621 is committed 00002000:00010000:1.0:1789760775.158102:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62717952 -> 62722048 00002000:00010000:1.0:1789760775.158105:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122520 -> 122528 00010000:00010000:1.0:1789760775.158172:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ce6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62717952->62722047] (req 62717952->62722047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.158189:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ce6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62717952->62722047] (req 62717952->62722047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.158199:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ce6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62717952->62722047] (req 62717952->62722047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.158206:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760775.158211:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ce6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62717952->62722047] (req 62717952->62722047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.158220:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ce6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62717952->62722047] (req 62717952->62722047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.158239:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ce6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62717952->62722047] (req 62717952->62722047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.158247:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ce6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62717952->62722047] (req 62717952->62722047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.158369:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e56d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554725376:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8870us (10049us total) trans 4294982621 rc 0/0 00000100:00100000:1.0:1789760775.158504:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15323 00000100:00100000:1.0:1789760775.158514:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718370861460 time 718395102009 state 0x2 ref 0 00000100:00100000:3.0:1789760775.163262:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760775.163363:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554725504 00000100:00100000:1.0:1789760775.163373:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554725504 00000100:00100000:1.0:1789760775.163779:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718370861460 time 718395102009 state 0x0 ref 1 00000100:00100000:1.0:1789760775.163796:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718395362425 time 718395102009 state 0x1 ref 1 00000100:00100000:1.0:1789760775.163825:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718395362425 time 718409444507 state 0x0 ref 1 00000100:00100000:1.0:1789760775.163838:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554725504:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760775.163886:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62722048, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1dea0 00010000:00010000:1.0:1789760775.163895:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ced lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.163909:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760775.163913:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.163919:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ced lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62722048->62726143] (req 62722048->62726143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.163932:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ced lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62722048->62726143] (req 62722048->62726143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.167945:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982622 is committed 00002000:00010000:1.0:1789760775.168027:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62722048 -> 62726144 00002000:00010000:1.0:1789760775.168030:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122528 -> 122536 00010000:00010000:1.0:1789760775.168041:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ced lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62722048->62726143] (req 62722048->62726143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.168050:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ced lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62722048->62726143] (req 62722048->62726143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.168056:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ced lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62722048->62726143] (req 62722048->62726143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.168065:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760775.168069:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ced lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62722048->62726143] (req 62722048->62726143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.168075:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ced lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62722048->62726143] (req 62722048->62726143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.168088:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ced lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62722048->62726143] (req 62722048->62726143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.168093:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ced lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62722048->62726143] (req 62722048->62726143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.168747:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554725504:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4898us (5481us total) trans 4294982622 rc 0/0 00000100:00100000:1.0:1789760775.168759:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15324 00000100:00100000:1.0:1789760775.168767:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718395362425 time 718409444507 state 0x2 ref 0 00000100:00100000:1.0:1789760775.173855:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760775.174017:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554725632 00000100:00100000:0.0:1789760775.174027:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554725632 00000100:00100000:0.0:1789760775.174082:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718395362425 time 718409444507 state 0x0 ref 1 00000100:00100000:0.0:1789760775.174097:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718409704923 time 718409444507 state 0x1 ref 1 00000100:00100000:0.0:1789760775.174162:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718409704923 time 718419780331 state 0x0 ref 1 00000100:00100000:0.0:1789760775.174178:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554725632:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760775.174349:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62726144, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1dea8 00010000:00010000:0.0:1789760775.174360:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cf4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.174408:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760775.174414:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.174420:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cf4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62726144->62730239] (req 62726144->62730239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.174434:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cf4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62726144->62730239] (req 62726144->62730239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.179798:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982623 is committed 00002000:00010000:0.0:1789760775.180256:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62726144 -> 62730240 00002000:00010000:0.0:1789760775.180262:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122536 -> 122544 00010000:00010000:0.0:1789760775.180274:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cf4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62726144->62730239] (req 62726144->62730239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.180287:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cf4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62726144->62730239] (req 62726144->62730239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.180295:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cf4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62726144->62730239] (req 62726144->62730239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.180301:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760775.180307:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cf4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62726144->62730239] (req 62726144->62730239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.180315:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cf4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62726144->62730239] (req 62726144->62730239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.180334:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cf4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62726144->62730239] (req 62726144->62730239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.180341:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cf4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62726144->62730239] (req 62726144->62730239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.180489:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554725632:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6307us (6638us total) trans 4294982623 rc 0/0 00000100:00100000:0.0:1789760775.180500:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15325 00000100:00100000:0.0:1789760775.180509:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718409704923 time 718419780331 state 0x2 ref 0 00000100:00100000:2.0:1789760775.188134:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760775.188899:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554725760 00000100:00100000:0.0:1789760775.188912:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554725760 00000100:00100000:0.0:1789760775.188967:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718409704923 time 718419780331 state 0x0 ref 1 00000100:00100000:0.0:1789760775.188983:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718420040747 time 718419780331 state 0x1 ref 1 00000100:00100000:0.0:1789760775.189015:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718420040747 time 718434633823 state 0x0 ref 1 00000100:00100000:0.0:1789760775.189033:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e54000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554725760:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760775.189079:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62730240, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1deb0 00010000:00010000:0.0:1789760775.189088:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cfb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.189106:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760775.189110:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.189116:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cfb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62730240->62734335] (req 62730240->62734335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.189130:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cfb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62730240->62734335] (req 62730240->62734335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.196997:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982624 is committed 00002000:00010000:0.0:1789760775.197200:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62730240 -> 62734336 00002000:00010000:0.0:1789760775.197205:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122544 -> 122552 00010000:00010000:0.0:1789760775.197219:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cfb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62730240->62734335] (req 62730240->62734335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.197231:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cfb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62730240->62734335] (req 62730240->62734335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.197240:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cfb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62730240->62734335] (req 62730240->62734335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.197248:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760775.197253:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cfb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62730240->62734335] (req 62730240->62734335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.197262:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cfb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62730240->62734335] (req 62730240->62734335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.197282:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cfb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62730240->62734335] (req 62730240->62734335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.197290:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4cfb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62730240->62734335] (req 62730240->62734335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.197514:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554725760:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8482us (9383us total) trans 4294982624 rc 0/0 00000100:00100000:0.0:1789760775.197525:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15326 00000100:00100000:0.0:1789760775.197533:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718420040747 time 718434633823 state 0x2 ref 0 00000100:00100000:3.0:1789760775.203275:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760775.203458:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554725888 00000100:00100000:1.0:1789760775.203468:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554725888 00000100:00100000:1.0:1789760775.203524:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718420040747 time 718434633823 state 0x0 ref 1 00000100:00100000:1.0:1789760775.203540:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718434894239 time 718434633823 state 0x1 ref 1 00000100:00100000:1.0:1789760775.203576:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718434894239 time 718449193868 state 0x0 ref 1 00000100:00100000:1.0:1789760775.203590:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554725888:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760775.203927:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62734336, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1deb8 00010000:00010000:1.0:1789760775.203937:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4d02 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.203954:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760775.203959:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.203965:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4d02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62734336->62738431] (req 62734336->62738431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.203979:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4d02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62734336->62738431] (req 62734336->62738431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.210579:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982625 is committed 00002000:00010000:1.0:1789760775.210709:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62734336 -> 62738432 00002000:00010000:1.0:1789760775.210715:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122552 -> 122560 00010000:00010000:1.0:1789760775.210727:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4d02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62734336->62738431] (req 62734336->62738431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.210750:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4d02 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62734336->62738431] (req 62734336->62738431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.210760:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4d02 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62734336->62738431] (req 62734336->62738431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.210767:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760775.210773:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4d02 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62734336->62738431] (req 62734336->62738431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.210782:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4d02 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62734336->62738431] (req 62734336->62738431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.210801:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4d02 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62734336->62738431] (req 62734336->62738431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.210810:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4d02 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62734336->62738431] (req 62734336->62738431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.210907:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554725888:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7314us (7637us total) trans 4294982625 rc 0/0 00000100:00100000:1.0:1789760775.210917:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15327 00000100:00100000:1.0:1789760775.210926:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718434894239 time 718449193868 state 0x2 ref 0 00000100:00100000:1.0:1789760775.222571:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760775.222752:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554726016 00000100:00100000:2.0:1789760775.222762:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554726016 00000100:00100000:2.0:1789760775.222812:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718434894239 time 718449193868 state 0x0 ref 1 00000100:00100000:2.0:1789760775.222827:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718449454284 time 718449193868 state 0x1 ref 1 00000100:00100000:2.0:1789760775.222860:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718449454284 time 718468478722 state 0x0 ref 1 00000100:00100000:2.0:1789760775.222873:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d82680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554726016:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760775.222919:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62738432, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1dec0 00010000:00010000:2.0:1789760775.222929:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4d09 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.222947:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760775.222951:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.222957:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4d09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62738432->62742527] (req 62738432->62742527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.222971:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4d09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62738432->62742527] (req 62738432->62742527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.232758:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982626 is committed 00002000:00010000:2.0:1789760775.233081:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62738432 -> 62742528 00002000:00010000:2.0:1789760775.233085:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122560 -> 122568 00010000:00010000:2.0:1789760775.233100:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4d09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62738432->62742527] (req 62738432->62742527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.233112:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4d09 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62738432->62742527] (req 62738432->62742527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.233122:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4d09 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62738432->62742527] (req 62738432->62742527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.233129:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760775.233134:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4d09 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62738432->62742527] (req 62738432->62742527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.233142:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4d09 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62738432->62742527] (req 62738432->62742527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.233162:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4d09 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62738432->62742527] (req 62738432->62742527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.233170:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4d09 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62738432->62742527] (req 62738432->62742527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.233266:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d82680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554726016:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10391us (10700us total) trans 4294982626 rc 0/0 00000100:00100000:2.0:1789760775.233277:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15328 00000100:00100000:2.0:1789760775.233287:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718449454284 time 718468478722 state 0x2 ref 0 00000100:00100000:2.0:1789760775.241237:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760775.242265:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554726144 00000100:00100000:0.0:1789760775.242277:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554726144 00000100:00100000:0.0:1789760775.242343:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718449454284 time 718468478722 state 0x0 ref 1 00000100:00100000:0.0:1789760775.242359:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718468739138 time 718468478722 state 0x1 ref 1 00000100:00100000:0.0:1789760775.242473:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718468739138 time 718488089256 state 0x0 ref 1 00000100:00100000:0.0:1789760775.242486:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e56300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554726144:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760775.242536:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62742528, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1dec8 00010000:00010000:0.0:1789760775.242544:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d10 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.242559:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760775.242564:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.242571:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62742528->62746623] (req 62742528->62746623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.242585:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62742528->62746623] (req 62742528->62746623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.251083:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982627 is committed 00002000:00010000:0.0:1789760775.251216:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62742528 -> 62746624 00002000:00010000:0.0:1789760775.251222:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122568 -> 122576 00010000:00010000:0.0:1789760775.251235:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62742528->62746623] (req 62742528->62746623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.251249:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d10 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62742528->62746623] (req 62742528->62746623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.251259:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62742528->62746623] (req 62742528->62746623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.251267:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760775.251272:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62742528->62746623] (req 62742528->62746623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.251282:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d10 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62742528->62746623] (req 62742528->62746623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.251302:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d10 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62742528->62746623] (req 62742528->62746623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.251316:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d10 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62742528->62746623] (req 62742528->62746623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.251446:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e56300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554726144:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8957us (10213us total) trans 4294982627 rc 0/0 00000100:00100000:0.0:1789760775.251457:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15329 00000100:00100000:0.0:1789760775.251467:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718468739138 time 718488089256 state 0x2 ref 0 00000100:00100000:3.0:1789760775.259257:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760775.259470:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554726272 00000100:00100000:0.0:1789760775.259480:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554726272 00000100:00100000:0.0:1789760775.259921:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718468739138 time 718488089256 state 0x0 ref 1 00000100:00100000:0.0:1789760775.259937:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718488349672 time 718488089256 state 0x1 ref 1 00000100:00100000:0.0:1789760775.260030:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718488349672 time 718505646845 state 0x0 ref 1 00000100:00100000:0.0:1789760775.260043:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06403480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554726272:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760775.260093:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62746624, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1ded0 00010000:00010000:0.0:1789760775.260101:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d17 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.260129:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760775.260134:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.260141:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62746624->62750719] (req 62746624->62750719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.260160:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62746624->62750719] (req 62746624->62750719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.267994:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982628 is committed 00002000:00010000:0.0:1789760775.268293:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62746624 -> 62750720 00002000:00010000:0.0:1789760775.268299:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122576 -> 122584 00010000:00010000:0.0:1789760775.268314:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62746624->62750719] (req 62746624->62750719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.268328:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d17 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62746624->62750719] (req 62746624->62750719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.268338:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62746624->62750719] (req 62746624->62750719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.268346:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760775.268351:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62746624->62750719] (req 62746624->62750719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.268359:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d17 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62746624->62750719] (req 62746624->62750719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.268440:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d17 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62746624->62750719] (req 62746624->62750719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.268449:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d17 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62746624->62750719] (req 62746624->62750719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.268725:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06403480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554726272:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8678us (9472us total) trans 4294982628 rc 0/0 00000100:00100000:0.0:1789760775.268747:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15330 00000100:00100000:0.0:1789760775.268985:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718488349672 time 718505646845 state 0x2 ref 0 00000100:00100000:1.0:1789760775.274155:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760775.276190:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554726400 00000100:00100000:2.0:1789760775.276205:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554726400 00000100:00100000:2.0:1789760775.276274:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718488349672 time 718505646845 state 0x0 ref 1 00000100:00100000:2.0:1789760775.276290:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718505907261 time 718505646845 state 0x1 ref 1 00000100:00100000:2.0:1789760775.276366:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718505907261 time 718521984324 state 0x0 ref 1 00000100:00100000:2.0:1789760775.276412:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d81500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554726400:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760775.276479:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62750720, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1ded8 00010000:00010000:2.0:1789760775.276498:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4d1e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.276523:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760775.276528:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.276535:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4d1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62750720->62754815] (req 62750720->62754815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.276549:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4d1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62750720->62754815] (req 62750720->62754815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.287653:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982629 is committed 00002000:00010000:2.0:1789760775.287977:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62750720 -> 62754816 00002000:00010000:2.0:1789760775.287983:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122584 -> 122592 00010000:00010000:2.0:1789760775.287998:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4d1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62750720->62754815] (req 62750720->62754815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.288011:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4d1e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62750720->62754815] (req 62750720->62754815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.288020:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4d1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62750720->62754815] (req 62750720->62754815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.288028:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760775.288033:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4d1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62750720->62754815] (req 62750720->62754815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.288042:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4d1e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62750720->62754815] (req 62750720->62754815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.288069:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4d1e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62750720->62754815] (req 62750720->62754815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.288078:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4d1e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62750720->62754815] (req 62750720->62754815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.288600:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d81500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554726400:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12216us (14450us total) trans 4294982629 rc 0/0 00000100:00100000:2.0:1789760775.288622:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15331 00000100:00100000:2.0:1789760775.288633:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718505907261 time 718521984324 state 0x2 ref 0 00000100:00100000:2.0:1789760775.292790:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760775.292983:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554726528 00000100:00100000:1.0:1789760775.292993:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554726528 00000100:00100000:1.0:1789760775.293042:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718505907261 time 718521984324 state 0x0 ref 1 00000100:00100000:1.0:1789760775.293058:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718522244740 time 718521984324 state 0x1 ref 1 00000100:00100000:1.0:1789760775.293085:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718522244740 time 718538703726 state 0x0 ref 1 00000100:00100000:1.0:1789760775.293100:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554726528:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760775.293159:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62754816, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1dee0 00010000:00010000:1.0:1789760775.293168:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c4d25 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.293185:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760775.293189:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.293196:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c4d25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62754816->62758911] (req 62754816->62758911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.293222:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c4d25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62754816->62758911] (req 62754816->62758911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.300997:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982630 is committed 00002000:00010000:1.0:1789760775.301116:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62754816 -> 62758912 00002000:00010000:1.0:1789760775.301121:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122592 -> 122600 00010000:00010000:1.0:1789760775.301135:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c4d25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62754816->62758911] (req 62754816->62758911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.301148:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c4d25 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62754816->62758911] (req 62754816->62758911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.301157:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c4d25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62754816->62758911] (req 62754816->62758911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.301165:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760775.301170:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c4d25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62754816->62758911] (req 62754816->62758911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.301180:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c4d25 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62754816->62758911] (req 62754816->62758911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.301199:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c4d25 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62754816->62758911] (req 62754816->62758911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.301207:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c4d25 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62754816->62758911] (req 62754816->62758911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.301302:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554726528:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8200us (8517us total) trans 4294982630 rc 0/0 00000100:00100000:1.0:1789760775.301314:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15332 00000100:00100000:1.0:1789760775.301323:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718522244740 time 718538703726 state 0x2 ref 0 00000100:00100000:3.0:1789760775.308507:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760775.308877:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554726656 00000100:00100000:0.0:1789760775.308887:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554726656 00000100:00100000:0.0:1789760775.308953:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718522244740 time 718538703726 state 0x0 ref 1 00000100:00100000:0.0:1789760775.308968:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718538964142 time 718538703726 state 0x1 ref 1 00000100:00100000:0.0:1789760775.309004:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718538964142 time 718554622377 state 0x0 ref 1 00000100:00100000:0.0:1789760775.309015:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06402a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554726656:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760775.309061:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62758912, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1dee8 00010000:00010000:0.0:1789760775.309070:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d2c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.309086:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760775.309091:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.309098:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62758912->62763007] (req 62758912->62763007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.309123:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62758912->62763007] (req 62758912->62763007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.314620:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982631 is committed 00002000:00010000:0.0:1789760775.314823:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62758912 -> 62763008 00002000:00010000:0.0:1789760775.314828:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122600 -> 122608 00010000:00010000:0.0:1789760775.314842:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62758912->62763007] (req 62758912->62763007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.314854:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d2c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62758912->62763007] (req 62758912->62763007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.314864:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62758912->62763007] (req 62758912->62763007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.314872:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760775.314876:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62758912->62763007] (req 62758912->62763007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.314887:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d2c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62758912->62763007] (req 62758912->62763007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.314908:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d2c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62758912->62763007] (req 62758912->62763007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.314925:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d2c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62758912->62763007] (req 62758912->62763007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.315026:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06402a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554726656:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6007us (6523us total) trans 4294982631 rc 0/0 00000100:00100000:0.0:1789760775.315039:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15333 00000100:00100000:0.0:1789760775.315048:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718538964142 time 718554622377 state 0x2 ref 0 00000100:00100000:3.0:1789760775.323052:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760775.323240:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554726784 00000100:00100000:0.0:1789760775.323250:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554726784 00000100:00100000:0.0:1789760775.323298:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718538964142 time 718554622377 state 0x0 ref 1 00000100:00100000:0.0:1789760775.323312:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718554882793 time 718554622377 state 0x1 ref 1 00000100:00100000:0.0:1789760775.323337:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718554882793 time 718568956333 state 0x0 ref 1 00000100:00100000:0.0:1789760775.323349:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06400e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554726784:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760775.323432:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62763008, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1def0 00010000:00010000:0.0:1789760775.323442:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d33 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.323457:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760775.323461:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.323468:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62763008->62767103] (req 62763008->62767103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.323481:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62763008->62767103] (req 62763008->62767103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.328792:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982632 is committed 00002000:00010000:0.0:1789760775.328909:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62763008 -> 62767104 00002000:00010000:0.0:1789760775.329055:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122608 -> 122616 00010000:00010000:0.0:1789760775.329071:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62763008->62767103] (req 62763008->62767103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.329084:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d33 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62763008->62767103] (req 62763008->62767103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.329094:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62763008->62767103] (req 62763008->62767103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.329101:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760775.329164:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62763008->62767103] (req 62763008->62767103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.329176:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d33 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62763008->62767103] (req 62763008->62767103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.329196:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d33 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62763008->62767103] (req 62763008->62767103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.329204:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c4d33 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62763008->62767103] (req 62763008->62767103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.329302:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06400e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554726784:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5950us (6254us total) trans 4294982632 rc 0/0 00000100:00100000:0.0:1789760775.329314:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15334 00000100:00100000:0.0:1789760775.329323:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718554882793 time 718568956333 state 0x2 ref 0 00000100:00100000:2.0:1789760775.335866:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760775.335963:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554726912 00000100:00100000:1.0:1789760775.335973:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554726912 00000100:00100000:1.0:1789760775.336027:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718554882793 time 718568956333 state 0x0 ref 1 00000100:00100000:1.0:1789760775.336042:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718569216749 time 718568956333 state 0x1 ref 1 00000100:00100000:1.0:1789760775.336076:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718569216749 time 718581694291 state 0x0 ref 1 00000100:00100000:1.0:1789760775.336088:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e56a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554726912:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760775.336134:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62767104, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1def8 00010000:00010000:1.0:1789760775.336144:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c4d3a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.336159:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760775.336163:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.336169:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c4d3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62767104->62771199] (req 62767104->62771199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.336183:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c4d3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62767104->62771199] (req 62767104->62771199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.342298:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982633 is committed 00002000:00010000:2.0:1789760775.347762:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62767104 -> 62771200 00002000:00010000:2.0:1789760775.347774:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122616 -> 122624 00010000:00010000:2.0:1789760775.347788:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c4d3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62767104->62771199] (req 62767104->62771199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.347802:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c4d3a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62767104->62771199] (req 62767104->62771199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.347812:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c4d3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62767104->62771199] (req 62767104->62771199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.347819:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760775.347825:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c4d3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62767104->62771199] (req 62767104->62771199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.347834:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c4d3a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62767104->62771199] (req 62767104->62771199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.347854:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c4d3a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62767104->62771199] (req 62767104->62771199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.347861:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c4d3a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62767104->62771199] (req 62767104->62771199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.348023:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e56a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554726912:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11931us (12160us total) trans 4294982633 rc 0/0 00000100:00100000:2.0:1789760775.348035:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15335 00000100:00100000:2.0:1789760775.348044:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718569216749 time 718581694291 state 0x2 ref 0 00000100:00100000:3.0:1789760775.359618:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760775.359823:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554727040 00000100:00100000:1.0:1789760775.359833:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554727040 00000100:00100000:1.0:1789760775.359887:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718569216749 time 718581694291 state 0x0 ref 1 00000100:00100000:1.0:1789760775.359902:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718581954707 time 718581694291 state 0x1 ref 1 00000100:00100000:1.0:1789760775.359935:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718581954707 time 718605553352 state 0x0 ref 1 00000100:00100000:1.0:1789760775.359948:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06403b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554727040:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760775.359998:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62771200, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1df00 00010000:00010000:1.0:1789760775.360009:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4d41 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.360026:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760775.360031:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.360038:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4d41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62771200->62775295] (req 62771200->62775295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.360053:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4d41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62771200->62775295] (req 62771200->62775295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.368129:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982634 is committed 00002000:00010000:1.0:1789760775.371306:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62771200 -> 62775296 00002000:00010000:1.0:1789760775.371316:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122624 -> 122632 00010000:00010000:1.0:1789760775.371329:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4d41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62771200->62775295] (req 62771200->62775295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.371344:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4d41 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62771200->62775295] (req 62771200->62775295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.371353:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4d41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62771200->62775295] (req 62771200->62775295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.371361:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760775.371366:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4d41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62771200->62775295] (req 62771200->62775295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.371374:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4d41 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62771200->62775295] (req 62771200->62775295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.371394:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4d41 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62771200->62775295] (req 62771200->62775295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.371408:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4d41 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62771200->62775295] (req 62771200->62775295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.371687:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06403b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554727040:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11735us (12073us total) trans 4294982634 rc 0/0 00000100:00100000:1.0:1789760775.371699:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15336 00000100:00100000:1.0:1789760775.371708:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718581954707 time 718605553352 state 0x2 ref 0 00000100:00100000:3.0:1789760775.376879:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760775.377072:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554727168 00000100:00100000:0.0:1789760775.377083:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554727168 00000100:00100000:0.0:1789760775.377148:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718581954707 time 718605553352 state 0x0 ref 1 00000100:00100000:0.0:1789760775.377164:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718605813768 time 718605553352 state 0x1 ref 1 00000100:00100000:0.0:1789760775.377193:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718605813768 time 718622811328 state 0x0 ref 1 00000100:00100000:0.0:1789760775.377206:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06402300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554727168:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760775.377253:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62775296, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1df08 00010000:00010000:0.0:1789760775.377263:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c4d48 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.377279:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760775.377284:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.377291:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c4d48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62775296->62779391] (req 62775296->62779391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.377304:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c4d48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62775296->62779391] (req 62775296->62779391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.382371:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982635 is committed 00002000:00010000:0.0:1789760775.382540:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62775296 -> 62779392 00002000:00010000:0.0:1789760775.382545:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122632 -> 122640 00010000:00010000:0.0:1789760775.382557:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c4d48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62775296->62779391] (req 62775296->62779391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.382574:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c4d48 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62775296->62779391] (req 62775296->62779391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.382583:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c4d48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62775296->62779391] (req 62775296->62779391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.382590:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760775.382594:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c4d48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62775296->62779391] (req 62775296->62779391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.382603:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c4d48 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62775296->62779391] (req 62775296->62779391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.382623:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c4d48 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62775296->62779391] (req 62775296->62779391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.382630:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c4d48 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62775296->62779391] (req 62775296->62779391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.382875:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06402300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554727168:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5665us (6380us total) trans 4294982635 rc 0/0 00000100:00100000:0.0:1789760775.382886:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15337 00000100:00100000:0.0:1789760775.382895:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718605813768 time 718622811328 state 0x2 ref 0 00000100:00100000:0.0:1789760775.387035:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760775.387136:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554727296 00000100:00100000:2.0:1789760775.387146:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554727296 00000100:00100000:2.0:1789760775.387197:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718605813768 time 718622811328 state 0x0 ref 1 00000100:00100000:2.0:1789760775.387213:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718623071744 time 718622811328 state 0x1 ref 1 00000100:00100000:2.0:1789760775.387246:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718623071744 time 718632864930 state 0x0 ref 1 00000100:00100000:2.0:1789760775.387258:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554727296:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760775.387303:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62779392, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1df10 00010000:00010000:2.0:1789760775.387313:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4d4f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.387339:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760775.387344:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.387351:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4d4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62779392->62783487] (req 62779392->62783487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.387365:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4d4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62779392->62783487] (req 62779392->62783487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.395629:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982636 is committed 00002000:00010000:2.0:1789760775.395783:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62779392 -> 62783488 00002000:00010000:2.0:1789760775.395789:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122640 -> 122648 00010000:00010000:2.0:1789760775.395801:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4d4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62779392->62783487] (req 62779392->62783487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.395815:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4d4f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62779392->62783487] (req 62779392->62783487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.395824:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4d4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62779392->62783487] (req 62779392->62783487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.395836:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760775.395841:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4d4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62779392->62783487] (req 62779392->62783487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.395851:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4d4f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62779392->62783487] (req 62779392->62783487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.395871:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4d4f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62779392->62783487] (req 62779392->62783487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.395878:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4d4f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62779392->62783487] (req 62779392->62783487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.395978:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554727296:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8716us (8946us total) trans 4294982636 rc 0/0 00000100:00100000:2.0:1789760775.395989:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15338 00000100:00100000:2.0:1789760775.395999:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718623071744 time 718632864930 state 0x2 ref 0 00000100:00100000:3.0:1789760775.399605:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760775.399689:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554727424 00000100:00100000:2.0:1789760775.399698:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554727424 00000100:00100000:2.0:1789760775.399756:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718623071744 time 718632864930 state 0x0 ref 1 00000100:00100000:2.0:1789760775.399771:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718633125346 time 718632864930 state 0x1 ref 1 00000100:00100000:2.0:1789760775.399840:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718633125346 time 718645458030 state 0x0 ref 1 00000100:00100000:2.0:1789760775.399854:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06403800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554727424:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760775.399899:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62783488, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1df18 00010000:00010000:2.0:1789760775.399907:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4d56 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.399922:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760775.399927:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.399933:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4d56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62783488->62787583] (req 62783488->62787583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.399946:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4d56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62783488->62787583] (req 62783488->62787583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.405042:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982637 is committed 00002000:00010000:2.0:1789760775.405174:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62783488 -> 62787584 00002000:00010000:2.0:1789760775.405179:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122648 -> 122656 00010000:00010000:2.0:1789760775.405193:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4d56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62783488->62787583] (req 62783488->62787583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.405205:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4d56 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62783488->62787583] (req 62783488->62787583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.405215:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4d56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62783488->62787583] (req 62783488->62787583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.405223:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760775.405228:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4d56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62783488->62787583] (req 62783488->62787583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.405237:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4d56 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62783488->62787583] (req 62783488->62787583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.405256:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4d56 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62783488->62787583] (req 62783488->62787583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.405270:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4d56 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62783488->62787583] (req 62783488->62787583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.405362:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06403800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554727424:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5506us (5761us total) trans 4294982637 rc 0/0 00000100:00100000:2.0:1789760775.405373:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15339 00000100:00100000:2.0:1789760775.405463:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718633125346 time 718645458030 state 0x2 ref 0 00000100:00100000:1.0:1789760775.410800:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760775.410897:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554727552 00000100:00100000:2.0:1789760775.410907:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554727552 00000100:00100000:2.0:1789760775.410956:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718633125346 time 718645458030 state 0x0 ref 1 00000100:00100000:2.0:1789760775.410971:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718645718446 time 718645458030 state 0x1 ref 1 00000100:00100000:2.0:1789760775.411037:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718645718446 time 718656654655 state 0x0 ref 1 00000100:00100000:2.0:1789760775.411051:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554727552:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760775.411100:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62787584, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1df20 00010000:00010000:2.0:1789760775.411121:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4d5d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.411137:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760775.411142:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.411148:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4d5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62787584->62791679] (req 62787584->62791679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.411169:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4d5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62787584->62791679] (req 62787584->62791679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.416818:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982638 is committed 00002000:00010000:2.0:1789760775.416919:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62787584 -> 62791680 00002000:00010000:2.0:1789760775.416924:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122656 -> 122664 00010000:00010000:2.0:1789760775.416937:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4d5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62787584->62791679] (req 62787584->62791679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.416950:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4d5d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62787584->62791679] (req 62787584->62791679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.416958:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4d5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62787584->62791679] (req 62787584->62791679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.416966:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760775.416971:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4d5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62787584->62791679] (req 62787584->62791679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.416981:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4d5d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62787584->62791679] (req 62787584->62791679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.416998:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4d5d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62787584->62791679] (req 62787584->62791679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.417005:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4d5d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62787584->62791679] (req 62787584->62791679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.417100:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554727552:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6047us (6306us total) trans 4294982638 rc 0/0 00000100:00100000:2.0:1789760775.417116:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15340 00000100:00100000:2.0:1789760775.417125:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718645718446 time 718656654655 state 0x2 ref 0 00000100:00100000:0.0:1789760775.420476:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760775.420625:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554727680 00000100:00100000:3.0:1789760775.420634:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554727680 00000100:00100000:3.0:1789760775.420685:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718645718446 time 718656654655 state 0x0 ref 1 00000100:00100000:3.0:1789760775.420699:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718656915071 time 718656654655 state 0x1 ref 1 00000100:00100000:3.0:1789760775.420765:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718656915071 time 718666383252 state 0x0 ref 1 00000100:00100000:3.0:1789760775.420779:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554727680:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760775.420821:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62791680, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1df28 00010000:00010000:3.0:1789760775.420830:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4d64 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.420845:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760775.420850:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.420857:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4d64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62791680->62795775] (req 62791680->62795775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.420870:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4d64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62791680->62795775] (req 62791680->62795775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.427371:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982639 is committed 00002000:00010000:3.0:1789760775.427528:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62791680 -> 62795776 00002000:00010000:3.0:1789760775.427533:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122664 -> 122672 00010000:00010000:3.0:1789760775.427546:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4d64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62791680->62795775] (req 62791680->62795775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.427558:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4d64 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62791680->62795775] (req 62791680->62795775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.427567:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4d64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62791680->62795775] (req 62791680->62795775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.427575:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760775.427579:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4d64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62791680->62795775] (req 62791680->62795775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.427588:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4d64 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62791680->62795775] (req 62791680->62795775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.427607:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4d64 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62791680->62795775] (req 62791680->62795775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.427620:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4d64 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62791680->62795775] (req 62791680->62795775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.428308:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554727680:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7524us (7834us total) trans 4294982639 rc 0/0 00000100:00100000:3.0:1789760775.428319:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15341 00000100:00100000:3.0:1789760775.428328:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718656915071 time 718666383252 state 0x2 ref 0 00000100:00100000:2.0:1789760775.431990:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760775.432072:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554727808 00000100:00100000:0.0:1789760775.432082:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554727808 00000100:00100000:0.0:1789760775.432150:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718656915071 time 718666383252 state 0x0 ref 1 00000100:00100000:0.0:1789760775.432166:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718666643668 time 718666383252 state 0x1 ref 1 00000100:00100000:0.0:1789760775.432195:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718666643668 time 718677813806 state 0x0 ref 1 00000100:00100000:0.0:1789760775.432210:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe1c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554727808:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760775.432260:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62795776, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1df30 00010000:00010000:0.0:1789760775.432270:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4d6b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.432286:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760775.432290:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.432296:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4d6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62795776->62799871] (req 62795776->62799871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.432310:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4d6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62795776->62799871] (req 62795776->62799871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.437822:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982640 is committed 00002000:00010000:0.0:1789760775.437933:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62795776 -> 62799872 00002000:00010000:0.0:1789760775.437939:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122672 -> 122680 00010000:00010000:0.0:1789760775.437952:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4d6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62795776->62799871] (req 62795776->62799871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.437979:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4d6b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62795776->62799871] (req 62795776->62799871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.437989:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4d6b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62795776->62799871] (req 62795776->62799871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.437996:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760775.438001:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4d6b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62795776->62799871] (req 62795776->62799871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.438010:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4d6b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62795776->62799871] (req 62795776->62799871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.438031:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4d6b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62795776->62799871] (req 62795776->62799871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.438039:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4d6b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62795776->62799871] (req 62795776->62799871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.438240:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe1c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554727808:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6028us (6253us total) trans 4294982640 rc 0/0 00000100:00100000:0.0:1789760775.438252:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15342 00000100:00100000:0.0:1789760775.438262:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718666643668 time 718677813806 state 0x2 ref 0 00000100:00100000:1.0:1789760775.444028:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760775.444141:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554727936 00000100:00100000:2.0:1789760775.444151:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554727936 00000100:00100000:2.0:1789760775.444200:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718666643668 time 718677813806 state 0x0 ref 1 00000100:00100000:2.0:1789760775.444215:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718678074222 time 718677813806 state 0x1 ref 1 00000100:00100000:2.0:1789760775.444280:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718678074222 time 718689898631 state 0x0 ref 1 00000100:00100000:2.0:1789760775.444293:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d83100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554727936:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760775.444337:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62799872, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1df38 00010000:00010000:2.0:1789760775.444345:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4d72 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.444361:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760775.444365:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.444371:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4d72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62799872->62803967] (req 62799872->62803967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.444407:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4d72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62799872->62803967] (req 62799872->62803967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.454774:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982641 is committed 00002000:00010000:2.0:1789760775.454889:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62799872 -> 62803968 00002000:00010000:2.0:1789760775.454894:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122680 -> 122688 00010000:00010000:2.0:1789760775.454907:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4d72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62799872->62803967] (req 62799872->62803967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.454919:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4d72 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62799872->62803967] (req 62799872->62803967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.454928:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4d72 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62799872->62803967] (req 62799872->62803967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.454936:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760775.454940:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4d72 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62799872->62803967] (req 62799872->62803967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.454949:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4d72 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62799872->62803967] (req 62799872->62803967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.454969:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4d72 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62799872->62803967] (req 62799872->62803967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.454976:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4d72 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62799872->62803967] (req 62799872->62803967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.455070:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d83100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554727936:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10774us (11046us total) trans 4294982641 rc 0/0 00000100:00100000:2.0:1789760775.455080:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15343 00000100:00100000:2.0:1789760775.455090:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718678074222 time 718689898631 state 0x2 ref 0 00000100:00100000:2.0:1789760775.458196:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760775.458372:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554728064 00000100:00100000:1.0:1789760775.458420:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554728064 00000100:00100000:1.0:1789760775.458474:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718678074222 time 718689898631 state 0x0 ref 1 00000100:00100000:1.0:1789760775.458497:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718690159047 time 718689898631 state 0x1 ref 1 00000100:00100000:1.0:1789760775.458512:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718690159047 time 718704131937 state 0x0 ref 1 00000100:00100000:1.0:1789760775.458525:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e57800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554728064:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760775.458572:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62803968, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1df40 00010000:00010000:1.0:1789760775.458582:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4d79 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.458595:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760775.458599:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.458604:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4d79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62803968->62808063] (req 62803968->62808063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.458618:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4d79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62803968->62808063] (req 62803968->62808063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.466220:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982642 is committed 00002000:00010000:1.0:1789760775.466357:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62803968 -> 62808064 00002000:00010000:1.0:1789760775.466363:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122688 -> 122696 00010000:00010000:1.0:1789760775.466415:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4d79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62803968->62808063] (req 62803968->62808063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.466428:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4d79 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62803968->62808063] (req 62803968->62808063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.466437:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4d79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62803968->62808063] (req 62803968->62808063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.466454:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760775.466460:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4d79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62803968->62808063] (req 62803968->62808063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.466468:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4d79 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62803968->62808063] (req 62803968->62808063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.466489:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4d79 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62803968->62808063] (req 62803968->62808063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.466497:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4d79 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62803968->62808063] (req 62803968->62808063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.466602:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e57800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554728064:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8074us (8411us total) trans 4294982642 rc 0/0 00000100:00100000:1.0:1789760775.466612:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15344 00000100:00100000:1.0:1789760775.466621:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718690159047 time 718704131937 state 0x2 ref 0 00000100:00100000:3.0:1789760775.470081:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760775.471610:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554728192 00000100:00100000:0.0:1789760775.471623:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554728192 00000100:00100000:0.0:1789760775.471678:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718690159047 time 718704131937 state 0x0 ref 1 00000100:00100000:0.0:1789760775.471694:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718704392353 time 718704131937 state 0x1 ref 1 00000100:00100000:0.0:1789760775.471769:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718704392353 time 718717387179 state 0x0 ref 1 00000100:00100000:0.0:1789760775.471792:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06401180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554728192:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760775.471839:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62808064, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1df48 00010000:00010000:0.0:1789760775.471848:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4d80 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.471864:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760775.471869:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.471876:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4d80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62808064->62812159] (req 62808064->62812159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.471891:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4d80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62808064->62812159] (req 62808064->62812159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.477241:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982643 is committed 00002000:00010000:0.0:1789760775.477436:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62808064 -> 62812160 00002000:00010000:0.0:1789760775.477441:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122696 -> 122704 00010000:00010000:0.0:1789760775.477518:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4d80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62808064->62812159] (req 62808064->62812159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.477537:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4d80 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62808064->62812159] (req 62808064->62812159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.477547:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4d80 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62808064->62812159] (req 62808064->62812159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.477554:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760775.477559:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4d80 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62808064->62812159] (req 62808064->62812159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.477568:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4d80 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62808064->62812159] (req 62808064->62812159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.477708:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4d80 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62808064->62812159] (req 62808064->62812159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.477718:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4d80 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62808064->62812159] (req 62808064->62812159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.477863:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06401180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554728192:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6069us (7791us total) trans 4294982643 rc 0/0 00000100:00100000:0.0:1789760775.477874:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15345 00000100:00100000:0.0:1789760775.477883:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718704392353 time 718717387179 state 0x2 ref 0 00000100:00100000:2.0:1789760775.481817:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760775.481902:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554728320 00000100:00100000:0.0:1789760775.481913:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554728320 00000100:00100000:0.0:1789760775.481962:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718704392353 time 718717387179 state 0x0 ref 1 00000100:00100000:0.0:1789760775.481976:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718717647595 time 718717387179 state 0x1 ref 1 00000100:00100000:0.0:1789760775.482050:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718717647595 time 718727667807 state 0x0 ref 1 00000100:00100000:0.0:1789760775.482063:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554728320:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760775.482108:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62812160, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1df50 00010000:00010000:0.0:1789760775.482116:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4d87 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.482132:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760775.482136:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.482142:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4d87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62812160->62816255] (req 62812160->62816255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.482155:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4d87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62812160->62816255] (req 62812160->62816255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.487878:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982644 is committed 00002000:00010000:0.0:1789760775.488040:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62812160 -> 62816256 00002000:00010000:0.0:1789760775.488045:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122704 -> 122712 00010000:00010000:0.0:1789760775.488058:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4d87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62812160->62816255] (req 62812160->62816255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.488071:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4d87 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62812160->62816255] (req 62812160->62816255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.488086:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4d87 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62812160->62816255] (req 62812160->62816255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.488095:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760775.488100:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4d87 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62812160->62816255] (req 62812160->62816255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.488121:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4d87 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62812160->62816255] (req 62812160->62816255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.488141:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4d87 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62812160->62816255] (req 62812160->62816255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.488150:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4d87 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62812160->62816255] (req 62812160->62816255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.488296:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554728320:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6229us (6482us total) trans 4294982644 rc 0/0 00000100:00100000:0.0:1789760775.488308:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15346 00000100:00100000:0.0:1789760775.488319:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718717647595 time 718727667807 state 0x2 ref 0 00000100:00100000:2.0:1789760775.493845:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760775.493938:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554728448 00000100:00100000:1.0:1789760775.493948:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554728448 00000100:00100000:1.0:1789760775.494001:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718717647595 time 718727667807 state 0x0 ref 1 00000100:00100000:1.0:1789760775.494015:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718727928223 time 718727667807 state 0x1 ref 1 00000100:00100000:1.0:1789760775.494079:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718727928223 time 718739696650 state 0x0 ref 1 00000100:00100000:1.0:1789760775.494091:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e54a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554728448:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760775.494149:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62816256, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1df58 00010000:00010000:1.0:1789760775.494158:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4d8e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.494174:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760775.494179:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.494185:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4d8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62816256->62820351] (req 62816256->62820351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.494198:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4d8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62816256->62820351] (req 62816256->62820351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.501297:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982645 is committed 00002000:00010000:1.0:1789760775.506207:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62816256 -> 62820352 00002000:00010000:1.0:1789760775.506217:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122712 -> 122720 00010000:00010000:1.0:1789760775.506230:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4d8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62816256->62820351] (req 62816256->62820351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.506243:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4d8e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62816256->62820351] (req 62816256->62820351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.506252:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4d8e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62816256->62820351] (req 62816256->62820351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.506260:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760775.506265:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4d8e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62816256->62820351] (req 62816256->62820351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.506275:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4d8e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62816256->62820351] (req 62816256->62820351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.506363:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4d8e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62816256->62820351] (req 62816256->62820351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.506453:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4d8e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62816256->62820351] (req 62816256->62820351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.506596:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554728448:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12502us (12755us total) trans 4294982645 rc 0/0 00000100:00100000:1.0:1789760775.506607:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15347 00000100:00100000:1.0:1789760775.506618:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718727928223 time 718739696650 state 0x2 ref 0 00000100:00100000:3.0:1789760775.511550:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760775.511825:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554728576 00000100:00100000:1.0:1789760775.511836:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554728576 00000100:00100000:1.0:1789760775.511951:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718727928223 time 718739696650 state 0x0 ref 1 00000100:00100000:1.0:1789760775.511967:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718739957066 time 718739696650 state 0x1 ref 1 00000100:00100000:1.0:1789760775.511997:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718739957066 time 718757615107 state 0x0 ref 1 00000100:00100000:1.0:1789760775.512011:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06401c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554728576:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760775.512070:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62820352, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1df60 00010000:00010000:1.0:1789760775.512079:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c4d95 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.512094:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760775.512098:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.512105:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c4d95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62820352->62824447] (req 62820352->62824447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.512122:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c4d95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62820352->62824447] (req 62820352->62824447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.518802:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982646 is committed 00002000:00010000:1.0:1789760775.518976:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62820352 -> 62824448 00002000:00010000:1.0:1789760775.518981:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122720 -> 122728 00010000:00010000:1.0:1789760775.518993:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c4d95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62820352->62824447] (req 62820352->62824447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.519005:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c4d95 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62820352->62824447] (req 62820352->62824447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.519013:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c4d95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62820352->62824447] (req 62820352->62824447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.519019:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760775.519024:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c4d95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62820352->62824447] (req 62820352->62824447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.519033:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c4d95 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62820352->62824447] (req 62820352->62824447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.519051:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c4d95 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62820352->62824447] (req 62820352->62824447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.519058:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c4d95 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62820352->62824447] (req 62820352->62824447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.519148:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06401c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554728576:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7135us (7603us total) trans 4294982646 rc 0/0 00000100:00100000:1.0:1789760775.519159:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15348 00000100:00100000:1.0:1789760775.519168:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718739957066 time 718757615107 state 0x2 ref 0 00000100:00100000:2.0:1789760775.523058:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760775.523529:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554728704 00000100:00100000:0.0:1789760775.523539:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554728704 00000100:00100000:0.0:1789760775.523593:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718739957066 time 718757615107 state 0x0 ref 1 00000100:00100000:0.0:1789760775.523611:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718757875523 time 718757615107 state 0x1 ref 1 00000100:00100000:0.0:1789760775.523632:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718757875523 time 718769250914 state 0x0 ref 1 00000100:00100000:0.0:1789760775.523644:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554728704:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760775.523692:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62824448, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1df68 00010000:00010000:0.0:1789760775.523702:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4d9c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.523721:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760775.523726:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.523744:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4d9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62824448->62828543] (req 62824448->62828543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.523762:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4d9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62824448->62828543] (req 62824448->62828543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.528998:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982647 is committed 00002000:00010000:0.0:1789760775.529163:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62824448 -> 62828544 00002000:00010000:0.0:1789760775.529169:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122728 -> 122736 00010000:00010000:0.0:1789760775.529181:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4d9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62824448->62828543] (req 62824448->62828543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.529195:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4d9c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62824448->62828543] (req 62824448->62828543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.529204:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4d9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62824448->62828543] (req 62824448->62828543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.529212:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760775.529216:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4d9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62824448->62828543] (req 62824448->62828543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.529225:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4d9c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62824448->62828543] (req 62824448->62828543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.529244:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4d9c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62824448->62828543] (req 62824448->62828543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.529252:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4d9c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62824448->62828543] (req 62824448->62828543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.529354:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554728704:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5706us (6298us total) trans 4294982647 rc 0/0 00000100:00100000:0.0:1789760775.529364:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15349 00000100:00100000:0.0:1789760775.529373:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718757875523 time 718769250914 state 0x2 ref 0 00000100:00100000:2.0:1789760775.534205:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760775.534283:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554728832 00000100:00100000:0.0:1789760775.534293:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554728832 00000100:00100000:0.0:1789760775.534355:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718757875523 time 718769250914 state 0x0 ref 1 00000100:00100000:0.0:1789760775.534371:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718769511330 time 718769250914 state 0x1 ref 1 00000100:00100000:0.0:1789760775.534465:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718769511330 time 718780082144 state 0x0 ref 1 00000100:00100000:0.0:1789760775.534480:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e56680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554728832:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760775.534528:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62828544, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1df70 00010000:00010000:0.0:1789760775.534537:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4da3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.534552:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760775.534556:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.534562:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4da3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62828544->62832639] (req 62828544->62832639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.534576:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4da3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62828544->62832639] (req 62828544->62832639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.540181:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982648 is committed 00002000:00010000:0.0:1789760775.544425:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62828544 -> 62832640 00002000:00010000:0.0:1789760775.544442:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122736 -> 122744 00010000:00010000:0.0:1789760775.544464:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4da3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62828544->62832639] (req 62828544->62832639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.544477:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4da3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62828544->62832639] (req 62828544->62832639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.544486:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4da3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62828544->62832639] (req 62828544->62832639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.544494:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760775.544499:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4da3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62828544->62832639] (req 62828544->62832639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.544509:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4da3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62828544->62832639] (req 62828544->62832639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.544529:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4da3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62828544->62832639] (req 62828544->62832639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.544537:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4da3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62828544->62832639] (req 62828544->62832639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.544639:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e56680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554728832:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10155us (10439us total) trans 4294982648 rc 0/0 00000100:00100000:0.0:1789760775.544650:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15350 00000100:00100000:0.0:1789760775.544659:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718769511330 time 718780082144 state 0x2 ref 0 00000100:00100000:3.0:1789760775.553094:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760775.553440:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554728960 00000100:00100000:1.0:1789760775.553459:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554728960 00000100:00100000:1.0:1789760775.553515:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718769511330 time 718780082144 state 0x0 ref 1 00000100:00100000:1.0:1789760775.553530:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718780342560 time 718780082144 state 0x1 ref 1 00000100:00100000:1.0:1789760775.553564:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718780342560 time 718799181772 state 0x0 ref 1 00000100:00100000:1.0:1789760775.553576:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06402d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554728960:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760775.553626:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62832640, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1df78 00010000:00010000:1.0:1789760775.553636:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c4daa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.553651:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760775.553655:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.553661:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c4daa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62832640->62836735] (req 62832640->62836735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.553675:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c4daa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62832640->62836735] (req 62832640->62836735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.563822:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982649 is committed 00002000:00010000:1.0:1789760775.564019:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62832640 -> 62836736 00002000:00010000:1.0:1789760775.564025:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122744 -> 122752 00010000:00010000:1.0:1789760775.564038:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c4daa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62832640->62836735] (req 62832640->62836735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.564051:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c4daa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62832640->62836735] (req 62832640->62836735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.564060:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c4daa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62832640->62836735] (req 62832640->62836735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.564066:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760775.564071:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c4daa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62832640->62836735] (req 62832640->62836735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.564079:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c4daa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62832640->62836735] (req 62832640->62836735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.564099:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c4daa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62832640->62836735] (req 62832640->62836735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.564164:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c4daa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62832640->62836735] (req 62832640->62836735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.564305:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06402d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554728960:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10725us (11216us total) trans 4294982649 rc 0/0 00000100:00100000:1.0:1789760775.564316:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15351 00000100:00100000:1.0:1789760775.564326:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718780342560 time 718799181772 state 0x2 ref 0 00000100:00100000:0.0:1789760775.571205:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03e55f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3648:1876699573914496:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789760775.571278:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760775.571453:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573914496 00000100:00100000:2.0:1789760775.571462:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573914496 00000100:00100000:2.0:1789760775.571523:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 382 00000100:00100000:2.0:1789760775.571533:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3648:x1876699573914496:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789760775.571566:0:6525:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446383616 avail 1300144128 00000100:00100000:2.0:1789760775.571665:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3648:x1876699573914496:12345-0@lo:41:osp-pre-1-0.0 Request processed in 132us (385us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760775.571672:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 382 00000100:00100000:3.0:1789760775.572053:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03e55f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3648:1876699573914496:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789760775.572162:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760775.572278:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554729088 00000100:00100000:3.0:1789760775.572284:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554729088 00000100:00100000:3.0:1789760775.572316:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718780342560 time 718799181772 state 0x0 ref 1 00000100:00100000:3.0:1789760775.572326:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718799442188 time 718799181772 state 0x1 ref 1 00000100:00100000:3.0:1789760775.572348:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718799442188 time 718817968040 state 0x0 ref 1 00000100:00100000:3.0:1789760775.572355:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d83480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554729088:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760775.572405:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62836736, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1df80 00010000:00010000:3.0:1789760775.572412:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4db1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.572428:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760775.572432:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.572438:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4db1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62836736->62840831] (req 62836736->62840831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.572690:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4db1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62836736->62840831] (req 62836736->62840831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.581030:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982650 is committed 00002000:00010000:1.0:1789760775.581452:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62836736 -> 62840832 00002000:00010000:1.0:1789760775.581458:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122752 -> 122760 00010000:00010000:1.0:1789760775.581473:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4db1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62836736->62840831] (req 62836736->62840831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.581590:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4db1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62836736->62840831] (req 62836736->62840831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.581610:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4db1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62836736->62840831] (req 62836736->62840831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.581619:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760775.581624:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4db1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62836736->62840831] (req 62836736->62840831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.581634:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4db1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62836736->62840831] (req 62836736->62840831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.581653:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4db1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62836736->62840831] (req 62836736->62840831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.581661:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c4db1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62836736->62840831] (req 62836736->62840831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.581767:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d83480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554729088:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9407us (9603us total) trans 4294982650 rc 0/0 00000100:00100000:1.0:1789760775.581778:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15352 00000100:00100000:1.0:1789760775.581787:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718799442188 time 718817968040 state 0x2 ref 0 00000100:00100000:3.0:1789760775.585780:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760775.586096:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554729216 00000100:00100000:0.0:1789760775.586158:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554729216 00000100:00100000:0.0:1789760775.586348:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718799442188 time 718817968040 state 0x0 ref 1 00000100:00100000:0.0:1789760775.586365:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718818228456 time 718817968040 state 0x1 ref 1 00000100:00100000:0.0:1789760775.586589:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718818228456 time 718832205768 state 0x0 ref 1 00000100:00100000:0.0:1789760775.586608:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06400000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554729216:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760775.586658:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62840832, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1df88 00010000:00010000:0.0:1789760775.586667:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4db8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.586683:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760775.586687:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.586693:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4db8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62840832->62844927] (req 62840832->62844927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.586709:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4db8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62840832->62844927] (req 62840832->62844927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.593301:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982651 is committed 00002000:00010000:0.0:1789760775.593604:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62840832 -> 62844928 00002000:00010000:0.0:1789760775.593609:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122760 -> 122768 00010000:00010000:0.0:1789760775.593624:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4db8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62840832->62844927] (req 62840832->62844927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.593636:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4db8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62840832->62844927] (req 62840832->62844927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.593646:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4db8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62840832->62844927] (req 62840832->62844927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.593653:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760775.593659:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4db8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62840832->62844927] (req 62840832->62844927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.593668:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4db8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62840832->62844927] (req 62840832->62844927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.593829:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4db8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62840832->62844927] (req 62840832->62844927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.593839:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4db8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62840832->62844927] (req 62840832->62844927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.593974:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06400000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554729216:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7364us (8200us total) trans 4294982651 rc 0/0 00000100:00100000:0.0:1789760775.593985:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15353 00000100:00100000:0.0:1789760775.593994:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718818228456 time 718832205768 state 0x2 ref 0 00000100:00100000:3.0:1789760775.598962:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760775.599100:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554729344 00000100:00100000:1.0:1789760775.599115:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554729344 00000100:00100000:1.0:1789760775.599159:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718818228456 time 718832205768 state 0x0 ref 1 00000100:00100000:1.0:1789760775.599171:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718832466184 time 718832205768 state 0x1 ref 1 00000100:00100000:1.0:1789760775.599194:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718832466184 time 718844814246 state 0x0 ref 1 00000100:00100000:1.0:1789760775.599202:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06403100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554729344:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760775.599238:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62844928, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1df90 00010000:00010000:1.0:1789760775.599244:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4dbf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.599256:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760775.599260:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.599264:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4dbf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62844928->62849023] (req 62844928->62849023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.599274:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4dbf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62844928->62849023] (req 62844928->62849023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760775.605506:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982652 is committed 00002000:00010000:1.0:1789760775.605630:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62844928 -> 62849024 00002000:00010000:1.0:1789760775.605635:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122768 -> 122776 00010000:00010000:1.0:1789760775.605648:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4dbf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62844928->62849023] (req 62844928->62849023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.605661:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4dbf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62844928->62849023] (req 62844928->62849023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.605671:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4dbf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62844928->62849023] (req 62844928->62849023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.605690:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760775.605696:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4dbf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62844928->62849023] (req 62844928->62849023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.605706:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4dbf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62844928->62849023] (req 62844928->62849023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.605727:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4dbf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62844928->62849023] (req 62844928->62849023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.605745:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4dbf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62844928->62849023] (req 62844928->62849023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.605839:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06403100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554729344:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6631us (6881us total) trans 4294982652 rc 0/0 00000100:00100000:1.0:1789760775.605851:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15354 00000100:00100000:1.0:1789760775.605860:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718832466184 time 718844814246 state 0x2 ref 0 00000100:00100000:1.0:1789760775.612830:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760775.612932:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554729472 00000100:00100000:2.0:1789760775.612942:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554729472 00000100:00100000:2.0:1789760775.613001:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718832466184 time 718844814246 state 0x0 ref 1 00000100:00100000:2.0:1789760775.613020:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718845074662 time 718844814246 state 0x1 ref 1 00000100:00100000:2.0:1789760775.613091:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718845074662 time 718858708572 state 0x0 ref 1 00000100:00100000:2.0:1789760775.613115:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554729472:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760775.613165:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62849024, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1df98 00010000:00010000:2.0:1789760775.613175:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4dc6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.613191:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760775.613195:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.613202:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4dc6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62849024->62853119] (req 62849024->62853119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.613216:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4dc6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62849024->62853119] (req 62849024->62853119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.620235:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982653 is committed 00002000:00010000:2.0:1789760775.620427:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62849024 -> 62853120 00002000:00010000:2.0:1789760775.620433:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122776 -> 122784 00010000:00010000:2.0:1789760775.620446:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4dc6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62849024->62853119] (req 62849024->62853119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.620458:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4dc6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62849024->62853119] (req 62849024->62853119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.620466:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4dc6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62849024->62853119] (req 62849024->62853119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.620473:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760775.620477:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4dc6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62849024->62853119] (req 62849024->62853119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.620555:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4dc6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62849024->62853119] (req 62849024->62853119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.620577:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4dc6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62849024->62853119] (req 62849024->62853119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.620585:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c4dc6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62849024->62853119] (req 62849024->62853119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.620804:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554729472:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7697us (7978us total) trans 4294982653 rc 0/0 00000100:00100000:2.0:1789760775.620815:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15355 00000100:00100000:2.0:1789760775.620824:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718845074662 time 718858708572 state 0x2 ref 0 00000100:00100000:3.0:1789760775.624927:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760775.625771:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554729600 00000100:00100000:1.0:1789760775.625782:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554729600 00000100:00100000:1.0:1789760775.625836:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718845074662 time 718858708572 state 0x0 ref 1 00000100:00100000:1.0:1789760775.625850:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718858968988 time 718858708572 state 0x1 ref 1 00000100:00100000:1.0:1789760775.625924:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718858968988 time 718871541463 state 0x0 ref 1 00000100:00100000:1.0:1789760775.625936:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06400a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554729600:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760775.625983:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62853120, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1dfa0 00010000:00010000:1.0:1789760775.625991:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4dcd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.626007:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760775.626011:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.626018:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4dcd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62853120->62857215] (req 62853120->62857215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.626031:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4dcd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62853120->62857215] (req 62853120->62857215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.632111:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982654 is committed 00002000:00010000:1.0:1789760775.638932:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62853120 -> 62857216 00002000:00010000:1.0:1789760775.638941:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122784 -> 122792 00010000:00010000:1.0:1789760775.638954:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4dcd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62853120->62857215] (req 62853120->62857215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.638967:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4dcd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62853120->62857215] (req 62853120->62857215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.638976:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4dcd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62853120->62857215] (req 62853120->62857215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.638983:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760775.638988:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4dcd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62853120->62857215] (req 62853120->62857215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.638997:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4dcd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62853120->62857215] (req 62853120->62857215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.639016:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4dcd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62853120->62857215] (req 62853120->62857215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.639023:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c4dcd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62853120->62857215] (req 62853120->62857215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.639246:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06400a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554729600:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13307us (14324us total) trans 4294982654 rc 0/0 00000100:00100000:1.0:1789760775.639258:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15356 00000100:00100000:1.0:1789760775.639267:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718858968988 time 718871541463 state 0x2 ref 0 00000100:00100000:3.0:1789760775.648962:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760775.649836:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554729728 00000100:00100000:0.0:1789760775.649847:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554729728 00000100:00100000:0.0:1789760775.649901:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718858968988 time 718871541463 state 0x0 ref 1 00000100:00100000:0.0:1789760775.649916:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718871801879 time 718871541463 state 0x1 ref 1 00000100:00100000:0.0:1789760775.649985:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718871801879 time 718895603069 state 0x0 ref 1 00000100:00100000:0.0:1789760775.649999:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06402680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554729728:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760775.650050:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62857216, mtime: 0x6aad9506, atime: 0x0, ctime: 0x6aad9506, blocks: 0x1dfa8 00010000:00010000:0.0:1789760775.650060:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4dd4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.650077:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760775.650082:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.650088:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4dd4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62857216->62861311] (req 62857216->62861311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.650115:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4dd4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62857216->62861311] (req 62857216->62861311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.654568:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982655 is committed 00002000:00010000:0.0:1789760775.656910:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62857216 -> 62861312 00002000:00010000:0.0:1789760775.656919:0:8441:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760774 -> 1789760775 00002000:00010000:0.0:1789760775.656923:0:8441:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760774 -> 1789760775 00002000:00010000:0.0:1789760775.656925:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122792 -> 122800 00010000:00010000:0.0:1789760775.656943:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4dd4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62857216->62861311] (req 62857216->62861311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.656956:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4dd4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62857216->62861311] (req 62857216->62861311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.656965:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4dd4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62857216->62861311] (req 62857216->62861311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.656973:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760775.656978:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4dd4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62857216->62861311] (req 62857216->62861311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.656987:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4dd4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62857216->62861311] (req 62857216->62861311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.657006:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4dd4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62857216->62861311] (req 62857216->62861311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.657013:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4dd4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62857216->62861311] (req 62857216->62861311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.657234:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06402680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554729728:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7232us (8275us total) trans 4294982655 rc 0/0 00000100:00100000:0.0:1789760775.657245:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15357 00000100:00100000:0.0:1789760775.657255:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718871801879 time 718895603069 state 0x2 ref 0 00000100:00100000:2.0:1789760775.662769:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760775.663168:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554729856 00000100:00100000:0.0:1789760775.663176:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554729856 00000100:00100000:0.0:1789760775.663221:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718871801879 time 718895603069 state 0x0 ref 1 00000100:00100000:0.0:1789760775.663234:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718895863485 time 718895603069 state 0x1 ref 1 00000100:00100000:0.0:1789760775.663296:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718895863485 time 718908915373 state 0x0 ref 1 00000100:00100000:0.0:1789760775.663306:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e54380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554729856:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760775.663344:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62861312, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1dfb0 00010000:00010000:0.0:1789760775.663351:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4ddb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.663363:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760775.663367:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.663371:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4ddb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62861312->62865407] (req 62861312->62865407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.663407:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4ddb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62861312->62865407] (req 62861312->62865407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.668122:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982656 is committed 00002000:00010000:0.0:1789760775.668276:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62861312 -> 62865408 00002000:00010000:0.0:1789760775.668282:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122800 -> 122808 00010000:00010000:0.0:1789760775.668296:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4ddb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62861312->62865407] (req 62861312->62865407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.668310:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4ddb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62861312->62865407] (req 62861312->62865407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.668319:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4ddb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62861312->62865407] (req 62861312->62865407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.668326:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760775.668331:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4ddb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62861312->62865407] (req 62861312->62865407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.668339:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4ddb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62861312->62865407] (req 62861312->62865407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.668359:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4ddb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62861312->62865407] (req 62861312->62865407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.668367:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4ddb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62861312->62865407] (req 62861312->62865407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.668476:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554729856:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5166us (5713us total) trans 4294982656 rc 0/0 00000100:00100000:0.0:1789760775.668495:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15358 00000100:00100000:0.0:1789760775.668505:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718895863485 time 718908915373 state 0x2 ref 0 00000100:00100000:3.0:1789760775.673334:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760775.673713:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554729984 00000100:00100000:2.0:1789760775.673724:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554729984 00000100:00100000:2.0:1789760775.673824:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718895863485 time 718908915373 state 0x0 ref 1 00000100:00100000:2.0:1789760775.673839:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718909175789 time 718908915373 state 0x1 ref 1 00000100:00100000:2.0:1789760775.673901:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718909175789 time 718919518945 state 0x0 ref 1 00000100:00100000:2.0:1789760775.673916:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06401500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554729984:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760775.673960:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62865408, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1dfb8 00010000:00010000:2.0:1789760775.673968:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4de2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.673985:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:2.0:1789760775.673989:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.673994:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4de2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62865408->62869503] (req 62865408->62869503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.674008:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4de2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62865408->62869503] (req 62865408->62869503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.678054:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982657 is committed 00002000:00010000:2.0:1789760775.678220:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62865408 -> 62869504 00002000:00010000:2.0:1789760775.678226:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122808 -> 122816 00010000:00010000:2.0:1789760775.678242:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4de2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62865408->62869503] (req 62865408->62869503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.678256:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4de2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62865408->62869503] (req 62865408->62869503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.678266:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4de2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62865408->62869503] (req 62865408->62869503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.678274:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760775.678280:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4de2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62865408->62869503] (req 62865408->62869503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.678289:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4de2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62865408->62869503] (req 62865408->62869503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.678308:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4de2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62865408->62869503] (req 62865408->62869503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.678317:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4de2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62865408->62869503] (req 62865408->62869503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.678498:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06401500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554729984:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4571us (5159us total) trans 4294982657 rc 0/0 00000100:00100000:2.0:1789760775.678509:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15359 00000100:00100000:2.0:1789760775.678519:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718909175789 time 718919518945 state 0x2 ref 0 00000100:00100000:2.0:1789760775.682954:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760775.683066:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554730112 00000100:00100000:1.0:1789760775.683075:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554730112 00000100:00100000:1.0:1789760775.683196:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718909175789 time 718919518945 state 0x0 ref 1 00000100:00100000:1.0:1789760775.683212:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718919779361 time 718919518945 state 0x1 ref 1 00000100:00100000:1.0:1789760775.683285:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718919779361 time 718928902728 state 0x0 ref 1 00000100:00100000:1.0:1789760775.683299:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e57100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554730112:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760775.683344:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62869504, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1dfc0 00010000:00010000:1.0:1789760775.683354:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4de9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.683371:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760775.683404:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.683412:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4de9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62869504->62873599] (req 62869504->62873599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.683426:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4de9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62869504->62873599] (req 62869504->62873599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.687818:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982658 is committed 00002000:00010000:1.0:1789760775.687956:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62869504 -> 62873600 00002000:00010000:1.0:1789760775.687961:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122816 -> 122824 00010000:00010000:1.0:1789760775.687974:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4de9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62869504->62873599] (req 62869504->62873599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.687986:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4de9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62869504->62873599] (req 62869504->62873599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.687995:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4de9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62869504->62873599] (req 62869504->62873599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.688001:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760775.688006:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4de9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62869504->62873599] (req 62869504->62873599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.688014:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4de9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62869504->62873599] (req 62869504->62873599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.688031:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4de9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62869504->62873599] (req 62869504->62873599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.688037:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4de9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62869504->62873599] (req 62869504->62873599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.688130:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e57100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554730112:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4830us (5181us total) trans 4294982658 rc 0/0 00000100:00100000:1.0:1789760775.688140:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15360 00000100:00100000:1.0:1789760775.688149:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718919779361 time 718928902728 state 0x2 ref 0 00000100:00100000:1.0:1789760775.691635:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760775.691725:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554730240 00000100:00100000:0.0:1789760775.691746:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554730240 00000100:00100000:0.0:1789760775.691801:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718919779361 time 718928902728 state 0x0 ref 1 00000100:00100000:0.0:1789760775.691817:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718929163144 time 718928902728 state 0x1 ref 1 00000100:00100000:0.0:1789760775.691853:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718929163144 time 718937470680 state 0x0 ref 1 00000100:00100000:0.0:1789760775.691866:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d80700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554730240:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760775.691914:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62873600, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1dfc8 00010000:00010000:0.0:1789760775.691923:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4df0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.691940:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760775.691945:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.691951:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4df0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62873600->62877695] (req 62873600->62877695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.691964:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4df0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62873600->62877695] (req 62873600->62877695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.697921:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982659 is committed 00002000:00010000:0.0:1789760775.698150:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62873600 -> 62877696 00002000:00010000:0.0:1789760775.698157:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122824 -> 122832 00010000:00010000:0.0:1789760775.698172:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4df0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62873600->62877695] (req 62873600->62877695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.698196:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4df0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62873600->62877695] (req 62873600->62877695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.698211:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4df0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62873600->62877695] (req 62873600->62877695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.698220:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760775.698225:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4df0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62873600->62877695] (req 62873600->62877695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.698234:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4df0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62873600->62877695] (req 62873600->62877695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.698254:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4df0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62873600->62877695] (req 62873600->62877695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.698268:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4df0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62873600->62877695] (req 62873600->62877695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.699881:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d80700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554730240:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6492us (6730us total) trans 4294982659 rc 0/0 00000100:00100000:0.0:1789760775.699901:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15361 00000100:00100000:0.0:1789760775.699911:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718929163144 time 718937470680 state 0x2 ref 0 00000100:00100000:3.0:1789760775.706683:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760775.706946:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554730368 00000100:00100000:0.0:1789760775.706956:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554730368 00000100:00100000:0.0:1789760775.707010:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718929163144 time 718937470680 state 0x0 ref 1 00000100:00100000:0.0:1789760775.707026:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718937731096 time 718937470680 state 0x1 ref 1 00000100:00100000:0.0:1789760775.707057:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718937731096 time 718952675966 state 0x0 ref 1 00000100:00100000:0.0:1789760775.707070:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06401880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554730368:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760775.707127:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62877696, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1dfd0 00010000:00010000:0.0:1789760775.707138:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4df7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.707152:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760775.707157:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.707162:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4df7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62877696->62881791] (req 62877696->62881791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.707179:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4df7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62877696->62881791] (req 62877696->62881791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.711361:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982660 is committed 00002000:00010000:0.0:1789760775.711491:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62877696 -> 62881792 00002000:00010000:0.0:1789760775.711496:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122832 -> 122840 00010000:00010000:0.0:1789760775.711509:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4df7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62877696->62881791] (req 62877696->62881791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.711521:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4df7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62877696->62881791] (req 62877696->62881791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.711530:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4df7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62877696->62881791] (req 62877696->62881791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.711537:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760775.711542:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4df7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62877696->62881791] (req 62877696->62881791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.711551:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4df7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62877696->62881791] (req 62877696->62881791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.711571:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4df7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62877696->62881791] (req 62877696->62881791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.711579:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4df7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62877696->62881791] (req 62877696->62881791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.711913:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06401880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554730368:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4839us (5233us total) trans 4294982660 rc 0/0 00000100:00100000:0.0:1789760775.711924:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15362 00000100:00100000:0.0:1789760775.711934:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718937731096 time 718952675966 state 0x2 ref 0 00000100:00100000:2.0:1789760775.716858:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760775.716928:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554730496 00000100:00100000:1.0:1789760775.716938:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554730496 00000100:00100000:1.0:1789760775.716988:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718937731096 time 718952675966 state 0x0 ref 1 00000100:00100000:1.0:1789760775.717003:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718952936382 time 718952675966 state 0x1 ref 1 00000100:00100000:1.0:1789760775.717033:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718952936382 time 718962651861 state 0x0 ref 1 00000100:00100000:1.0:1789760775.717044:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554730496:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760775.717088:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62881792, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1dfd8 00010000:00010000:1.0:1789760775.717097:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4dfe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.717123:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:1.0:1789760775.717127:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.717133:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4dfe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62881792->62885887] (req 62881792->62885887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.717147:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4dfe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62881792->62885887] (req 62881792->62885887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.722752:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982661 is committed 00002000:00010000:2.0:1789760775.722883:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62881792 -> 62885888 00002000:00010000:2.0:1789760775.722888:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122840 -> 122848 00010000:00010000:2.0:1789760775.722901:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4dfe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62881792->62885887] (req 62881792->62885887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.722913:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4dfe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62881792->62885887] (req 62881792->62885887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.722926:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4dfe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62881792->62885887] (req 62881792->62885887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.722933:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760775.722938:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4dfe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62881792->62885887] (req 62881792->62885887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.722947:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4dfe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62881792->62885887] (req 62881792->62885887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.722967:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4dfe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62881792->62885887] (req 62881792->62885887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.722974:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c4dfe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62881792->62885887] (req 62881792->62885887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.723062:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554730496:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6014us (6209us total) trans 4294982661 rc 0/0 00000100:00100000:2.0:1789760775.723073:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15363 00000100:00100000:2.0:1789760775.723083:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718952936382 time 718962651861 state 0x2 ref 0 00000100:00100000:1.0:1789760775.729540:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760775.729935:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554730624 00000100:00100000:3.0:1789760775.729944:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554730624 00000100:00100000:3.0:1789760775.729999:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718952936382 time 718962651861 state 0x0 ref 1 00000100:00100000:3.0:1789760775.730015:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718962912277 time 718962651861 state 0x1 ref 1 00000100:00100000:3.0:1789760775.730048:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718962912277 time 718975666930 state 0x0 ref 1 00000100:00100000:3.0:1789760775.730061:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d80380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554730624:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760775.730104:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62885888, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1dfe0 00010000:00010000:3.0:1789760775.730184:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4e05 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.730199:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:3.0:1789760775.730454:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.730462:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4e05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62885888->62889983] (req 62885888->62889983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.730477:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4e05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62885888->62889983] (req 62885888->62889983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.735699:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982662 is committed 00002000:00010000:3.0:1789760775.736078:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62885888 -> 62889984 00002000:00010000:3.0:1789760775.736083:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122848 -> 122856 00010000:00010000:3.0:1789760775.736098:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4e05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62885888->62889983] (req 62885888->62889983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.736168:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4e05 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62885888->62889983] (req 62885888->62889983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.736178:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4e05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62885888->62889983] (req 62885888->62889983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.736189:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760775.736195:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4e05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62885888->62889983] (req 62885888->62889983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.736203:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4e05 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62885888->62889983] (req 62885888->62889983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.736222:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4e05 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62885888->62889983] (req 62885888->62889983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.736228:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4e05 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62885888->62889983] (req 62885888->62889983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.736457:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d80380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554730624:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6393us (6920us total) trans 4294982662 rc 0/0 00000100:00100000:3.0:1789760775.736468:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15364 00000100:00100000:3.0:1789760775.736477:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718962912277 time 718975666930 state 0x2 ref 0 00000100:00100000:3.0:1789760775.740207:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760775.740317:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554730752 00000100:00100000:0.0:1789760775.740327:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554730752 00000100:00100000:0.0:1789760775.741596:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718962912277 time 718975666930 state 0x0 ref 1 00000100:00100000:0.0:1789760775.741620:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718975927346 time 718975666930 state 0x1 ref 1 00000100:00100000:0.0:1789760775.741661:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718975927346 time 718987278933 state 0x0 ref 1 00000100:00100000:0.0:1789760775.741677:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06401f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554730752:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760775.741736:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62889984, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1dfe8 00010000:00010000:0.0:1789760775.741746:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e0c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.741763:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760775.741767:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.741773:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62889984->62894079] (req 62889984->62894079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.741786:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62889984->62894079] (req 62889984->62894079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.748307:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982663 is committed 00002000:00010000:0.0:1789760775.748434:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62889984 -> 62894080 00002000:00010000:0.0:1789760775.748440:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122856 -> 122864 00010000:00010000:0.0:1789760775.748453:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62889984->62894079] (req 62889984->62894079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.748467:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e0c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62889984->62894079] (req 62889984->62894079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.748477:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62889984->62894079] (req 62889984->62894079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.748495:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760775.748500:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62889984->62894079] (req 62889984->62894079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.748509:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e0c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62889984->62894079] (req 62889984->62894079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.748530:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e0c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62889984->62894079] (req 62889984->62894079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.748538:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e0c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62889984->62894079] (req 62889984->62894079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.748631:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06401f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554730752:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6955us (8428us total) trans 4294982663 rc 0/0 00000100:00100000:0.0:1789760775.748643:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15365 00000100:00100000:0.0:1789760775.749253:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718975927346 time 718987278933 state 0x2 ref 0 00000100:00100000:2.0:1789760775.752907:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760775.752993:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554730880 00000100:00100000:0.0:1789760775.753002:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554730880 00000100:00100000:0.0:1789760775.753051:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718975927346 time 718987278933 state 0x0 ref 1 00000100:00100000:0.0:1789760775.753065:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718987539349 time 718987278933 state 0x1 ref 1 00000100:00100000:0.0:1789760775.753095:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718987539349 time 718998714056 state 0x0 ref 1 00000100:00100000:0.0:1789760775.753215:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e54e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554730880:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760775.753260:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62894080, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1dff0 00010000:00010000:0.0:1789760775.753268:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4e13 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.753283:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=718s v=(0 0) 00010000:00010000:0.0:1789760775.753287:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.753293:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4e13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62894080->62898175] (req 62894080->62898175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.753307:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4e13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62894080->62898175] (req 62894080->62898175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.761113:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982664 is committed 00010000:00010000:2.0:1789760775.761562:0:6509:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:0.0:1789760775.761701:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62894080 -> 62898176 00002000:00010000:0.0:1789760775.761706:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122864 -> 122872 00010000:00010000:0.0:1789760775.761799:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4e13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62894080->62898175] (req 62894080->62898175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.761811:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4e13 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62894080->62898175] (req 62894080->62898175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.761819:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4e13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62894080->62898175] (req 62894080->62898175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.761827:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760775.761831:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4e13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62894080->62898175] (req 62894080->62898175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.761841:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4e13 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62894080->62898175] (req 62894080->62898175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.761860:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4e13 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62894080->62898175] (req 62894080->62898175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.761868:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c4e13 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62894080->62898175] (req 62894080->62898175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.761959:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554730880:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8849us (9056us total) trans 4294982664 rc 0/0 00000100:00100000:0.0:1789760775.761970:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15366 00000100:00100000:0.0:1789760775.761979:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718987539349 time 718998714056 state 0x2 ref 0 00000100:00100000:1.0:1789760775.766295:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760775.766431:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554731008 00000100:00100000:2.0:1789760775.766442:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554731008 00000100:00100000:2.0:1789760775.766494:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718987539349 time 718998714056 state 0x0 ref 1 00000100:00100000:2.0:1789760775.766511:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718998974472 time 718998714056 state 0x1 ref 1 00000100:00100000:2.0:1789760775.766533:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718998974472 time 719012152314 state 0x0 ref 1 00000100:00100000:2.0:1789760775.766545:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554731008:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760775.766594:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62898176, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1dff8 00010000:00010000:2.0:1789760775.766604:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e1a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.766621:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760775.766625:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.766631:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62898176->62902271] (req 62898176->62902271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.766645:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62898176->62902271] (req 62898176->62902271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.772060:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982665 is committed 00002000:00010000:2.0:1789760775.772469:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62898176 -> 62902272 00002000:00010000:2.0:1789760775.772476:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122872 -> 122880 00010000:00010000:2.0:1789760775.772490:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62898176->62902271] (req 62898176->62902271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.772502:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e1a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62898176->62902271] (req 62898176->62902271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.772510:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62898176->62902271] (req 62898176->62902271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.772518:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760775.772523:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62898176->62902271] (req 62898176->62902271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.772530:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e1a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62898176->62902271] (req 62898176->62902271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.772549:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e1a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62898176->62902271] (req 62898176->62902271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.772556:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e1a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62898176->62902271] (req 62898176->62902271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.772715:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554731008:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6167us (6425us total) trans 4294982665 rc 0/0 00000100:00100000:2.0:1789760775.772725:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15367 00000100:00100000:2.0:1789760775.772746:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718998974472 time 719012152314 state 0x2 ref 0 00000100:00100000:3.0:1789760775.777830:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760775.777905:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554731136 00000100:00100000:1.0:1789760775.777915:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554731136 00000100:00100000:1.0:1789760775.777967:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 718998974472 time 719012152314 state 0x0 ref 1 00000100:00100000:1.0:1789760775.777984:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719012412730 time 719012152314 state 0x1 ref 1 00000100:00100000:1.0:1789760775.778018:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719012412730 time 719023636953 state 0x0 ref 1 00000100:00100000:1.0:1789760775.778031:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06403480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554731136:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760775.778076:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62902272, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e000 00010000:00010000:1.0:1789760775.778085:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4e21 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.778101:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760775.778117:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.778124:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4e21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62902272->62906367] (req 62902272->62906367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.778138:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4e21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62902272->62906367] (req 62902272->62906367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.782771:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982666 is committed 00002000:00010000:0.0:1789760775.783758:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62902272 -> 62906368 00002000:00010000:0.0:1789760775.783765:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122880 -> 122888 00010000:00010000:0.0:1789760775.783779:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4e21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62902272->62906367] (req 62902272->62906367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.783793:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4e21 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62902272->62906367] (req 62902272->62906367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.783802:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4e21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62902272->62906367] (req 62902272->62906367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.783811:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760775.783816:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4e21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62902272->62906367] (req 62902272->62906367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.783824:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4e21 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62902272->62906367] (req 62902272->62906367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.783844:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4e21 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62902272->62906367] (req 62902272->62906367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.783852:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c4e21 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62902272->62906367] (req 62902272->62906367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.783951:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06403480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554731136:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5917us (6126us total) trans 4294982666 rc 0/0 00000100:00100000:0.0:1789760775.783966:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15368 00000100:00100000:0.0:1789760775.783976:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719012412730 time 719023636953 state 0x2 ref 0 00000100:00100000:0.0:1789760775.788100:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760775.788629:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554731264 00000100:00100000:3.0:1789760775.788640:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554731264 00000100:00100000:3.0:1789760775.788699:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719012412730 time 719023636953 state 0x0 ref 1 00000100:00100000:3.0:1789760775.788715:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719023897369 time 719023636953 state 0x1 ref 1 00000100:00100000:3.0:1789760775.788761:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719023897369 time 719034378575 state 0x0 ref 1 00000100:00100000:3.0:1789760775.788775:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554731264:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760775.788826:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62906368, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e008 00010000:00010000:3.0:1789760775.788837:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4e28 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.788855:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760775.788860:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.788867:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4e28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62906368->62910463] (req 62906368->62910463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.788883:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4e28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62906368->62910463] (req 62906368->62910463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.797723:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982667 is committed 00002000:00010000:2.0:1789760775.799624:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62906368 -> 62910464 00002000:00010000:2.0:1789760775.799633:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122888 -> 122896 00010000:00010000:2.0:1789760775.799646:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4e28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62906368->62910463] (req 62906368->62910463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.799658:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4e28 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62906368->62910463] (req 62906368->62910463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.799672:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4e28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62906368->62910463] (req 62906368->62910463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.799679:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760775.799684:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4e28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62906368->62910463] (req 62906368->62910463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.799695:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4e28 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62906368->62910463] (req 62906368->62910463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.799714:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4e28 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62906368->62910463] (req 62906368->62910463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.799721:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c4e28 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62906368->62910463] (req 62906368->62910463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.799827:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554731264:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11050us (11733us total) trans 4294982667 rc 0/0 00000100:00100000:2.0:1789760775.799838:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15369 00000100:00100000:2.0:1789760775.799848:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719023897369 time 719034378575 state 0x2 ref 0 00000100:00100000:1.0:1789760775.805840:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760775.808770:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554731392 00000100:00100000:2.0:1789760775.808785:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554731392 00000100:00100000:2.0:1789760775.808847:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719023897369 time 719034378575 state 0x0 ref 1 00000100:00100000:2.0:1789760775.808862:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719034638991 time 719034378575 state 0x1 ref 1 00000100:00100000:2.0:1789760775.808887:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719034638991 time 719054506246 state 0x0 ref 1 00000100:00100000:2.0:1789760775.808899:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554731392:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760775.808946:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62910464, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e010 00010000:00010000:2.0:1789760775.808955:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e2f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.808970:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760775.808974:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.808980:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62910464->62914559] (req 62910464->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.808995:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62910464->62914559] (req 62910464->62914559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.813927:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982668 is committed 00002000:00010000:2.0:1789760775.814057:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62910464 -> 62914560 00002000:00010000:2.0:1789760775.814062:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122896 -> 122904 00010000:00010000:2.0:1789760775.814076:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62910464->62914559] (req 62910464->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.814089:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e2f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62910464->62914559] (req 62910464->62914559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.814098:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62910464->62914559] (req 62910464->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.814122:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760775.814126:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62910464->62914559] (req 62910464->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.814135:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e2f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62910464->62914559] (req 62910464->62914559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.814156:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e2f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62910464->62914559] (req 62910464->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.814163:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e2f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62910464->62914559] (req 62910464->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.814253:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554731392:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5351us (8418us total) trans 4294982668 rc 0/0 00000100:00100000:2.0:1789760775.814262:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15370 00000100:00100000:2.0:1789760775.814271:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719034638991 time 719054506246 state 0x2 ref 0 00000100:00100000:3.0:1789760775.817990:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760775.818080:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554731520 00000100:00100000:2.0:1789760775.818089:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554731520 00000100:00100000:2.0:1789760775.818142:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719034638991 time 719054506246 state 0x0 ref 1 00000100:00100000:2.0:1789760775.818158:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719054766662 time 719054506246 state 0x1 ref 1 00000100:00100000:2.0:1789760775.818194:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719054766662 time 719063812532 state 0x0 ref 1 00000100:00100000:2.0:1789760775.818211:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06402a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554731520:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760775.818254:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62914560, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e018 00010000:00010000:2.0:1789760775.818263:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4e36 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.818278:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760775.818282:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.818289:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4e36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62914560->62918655] (req 62914560->62918655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.818302:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4e36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62914560->62918655] (req 62914560->62918655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.824465:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982669 is committed 00002000:00010000:2.0:1789760775.824884:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62914560 -> 62918656 00002000:00010000:2.0:1789760775.824890:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122904 -> 122912 00010000:00010000:2.0:1789760775.824903:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4e36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62914560->62918655] (req 62914560->62918655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.824916:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4e36 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62914560->62918655] (req 62914560->62918655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.824924:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4e36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62914560->62918655] (req 62914560->62918655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.824931:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760775.824936:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4e36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62914560->62918655] (req 62914560->62918655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.824946:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4e36 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62914560->62918655] (req 62914560->62918655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.824964:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4e36 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62914560->62918655] (req 62914560->62918655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.824971:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4e36 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62914560->62918655] (req 62914560->62918655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.825072:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06402a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554731520:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6857us (7085us total) trans 4294982669 rc 0/0 00000100:00100000:2.0:1789760775.825084:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15371 00000100:00100000:2.0:1789760775.825093:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719054766662 time 719063812532 state 0x2 ref 0 00000100:00100000:0.0:1789760775.830623:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760775.830845:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554731648 00000100:00100000:1.0:1789760775.830855:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554731648 00000100:00100000:1.0:1789760775.830912:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719054766662 time 719063812532 state 0x0 ref 1 00000100:00100000:1.0:1789760775.830930:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719064072948 time 719063812532 state 0x1 ref 1 00000100:00100000:1.0:1789760775.830962:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719064072948 time 719076580332 state 0x0 ref 1 00000100:00100000:1.0:1789760775.830976:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554731648:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760775.831027:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62918656, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e020 00010000:00010000:1.0:1789760775.831036:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c4e3d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.831052:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760775.831057:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.831063:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c4e3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62918656->62922751] (req 62918656->62922751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.831076:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c4e3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62918656->62922751] (req 62918656->62922751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760775.839434:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982670 is committed 00002000:00010000:1.0:1789760775.839539:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62918656 -> 62922752 00002000:00010000:1.0:1789760775.839544:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122912 -> 122920 00010000:00010000:1.0:1789760775.839558:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c4e3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62918656->62922751] (req 62918656->62922751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.839572:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c4e3d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62918656->62922751] (req 62918656->62922751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.839583:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c4e3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62918656->62922751] (req 62918656->62922751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.839592:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655de00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760775.839597:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c4e3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62918656->62922751] (req 62918656->62922751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.839607:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c4e3d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62918656->62922751] (req 62918656->62922751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.839626:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c4e3d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62918656->62922751] (req 62918656->62922751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.839635:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c4e3d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62918656->62922751] (req 62918656->62922751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.839780:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554731648:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8801us (9163us total) trans 4294982670 rc 0/0 00000100:00100000:1.0:1789760775.839791:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15372 00000100:00100000:1.0:1789760775.839800:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719064072948 time 719076580332 state 0x2 ref 0 00000100:00100000:1.0:1789760775.845712:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760775.846089:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554731776 00000100:00100000:0.0:1789760775.846098:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554731776 00000100:00100000:0.0:1789760775.846148:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719064072948 time 719076580332 state 0x0 ref 1 00000100:00100000:0.0:1789760775.846163:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719076840748 time 719076580332 state 0x1 ref 1 00000100:00100000:0.0:1789760775.846196:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719076840748 time 719091814617 state 0x0 ref 1 00000100:00100000:0.0:1789760775.846208:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d81c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554731776:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760775.846254:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62922752, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e028 00010000:00010000:0.0:1789760775.846262:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e44 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.846277:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760775.846281:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.846287:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62922752->62926847] (req 62922752->62926847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.846301:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62922752->62926847] (req 62922752->62926847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.852474:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982671 is committed 00002000:00010000:0.0:1789760775.852612:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62922752 -> 62926848 00002000:00010000:0.0:1789760775.852618:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122920 -> 122928 00010000:00010000:0.0:1789760775.852631:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62922752->62926847] (req 62922752->62926847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.852643:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e44 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62922752->62926847] (req 62922752->62926847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.852652:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62922752->62926847] (req 62922752->62926847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.852660:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760775.852664:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62922752->62926847] (req 62922752->62926847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.852672:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e44 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62922752->62926847] (req 62922752->62926847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.852691:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e44 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62922752->62926847] (req 62922752->62926847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.852699:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e44 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62922752->62926847] (req 62922752->62926847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.852806:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d81c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554731776:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6595us (7100us total) trans 4294982671 rc 0/0 00000100:00100000:0.0:1789760775.852819:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15373 00000100:00100000:0.0:1789760775.852828:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719076840748 time 719091814617 state 0x2 ref 0 00000100:00100000:3.0:1789760775.856813:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760775.856930:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554731904 00000100:00100000:2.0:1789760775.856938:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554731904 00000100:00100000:2.0:1789760775.856990:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719076840748 time 719091814617 state 0x0 ref 1 00000100:00100000:2.0:1789760775.857004:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719092075033 time 719091814617 state 0x1 ref 1 00000100:00100000:2.0:1789760775.857037:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719092075033 time 719102656051 state 0x0 ref 1 00000100:00100000:2.0:1789760775.857049:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06400e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554731904:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760775.857093:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62926848, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e030 00010000:00010000:2.0:1789760775.857112:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4e4b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.857124:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760775.857127:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.857131:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4e4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62926848->62930943] (req 62926848->62930943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.857142:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4e4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62926848->62930943] (req 62926848->62930943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.862537:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982672 is committed 00002000:00010000:2.0:1789760775.862661:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62926848 -> 62930944 00002000:00010000:2.0:1789760775.862666:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122928 -> 122936 00010000:00010000:2.0:1789760775.862681:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4e4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62926848->62930943] (req 62926848->62930943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.862694:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4e4b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62926848->62930943] (req 62926848->62930943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.862703:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4e4b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62926848->62930943] (req 62926848->62930943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.862712:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760775.862717:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4e4b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62926848->62930943] (req 62926848->62930943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.862737:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4e4b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62926848->62930943] (req 62926848->62930943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.862758:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4e4b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62926848->62930943] (req 62926848->62930943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.862766:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4e4b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62926848->62930943] (req 62926848->62930943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.862898:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06400e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554731904:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5846us (6090us total) trans 4294982672 rc 0/0 00000100:00100000:2.0:1789760775.862908:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15374 00000100:00100000:2.0:1789760775.862918:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719092075033 time 719102656051 state 0x2 ref 0 00000100:00100000:2.0:1789760775.869331:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760775.869538:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554732032 00000100:00100000:1.0:1789760775.869552:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554732032 00000100:00100000:1.0:1789760775.869623:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719092075033 time 719102656051 state 0x0 ref 1 00000100:00100000:1.0:1789760775.869642:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719102916467 time 719102656051 state 0x1 ref 1 00000100:00100000:1.0:1789760775.869689:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719102916467 time 719115305348 state 0x0 ref 1 00000100:00100000:1.0:1789760775.869707:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554732032:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760775.869780:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62930944, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e038 00010000:00010000:1.0:1789760775.869793:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4e52 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.869812:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760775.869817:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.869823:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4e52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62930944->62935039] (req 62930944->62935039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.869837:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4e52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62930944->62935039] (req 62930944->62935039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.876042:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982673 is committed 00002000:00010000:2.0:1789760775.876203:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62930944 -> 62935040 00002000:00010000:2.0:1789760775.876208:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122936 -> 122944 00010000:00010000:2.0:1789760775.876221:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4e52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62930944->62935039] (req 62930944->62935039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.876233:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4e52 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62930944->62935039] (req 62930944->62935039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.876242:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4e52 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62930944->62935039] (req 62930944->62935039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.876249:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760775.876254:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4e52 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62930944->62935039] (req 62930944->62935039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.876262:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4e52 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62930944->62935039] (req 62930944->62935039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.876281:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4e52 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62930944->62935039] (req 62930944->62935039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.876289:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c4e52 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62930944->62935039] (req 62930944->62935039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.876467:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554732032:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6671us (7055us total) trans 4294982673 rc 0/0 00000100:00100000:2.0:1789760775.876478:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15375 00000100:00100000:2.0:1789760775.876499:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719102916467 time 719115305348 state 0x2 ref 0 00000100:00100000:1.0:1789760775.882334:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760775.882473:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554732160 00000100:00100000:0.0:1789760775.882484:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554732160 00000100:00100000:0.0:1789760775.886647:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719102916467 time 719115305348 state 0x0 ref 1 00000100:00100000:0.0:1789760775.886688:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719115565764 time 719115305348 state 0x1 ref 1 00000100:00100000:0.0:1789760775.887632:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719115565764 time 719133244106 state 0x0 ref 1 00000100:00100000:0.0:1789760775.887660:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d81880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554732160:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760775.887712:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62935040, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e040 00010000:00010000:0.0:1789760775.887720:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e59 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.887746:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760775.887750:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.887757:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62935040->62939135] (req 62935040->62939135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.887771:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62935040->62939135] (req 62935040->62939135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.892013:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982674 is committed 00002000:00010000:0.0:1789760775.892298:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62935040 -> 62939136 00002000:00010000:0.0:1789760775.892303:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122944 -> 122952 00010000:00010000:0.0:1789760775.892317:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62935040->62939135] (req 62935040->62939135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.892330:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e59 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62935040->62939135] (req 62935040->62939135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.892338:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62935040->62939135] (req 62935040->62939135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.892345:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760775.892350:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62935040->62939135] (req 62935040->62939135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.892360:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e59 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62935040->62939135] (req 62935040->62939135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.892620:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e59 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62935040->62939135] (req 62935040->62939135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.892630:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c4e59 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62935040->62939135] (req 62935040->62939135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.892738:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d81880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554732160:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5078us (10402us total) trans 4294982674 rc 0/0 00000100:00100000:0.0:1789760775.892750:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15376 00000100:00100000:0.0:1789760775.892760:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719115565764 time 719133244106 state 0x2 ref 0 00000100:00100000:3.0:1789760775.898817:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760775.899407:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554732288 00000100:00100000:0.0:1789760775.899417:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554732288 00000100:00100000:0.0:1789760775.899476:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719115565764 time 719133244106 state 0x0 ref 1 00000100:00100000:0.0:1789760775.899502:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719133504522 time 719133244106 state 0x1 ref 1 00000100:00100000:0.0:1789760775.899533:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719133504522 time 719145151779 state 0x0 ref 1 00000100:00100000:0.0:1789760775.899547:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06403b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554732288:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760775.899594:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62939136, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e048 00010000:00010000:0.0:1789760775.899603:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4e60 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.899618:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760775.899623:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.899629:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4e60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62939136->62943231] (req 62939136->62943231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.899643:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4e60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62939136->62943231] (req 62939136->62943231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.904227:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982675 is committed 00002000:00010000:0.0:1789760775.904353:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62939136 -> 62943232 00002000:00010000:0.0:1789760775.904360:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122952 -> 122960 00010000:00010000:0.0:1789760775.904372:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4e60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62939136->62943231] (req 62939136->62943231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.904411:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4e60 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62939136->62943231] (req 62939136->62943231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.904621:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4e60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62939136->62943231] (req 62939136->62943231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.904630:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760775.904635:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4e60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62939136->62943231] (req 62939136->62943231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.904644:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4e60 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62939136->62943231] (req 62939136->62943231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.904664:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4e60 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62939136->62943231] (req 62939136->62943231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.904672:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4e60 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62939136->62943231] (req 62939136->62943231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.904844:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06403b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554732288:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5295us (6036us total) trans 4294982675 rc 0/0 00000100:00100000:0.0:1789760775.904854:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15377 00000100:00100000:0.0:1789760775.904863:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719133504522 time 719145151779 state 0x2 ref 0 00000100:00100000:2.0:1789760775.908513:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760775.908754:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554732416 00000100:00100000:1.0:1789760775.908764:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554732416 00000100:00100000:1.0:1789760775.908814:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719133504522 time 719145151779 state 0x0 ref 1 00000100:00100000:1.0:1789760775.908828:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719145412195 time 719145151779 state 0x1 ref 1 00000100:00100000:1.0:1789760775.908859:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719145412195 time 719154478666 state 0x0 ref 1 00000100:00100000:1.0:1789760775.908871:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e56d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554732416:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760775.908917:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62943232, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e050 00010000:00010000:1.0:1789760775.908926:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4e67 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.908942:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760775.908946:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.908953:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4e67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62943232->62947327] (req 62943232->62947327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.908967:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4e67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62943232->62947327] (req 62943232->62947327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.914682:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982676 is committed 00002000:00010000:2.0:1789760775.915239:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62943232 -> 62947328 00002000:00010000:2.0:1789760775.915246:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122960 -> 122968 00010000:00010000:2.0:1789760775.915260:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4e67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62943232->62947327] (req 62943232->62947327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.915273:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4e67 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62943232->62947327] (req 62943232->62947327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.915283:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4e67 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62943232->62947327] (req 62943232->62947327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.915290:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760775.915296:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4e67 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62943232->62947327] (req 62943232->62947327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.915305:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4e67 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62943232->62947327] (req 62943232->62947327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.915323:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4e67 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62943232->62947327] (req 62943232->62947327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.915331:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4e67 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62943232->62947327] (req 62943232->62947327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.915676:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e56d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554732416:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6801us (7305us total) trans 4294982676 rc 0/0 00000100:00100000:2.0:1789760775.915688:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15378 00000100:00100000:2.0:1789760775.915699:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719145412195 time 719154478666 state 0x2 ref 0 00000100:00100000:1.0:1789760775.920766:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760775.920862:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554732544 00000100:00100000:3.0:1789760775.920872:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554732544 00000100:00100000:3.0:1789760775.920929:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719145412195 time 719154478666 state 0x0 ref 1 00000100:00100000:3.0:1789760775.920945:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719154739082 time 719154478666 state 0x1 ref 1 00000100:00100000:3.0:1789760775.920977:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719154739082 time 719166595562 state 0x0 ref 1 00000100:00100000:3.0:1789760775.920990:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554732544:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760775.921035:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62947328, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e058 00010000:00010000:3.0:1789760775.921045:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4e6e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.921063:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760775.921068:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760775.921074:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4e6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62947328->62951423] (req 62947328->62951423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.921089:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4e6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62947328->62951423] (req 62947328->62951423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.926208:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982677 is committed 00002000:00010000:3.0:1789760775.926325:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62947328 -> 62951424 00002000:00010000:3.0:1789760775.926331:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122968 -> 122976 00010000:00010000:3.0:1789760775.926346:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4e6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62947328->62951423] (req 62947328->62951423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.926360:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4e6e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62947328->62951423] (req 62947328->62951423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.926369:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4e6e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62947328->62951423] (req 62947328->62951423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.926428:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760775.926480:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4e6e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62947328->62951423] (req 62947328->62951423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.926498:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4e6e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62947328->62951423] (req 62947328->62951423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.926518:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4e6e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62947328->62951423] (req 62947328->62951423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760775.926525:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4e6e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62947328->62951423] (req 62947328->62951423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760775.926629:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554732544:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5635us (5866us total) trans 4294982677 rc 0/0 00000100:00100000:3.0:1789760775.926640:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15379 00000100:00100000:3.0:1789760775.926650:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719154739082 time 719166595562 state 0x2 ref 0 00000100:00100000:3.0:1789760775.930229:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760775.930318:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554732672 00000100:00100000:1.0:1789760775.930327:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554732672 00000100:00100000:1.0:1789760775.930431:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719154739082 time 719166595562 state 0x0 ref 1 00000100:00100000:1.0:1789760775.930447:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719166855978 time 719166595562 state 0x1 ref 1 00000100:00100000:1.0:1789760775.930473:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719166855978 time 719176091545 state 0x0 ref 1 00000100:00100000:1.0:1789760775.930497:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06402300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554732672:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760775.930543:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62951424, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e060 00010000:00010000:1.0:1789760775.930552:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4e75 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.930568:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760775.930572:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.930578:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4e75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62951424->62955519] (req 62951424->62955519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.930592:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4e75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62951424->62955519] (req 62951424->62955519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.935360:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982678 is committed 00002000:00010000:0.0:1789760775.935858:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62951424 -> 62955520 00002000:00010000:0.0:1789760775.935865:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122976 -> 122984 00010000:00010000:0.0:1789760775.935881:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4e75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62951424->62955519] (req 62951424->62955519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.936024:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4e75 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62951424->62955519] (req 62951424->62955519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.936038:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4e75 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62951424->62955519] (req 62951424->62955519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.936046:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760775.936051:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4e75 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62951424->62955519] (req 62951424->62955519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.936061:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4e75 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62951424->62955519] (req 62951424->62955519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.936082:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4e75 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62951424->62955519] (req 62951424->62955519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.936091:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4e75 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62951424->62955519] (req 62951424->62955519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760775.936248:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06402300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554732672:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5757us (6025us total) trans 4294982678 rc 0/0 00000100:00100000:0.0:1789760775.936259:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15380 00000100:00100000:0.0:1789760775.936269:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719166855978 time 719176091545 state 0x2 ref 0 00000100:00100000:2.0:1789760775.944635:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760775.945284:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554732800 00000100:00100000:0.0:1789760775.945293:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554732800 00000100:00100000:0.0:1789760775.945344:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719166855978 time 719176091545 state 0x0 ref 1 00000100:00100000:0.0:1789760775.945358:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719176351961 time 719176091545 state 0x1 ref 1 00000100:00100000:0.0:1789760775.945474:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719176351961 time 719191091508 state 0x0 ref 1 00000100:00100000:0.0:1789760775.945497:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e54000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554732800:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760775.945545:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62955520, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e068 00010000:00010000:0.0:1789760775.945555:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4e7c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.945571:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760775.945576:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760775.945582:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4e7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62955520->62959615] (req 62955520->62959615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760775.945596:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4e7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62955520->62959615] (req 62955520->62959615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.952995:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982679 is committed 00002000:00010000:2.0:1789760775.953193:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62955520 -> 62959616 00002000:00010000:2.0:1789760775.953198:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122984 -> 122992 00010000:00010000:2.0:1789760775.953212:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4e7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62955520->62959615] (req 62955520->62959615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.953224:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4e7c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62955520->62959615] (req 62955520->62959615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.953234:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4e7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62955520->62959615] (req 62955520->62959615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.953241:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760775.953246:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4e7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62955520->62959615] (req 62955520->62959615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.953255:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4e7c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62955520->62959615] (req 62955520->62959615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.953274:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4e7c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62955520->62959615] (req 62955520->62959615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.953282:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c4e7c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62955520->62959615] (req 62955520->62959615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.953425:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554732800:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7886us (8747us total) trans 4294982679 rc 0/0 00000100:00100000:2.0:1789760775.953437:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15381 00000100:00100000:2.0:1789760775.953447:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719176351961 time 719191091508 state 0x2 ref 0 00000100:00100000:1.0:1789760775.964658:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760775.964842:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554732928 00000100:00100000:2.0:1789760775.964911:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554732928 00000100:00100000:2.0:1789760775.964968:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719176351961 time 719191091508 state 0x0 ref 1 00000100:00100000:2.0:1789760775.964984:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719191351924 time 719191091508 state 0x1 ref 1 00000100:00100000:2.0:1789760775.965015:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719191351924 time 719210633936 state 0x0 ref 1 00000100:00100000:2.0:1789760775.965028:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554732928:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760775.965076:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62959616, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e070 00010000:00010000:2.0:1789760775.965085:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4e83 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.965099:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760775.965104:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.965111:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4e83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62959616->62963711] (req 62959616->62963711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.965127:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4e83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62959616->62963711] (req 62959616->62963711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.971989:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982680 is committed 00002000:00010000:2.0:1789760775.972082:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62959616 -> 62963712 00002000:00010000:2.0:1789760775.972088:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 122992 -> 123000 00010000:00010000:2.0:1789760775.972100:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4e83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62959616->62963711] (req 62959616->62963711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.972113:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4e83 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62959616->62963711] (req 62959616->62963711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.972122:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4e83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62959616->62963711] (req 62959616->62963711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.972134:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760775.972139:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4e83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62959616->62963711] (req 62959616->62963711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.972148:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4e83 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62959616->62963711] (req 62959616->62963711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.972170:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4e83 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62959616->62963711] (req 62959616->62963711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.972178:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4e83 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62959616->62963711] (req 62959616->62963711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.972278:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554732928:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7247us (7622us total) trans 4294982680 rc 0/0 00000100:00100000:2.0:1789760775.972289:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15382 00000100:00100000:2.0:1789760775.972298:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719191351924 time 719210633936 state 0x2 ref 0 00000100:00100000:3.0:1789760775.977295:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760775.977411:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554733056 00000100:00100000:2.0:1789760775.977421:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554733056 00000100:00100000:2.0:1789760775.977478:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719191351924 time 719210633936 state 0x0 ref 1 00000100:00100000:2.0:1789760775.977505:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719210894352 time 719210633936 state 0x1 ref 1 00000100:00100000:2.0:1789760775.977744:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719210894352 time 719223361386 state 0x0 ref 1 00000100:00100000:2.0:1789760775.977769:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06403800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554733056:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760775.977817:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62963712, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e078 00010000:00010000:2.0:1789760775.977827:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e8a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.977842:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760775.977846:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760775.977853:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62963712->62967807] (req 62963712->62967807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.977865:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62963712->62967807] (req 62963712->62967807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760775.983942:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982681 is committed 00002000:00010000:2.0:1789760775.984044:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62963712 -> 62967808 00002000:00010000:2.0:1789760775.984050:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123000 -> 123008 00010000:00010000:2.0:1789760775.984063:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62963712->62967807] (req 62963712->62967807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.984076:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e8a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62963712->62967807] (req 62963712->62967807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.984085:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62963712->62967807] (req 62963712->62967807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.984093:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760775.984098:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62963712->62967807] (req 62963712->62967807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.984107:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e8a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62963712->62967807] (req 62963712->62967807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.984125:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e8a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62963712->62967807] (req 62963712->62967807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760775.984132:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c4e8a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62963712->62967807] (req 62963712->62967807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760775.984302:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06403800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554733056:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6529us (7009us total) trans 4294982681 rc 0/0 00000100:00100000:2.0:1789760775.984312:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15383 00000100:00100000:2.0:1789760775.984322:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719210894352 time 719223361386 state 0x2 ref 0 00000100:00100000:0.0:1789760775.992301:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760775.992449:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554733184 00000100:00100000:1.0:1789760775.992457:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554733184 00000100:00100000:1.0:1789760775.992509:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719210894352 time 719223361386 state 0x0 ref 1 00000100:00100000:1.0:1789760775.992523:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719223621802 time 719223361386 state 0x1 ref 1 00000100:00100000:1.0:1789760775.992558:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719223621802 time 719238176388 state 0x0 ref 1 00000100:00100000:1.0:1789760775.992569:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554733184:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760775.992615:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62967808, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e080 00010000:00010000:1.0:1789760775.992625:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4e91 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.992641:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760775.992646:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760775.992652:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4e91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62967808->62971903] (req 62967808->62971903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.992665:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4e91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62967808->62971903] (req 62967808->62971903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760775.999094:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982682 is committed 00002000:00010000:1.0:1789760775.999255:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62967808 -> 62971904 00002000:00010000:1.0:1789760775.999262:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123008 -> 123016 00010000:00010000:1.0:1789760775.999280:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4e91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62967808->62971903] (req 62967808->62971903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.999294:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4e91 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62967808->62971903] (req 62967808->62971903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.999303:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4e91 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62967808->62971903] (req 62967808->62971903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.999310:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760775.999316:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4e91 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62967808->62971903] (req 62967808->62971903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.999332:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4e91 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62967808->62971903] (req 62967808->62971903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.999351:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4e91 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62967808->62971903] (req 62967808->62971903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760775.999358:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4e91 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62967808->62971903] (req 62967808->62971903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760775.999500:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554733184:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6928us (7212us total) trans 4294982682 rc 0/0 00000100:00100000:1.0:1789760775.999511:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15384 00000100:00100000:1.0:1789760775.999519:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719223621802 time 719238176388 state 0x2 ref 0 00000100:00100000:1.0:1789760776.005065:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760776.005195:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554733312 00000100:00100000:0.0:1789760776.005213:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554733312 00000100:00100000:0.0:1789760776.005266:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719223621802 time 719238176388 state 0x0 ref 1 00000100:00100000:0.0:1789760776.005282:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719238436804 time 719238176388 state 0x1 ref 1 00000100:00100000:0.0:1789760776.005313:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719238436804 time 719250931442 state 0x0 ref 1 00000100:00100000:0.0:1789760776.005325:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d82680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554733312:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760776.005372:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62971904, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e088 00010000:00010000:0.0:1789760776.005408:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4e98 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.005424:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760776.005428:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.005434:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4e98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62971904->62975999] (req 62971904->62975999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.005450:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4e98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62971904->62975999] (req 62971904->62975999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.010661:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982683 is committed 00002000:00010000:0.0:1789760776.010902:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62971904 -> 62976000 00002000:00010000:0.0:1789760776.010907:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123016 -> 123024 00010000:00010000:0.0:1789760776.010922:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4e98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62971904->62975999] (req 62971904->62975999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.010934:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4e98 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62971904->62975999] (req 62971904->62975999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.010944:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4e98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62971904->62975999] (req 62971904->62975999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.010956:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760776.010962:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4e98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62971904->62975999] (req 62971904->62975999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.010970:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4e98 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62971904->62975999] (req 62971904->62975999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.010989:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4e98 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62971904->62975999] (req 62971904->62975999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.010997:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4e98 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62971904->62975999] (req 62971904->62975999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.011096:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d82680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554733312:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5768us (6033us total) trans 4294982683 rc 0/0 00000100:00100000:0.0:1789760776.011115:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15385 00000100:00100000:0.0:1789760776.011123:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719238436804 time 719250931442 state 0x2 ref 0 00000100:00100000:3.0:1789760776.017360:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760776.018222:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554733440 00000100:00100000:2.0:1789760776.018233:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554733440 00000100:00100000:2.0:1789760776.018288:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719238436804 time 719250931442 state 0x0 ref 1 00000100:00100000:2.0:1789760776.018303:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719251191858 time 719250931442 state 0x1 ref 1 00000100:00100000:2.0:1789760776.018399:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719251191858 time 719263996393 state 0x0 ref 1 00000100:00100000:2.0:1789760776.018413:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f16a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554733440:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760776.018460:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62976000, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e090 00010000:00010000:2.0:1789760776.018469:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4e9f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.018484:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760776.018489:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.018495:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4e9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62976000->62980095] (req 62976000->62980095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.018510:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4e9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62976000->62980095] (req 62976000->62980095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.024682:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982684 is committed 00002000:00010000:1.0:1789760776.025211:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62976000 -> 62980096 00002000:00010000:1.0:1789760776.025217:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123024 -> 123032 00010000:00010000:1.0:1789760776.025231:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4e9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62976000->62980095] (req 62976000->62980095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.025244:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4e9f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62976000->62980095] (req 62976000->62980095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.025253:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4e9f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62976000->62980095] (req 62976000->62980095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.025265:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.025270:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4e9f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62976000->62980095] (req 62976000->62980095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.025279:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4e9f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62976000->62980095] (req 62976000->62980095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.025298:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4e9f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62976000->62980095] (req 62976000->62980095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.025305:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4e9f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62976000->62980095] (req 62976000->62980095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.026010:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f16a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554733440:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7591us (8650us total) trans 4294982684 rc 0/0 00000100:00100000:1.0:1789760776.026024:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15386 00000100:00100000:1.0:1789760776.026035:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719251191858 time 719263996393 state 0x2 ref 0 00000100:00100000:2.0:1789760776.030489:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760776.030601:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554733568 00000100:00100000:3.0:1789760776.030612:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554733568 00000100:00100000:3.0:1789760776.030668:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719251191858 time 719263996393 state 0x0 ref 1 00000100:00100000:3.0:1789760776.030680:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719264256809 time 719263996393 state 0x1 ref 1 00000100:00100000:3.0:1789760776.030701:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719264256809 time 719276320528 state 0x0 ref 1 00000100:00100000:3.0:1789760776.030712:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e56300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554733568:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760776.030766:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62980096, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e098 00010000:00010000:3.0:1789760776.030775:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ea6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.030791:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760776.030795:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.030802:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ea6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62980096->62984191] (req 62980096->62984191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.030816:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ea6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62980096->62984191] (req 62980096->62984191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.036122:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982685 is committed 00002000:00010000:3.0:1789760776.036307:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62980096 -> 62984192 00002000:00010000:3.0:1789760776.036312:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123032 -> 123040 00010000:00010000:3.0:1789760776.036326:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ea6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62980096->62984191] (req 62980096->62984191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.036339:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ea6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62980096->62984191] (req 62980096->62984191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.036348:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ea6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62980096->62984191] (req 62980096->62984191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.036367:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760776.036374:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ea6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62980096->62984191] (req 62980096->62984191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.036420:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ea6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62980096->62984191] (req 62980096->62984191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.036439:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ea6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62980096->62984191] (req 62980096->62984191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.036447:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ea6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62980096->62984191] (req 62980096->62984191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.036551:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e56300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554733568:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5832us (6074us total) trans 4294982685 rc 0/0 00000100:00100000:3.0:1789760776.036561:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15387 00000100:00100000:3.0:1789760776.036571:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719264256809 time 719276320528 state 0x2 ref 0 00000100:00100000:2.0:1789760776.042065:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.042189:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554733696 00000100:00100000:1.0:1789760776.042543:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554733696 00000100:00100000:1.0:1789760776.042598:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719264256809 time 719276320528 state 0x0 ref 1 00000100:00100000:1.0:1789760776.042613:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719276580944 time 719276320528 state 0x1 ref 1 00000100:00100000:1.0:1789760776.042681:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719276580944 time 719288298689 state 0x0 ref 1 00000100:00100000:1.0:1789760776.042700:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554733696:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.042755:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62984192, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e0a0 00010000:00010000:1.0:1789760776.042764:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ead lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.042781:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760776.042785:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.042790:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ead lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62984192->62988287] (req 62984192->62988287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.042803:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ead lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62984192->62988287] (req 62984192->62988287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.050454:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982686 is committed 00002000:00010000:1.0:1789760776.050581:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62984192 -> 62988288 00002000:00010000:1.0:1789760776.050586:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123040 -> 123048 00010000:00010000:1.0:1789760776.050599:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ead lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62984192->62988287] (req 62984192->62988287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.050613:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ead lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62984192->62988287] (req 62984192->62988287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.050623:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ead lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62984192->62988287] (req 62984192->62988287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.050632:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.050637:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ead lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62984192->62988287] (req 62984192->62988287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.050647:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ead lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62984192->62988287] (req 62984192->62988287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.050665:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ead lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62984192->62988287] (req 62984192->62988287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.050673:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ead lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62984192->62988287] (req 62984192->62988287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.050785:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554733696:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8087us (8726us total) trans 4294982686 rc 0/0 00000100:00100000:1.0:1789760776.050796:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15388 00000100:00100000:1.0:1789760776.050805:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719276580944 time 719288298689 state 0x2 ref 0 00000100:00100000:2.0:1789760776.056862:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760776.058370:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554733824 00000100:00100000:0.0:1789760776.058396:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554733824 00000100:00100000:0.0:1789760776.059599:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719276580944 time 719288298689 state 0x0 ref 1 00000100:00100000:0.0:1789760776.059617:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719288559105 time 719288298689 state 0x1 ref 1 00000100:00100000:0.0:1789760776.059654:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719288559105 time 719305273495 state 0x0 ref 1 00000100:00100000:0.0:1789760776.060233:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e56a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554733824:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760776.060278:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62988288, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e0a8 00010000:00010000:0.0:1789760776.060297:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4eb4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.060312:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760776.060316:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.060322:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4eb4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62988288->62992383] (req 62988288->62992383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.060334:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4eb4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62988288->62992383] (req 62988288->62992383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.068634:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982687 is committed 00002000:00010000:0.0:1789760776.068746:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62988288 -> 62992384 00002000:00010000:0.0:1789760776.068752:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123048 -> 123056 00010000:00010000:0.0:1789760776.068765:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4eb4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62988288->62992383] (req 62988288->62992383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.068777:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4eb4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62988288->62992383] (req 62988288->62992383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.068786:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4eb4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62988288->62992383] (req 62988288->62992383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.068794:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760776.068799:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4eb4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62988288->62992383] (req 62988288->62992383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.068808:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4eb4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62988288->62992383] (req 62988288->62992383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.068829:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4eb4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62988288->62992383] (req 62988288->62992383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.068837:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4eb4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62988288->62992383] (req 62988288->62992383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.068935:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e56a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554733824:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8707us (12076us total) trans 4294982687 rc 0/0 00000100:00100000:0.0:1789760776.068945:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15389 00000100:00100000:0.0:1789760776.068955:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719288559105 time 719305273495 state 0x2 ref 0 00000100:00100000:2.0:1789760776.074913:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.075507:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554733952 00000100:00100000:1.0:1789760776.075518:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554733952 00000100:00100000:1.0:1789760776.075574:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719288559105 time 719305273495 state 0x0 ref 1 00000100:00100000:1.0:1789760776.075590:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719305533911 time 719305273495 state 0x1 ref 1 00000100:00100000:1.0:1789760776.075668:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719305533911 time 719321286387 state 0x0 ref 1 00000100:00100000:1.0:1789760776.075681:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554733952:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.075743:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62992384, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e0b0 00010000:00010000:1.0:1789760776.075753:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ebb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.075769:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760776.075773:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.075780:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ebb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62992384->62996479] (req 62992384->62996479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.075794:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ebb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62992384->62996479] (req 62992384->62996479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.083675:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982688 is committed 00002000:00010000:1.0:1789760776.083804:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62992384 -> 62996480 00002000:00010000:1.0:1789760776.083810:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123056 -> 123064 00010000:00010000:1.0:1789760776.083822:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ebb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62992384->62996479] (req 62992384->62996479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.083835:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ebb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62992384->62996479] (req 62992384->62996479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.083845:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ebb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62992384->62996479] (req 62992384->62996479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.083852:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.083857:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ebb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62992384->62996479] (req 62992384->62996479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.083872:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ebb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62992384->62996479] (req 62992384->62996479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.083891:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ebb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62992384->62996479] (req 62992384->62996479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.083898:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ebb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62992384->62996479] (req 62992384->62996479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.083993:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554733952:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8309us (9084us total) trans 4294982688 rc 0/0 00000100:00100000:1.0:1789760776.084004:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15390 00000100:00100000:1.0:1789760776.084014:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719305533911 time 719321286387 state 0x2 ref 0 00000100:00100000:2.0:1789760776.097474:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760776.097541:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554734080 00000100:00100000:3.0:1789760776.097551:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554734080 00000100:00100000:3.0:1789760776.097611:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719305533911 time 719321286387 state 0x0 ref 1 00000100:00100000:3.0:1789760776.097628:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719321546803 time 719321286387 state 0x1 ref 1 00000100:00100000:3.0:1789760776.097654:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719321546803 time 719343272822 state 0x0 ref 1 00000100:00100000:3.0:1789760776.097667:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554734080:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760776.097712:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 62996480, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e0b8 00010000:00010000:3.0:1789760776.097722:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ec2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.097881:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760776.097888:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.097905:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ec2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62996480->63000575] (req 62996480->63000575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.097920:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ec2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62996480->63000575] (req 62996480->63000575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.105769:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982689 is committed 00002000:00010000:3.0:1789760776.110296:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 62996480 -> 63000576 00002000:00010000:3.0:1789760776.110312:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123064 -> 123072 00010000:00010000:3.0:1789760776.110326:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ec2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62996480->63000575] (req 62996480->63000575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.110338:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ec2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62996480->63000575] (req 62996480->63000575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.110347:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ec2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62996480->63000575] (req 62996480->63000575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.110353:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760776.110358:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ec2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62996480->63000575] (req 62996480->63000575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.110532:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ec2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62996480->63000575] (req 62996480->63000575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.110555:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ec2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62996480->63000575] (req 62996480->63000575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.110563:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ec2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [62996480->63000575] (req 62996480->63000575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.110661:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554734080:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12992us (13191us total) trans 4294982689 rc 0/0 00000100:00100000:3.0:1789760776.110679:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15391 00000100:00100000:3.0:1789760776.110689:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719321546803 time 719343272822 state 0x2 ref 0 00000100:00100000:2.0:1789760776.118110:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760776.118240:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554734208 00000100:00100000:0.0:1789760776.118251:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554734208 00000100:00100000:0.0:1789760776.118304:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719321546803 time 719343272822 state 0x0 ref 1 00000100:00100000:0.0:1789760776.118319:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719343533238 time 719343272822 state 0x1 ref 1 00000100:00100000:0.0:1789760776.118658:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719343533238 time 719364270462 state 0x0 ref 1 00000100:00100000:0.0:1789760776.118751:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554734208:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760776.118799:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63000576, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e0c0 00010000:00010000:0.0:1789760776.118809:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ec9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.118825:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760776.118829:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.118836:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ec9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63000576->63004671] (req 63000576->63004671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.118850:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ec9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63000576->63004671] (req 63000576->63004671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.125948:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982690 is committed 00002000:00010000:0.0:1789760776.127360:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63000576 -> 63004672 00002000:00010000:0.0:1789760776.127519:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123072 -> 123080 00010000:00010000:0.0:1789760776.127537:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ec9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63000576->63004671] (req 63000576->63004671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.127551:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ec9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63000576->63004671] (req 63000576->63004671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.127560:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ec9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63000576->63004671] (req 63000576->63004671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.127567:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760776.127571:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ec9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63000576->63004671] (req 63000576->63004671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.127580:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ec9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63000576->63004671] (req 63000576->63004671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.127603:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ec9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63000576->63004671] (req 63000576->63004671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.127611:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ec9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63000576->63004671] (req 63000576->63004671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.128277:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554734208:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9525us (10235us total) trans 4294982690 rc 0/0 00000100:00100000:0.0:1789760776.128287:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15392 00000100:00100000:0.0:1789760776.128296:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719343533238 time 719364270462 state 0x2 ref 0 00000100:00100000:2.0:1789760776.134118:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760776.134274:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554734336 00000100:00100000:0.0:1789760776.134293:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554734336 00000100:00100000:0.0:1789760776.134347:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719343533238 time 719364270462 state 0x0 ref 1 00000100:00100000:0.0:1789760776.134362:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719364530878 time 719364270462 state 0x1 ref 1 00000100:00100000:0.0:1789760776.134429:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719364530878 time 719380047455 state 0x0 ref 1 00000100:00100000:1.0:1789760776.134952:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554734336:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.135003:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63004672, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e0c8 00010000:00010000:1.0:1789760776.135013:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ed0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.135027:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760776.135031:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.135038:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ed0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63004672->63008767] (req 63004672->63008767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.135121:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ed0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63004672->63008767] (req 63004672->63008767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.143608:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982691 is committed 00002000:00010000:1.0:1789760776.144669:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63004672 -> 63008768 00002000:00010000:1.0:1789760776.144676:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123080 -> 123088 00010000:00010000:1.0:1789760776.144698:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ed0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63004672->63008767] (req 63004672->63008767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.144712:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ed0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63004672->63008767] (req 63004672->63008767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.144721:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ed0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63004672->63008767] (req 63004672->63008767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.144737:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.144742:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ed0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63004672->63008767] (req 63004672->63008767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.144751:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ed0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63004672->63008767] (req 63004672->63008767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.144770:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ed0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63004672->63008767] (req 63004672->63008767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.144778:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ed0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63004672->63008767] (req 63004672->63008767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.144873:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554734336:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10147us (10762us total) trans 4294982691 rc 0/0 00000100:00100000:1.0:1789760776.144884:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15393 00000100:00100000:1.0:1789760776.144893:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719364530878 time 719380047455 state 0x2 ref 0 00000100:00100000:2.0:1789760776.162069:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.162561:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554734464 00000100:00100000:1.0:1789760776.162570:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554734464 00000100:00100000:1.0:1789760776.162643:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719364530878 time 719380047455 state 0x0 ref 1 00000100:00100000:1.0:1789760776.162659:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719380307871 time 719380047455 state 0x1 ref 1 00000100:00100000:1.0:1789760776.162691:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719380307871 time 719408309599 state 0x0 ref 1 00000100:00100000:1.0:1789760776.162704:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554734464:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.162765:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63008768, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e0d0 00010000:00010000:1.0:1789760776.162774:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ed7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.162790:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760776.162794:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.162801:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ed7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63008768->63012863] (req 63008768->63012863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.162820:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ed7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63008768->63012863] (req 63008768->63012863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.168973:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982692 is committed 00002000:00010000:1.0:1789760776.169304:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63008768 -> 63012864 00002000:00010000:1.0:1789760776.169309:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123088 -> 123096 00010000:00010000:1.0:1789760776.169323:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ed7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63008768->63012863] (req 63008768->63012863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.169335:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ed7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63008768->63012863] (req 63008768->63012863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.169344:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ed7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63008768->63012863] (req 63008768->63012863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.169351:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.169356:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ed7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63008768->63012863] (req 63008768->63012863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.169365:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ed7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63008768->63012863] (req 63008768->63012863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.169522:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ed7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63008768->63012863] (req 63008768->63012863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.169532:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4ed7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63008768->63012863] (req 63008768->63012863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.169627:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554734464:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6920us (7563us total) trans 4294982692 rc 0/0 00000100:00100000:1.0:1789760776.169638:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15394 00000100:00100000:1.0:1789760776.169647:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719380307871 time 719408309599 state 0x2 ref 0 00000100:00100000:2.0:1789760776.172296:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760776.172583:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554734592 00000100:00100000:3.0:1789760776.172594:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554734592 00000100:00100000:3.0:1789760776.172658:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719380307871 time 719408309599 state 0x0 ref 1 00000100:00100000:3.0:1789760776.172674:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719408570015 time 719408309599 state 0x1 ref 1 00000100:00100000:3.0:1789760776.172736:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719408570015 time 719418345145 state 0x0 ref 1 00000100:00100000:3.0:1789760776.172749:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554734592:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760776.172798:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63012864, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e0d8 00010000:00010000:3.0:1789760776.172807:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ede lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.172823:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760776.172828:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.172834:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ede lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63012864->63016959] (req 63012864->63016959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.172849:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ede lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63012864->63016959] (req 63012864->63016959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.185168:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982693 is committed 00002000:00010000:2.0:1789760776.185280:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63012864 -> 63016960 00002000:00010000:2.0:1789760776.185286:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123096 -> 123104 00010000:00010000:2.0:1789760776.185299:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ede lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63012864->63016959] (req 63012864->63016959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.185312:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ede lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63012864->63016959] (req 63012864->63016959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.185320:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ede lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63012864->63016959] (req 63012864->63016959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.185328:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760776.185333:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ede lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63012864->63016959] (req 63012864->63016959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.185342:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ede lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63012864->63016959] (req 63012864->63016959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.185362:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ede lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63012864->63016959] (req 63012864->63016959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.185370:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c4ede lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63012864->63016959] (req 63012864->63016959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.185501:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554734592:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12749us (13208us total) trans 4294982693 rc 0/0 00000100:00100000:2.0:1789760776.185513:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15395 00000100:00100000:2.0:1789760776.185523:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719408570015 time 719418345145 state 0x2 ref 0 00000100:00100000:2.0:1789760776.189054:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760776.189126:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554734720 00000100:00100000:0.0:1789760776.189135:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554734720 00000100:00100000:0.0:1789760776.189176:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719408570015 time 719418345145 state 0x0 ref 1 00000100:00100000:0.0:1789760776.189187:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719418605561 time 719418345145 state 0x1 ref 1 00000100:00100000:0.0:1789760776.189209:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719418605561 time 719434829291 state 0x0 ref 1 00000100:00100000:0.0:1789760776.189218:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554734720:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760776.190681:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63016960, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e0e0 00010000:00010000:0.0:1789760776.190694:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ee5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.190708:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760776.190713:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.190719:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ee5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63016960->63021055] (req 63016960->63021055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.190742:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ee5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63016960->63021055] (req 63016960->63021055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.199440:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982694 is committed 00002000:00010000:0.0:1789760776.199871:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63016960 -> 63021056 00002000:00010000:0.0:1789760776.199876:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123104 -> 123112 00010000:00010000:0.0:1789760776.199888:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ee5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63016960->63021055] (req 63016960->63021055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.199900:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ee5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63016960->63021055] (req 63016960->63021055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.199909:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ee5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63016960->63021055] (req 63016960->63021055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.199917:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760776.199922:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ee5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63016960->63021055] (req 63016960->63021055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.199936:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ee5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63016960->63021055] (req 63016960->63021055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.199956:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ee5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63016960->63021055] (req 63016960->63021055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.199964:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4ee5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63016960->63021055] (req 63016960->63021055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.200068:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554734720:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10846us (11018us total) trans 4294982694 rc 0/0 00000100:00100000:0.0:1789760776.200079:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15396 00000100:00100000:0.0:1789760776.200089:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719418605561 time 719434829291 state 0x2 ref 0 00000100:00100000:1.0:1789760776.203678:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760776.203876:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554734848 00000100:00100000:2.0:1789760776.203888:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554734848 00000100:00100000:2.0:1789760776.203946:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719418605561 time 719434829291 state 0x0 ref 1 00000100:00100000:2.0:1789760776.203963:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719435089707 time 719434829291 state 0x1 ref 1 00000100:00100000:2.0:1789760776.203992:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719435089707 time 719449611106 state 0x0 ref 1 00000100:00100000:2.0:1789760776.204005:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d81500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554734848:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760776.204055:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63021056, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e0e8 00010000:00010000:2.0:1789760776.204064:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4eec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.204080:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760776.204085:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.204091:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4eec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63021056->63025151] (req 63021056->63025151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.204115:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4eec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63021056->63025151] (req 63021056->63025151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.210632:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982695 is committed 00002000:00010000:2.0:1789760776.210911:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63021056 -> 63025152 00002000:00010000:2.0:1789760776.210917:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123112 -> 123120 00010000:00010000:2.0:1789760776.210930:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4eec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63021056->63025151] (req 63021056->63025151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.210942:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4eec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63021056->63025151] (req 63021056->63025151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.210951:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4eec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63021056->63025151] (req 63021056->63025151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.210958:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760776.210962:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4eec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63021056->63025151] (req 63021056->63025151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.210969:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4eec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63021056->63025151] (req 63021056->63025151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.210989:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4eec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63021056->63025151] (req 63021056->63025151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.210996:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c4eec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63021056->63025151] (req 63021056->63025151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.211198:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d81500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554734848:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7190us (7523us total) trans 4294982695 rc 0/0 00000100:00100000:2.0:1789760776.211209:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15397 00000100:00100000:2.0:1789760776.211217:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719435089707 time 719449611106 state 0x2 ref 0 00000100:00100000:2.0:1789760776.215640:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.215743:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554734976 00000100:00100000:1.0:1789760776.215752:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554734976 00000100:00100000:1.0:1789760776.215804:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719435089707 time 719449611106 state 0x0 ref 1 00000100:00100000:1.0:1789760776.215819:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719449871522 time 719449611106 state 0x1 ref 1 00000100:00100000:1.0:1789760776.215839:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719449871522 time 719461457898 state 0x0 ref 1 00000100:00100000:1.0:1789760776.215851:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554734976:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.215898:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63025152, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e0f0 00010000:00010000:1.0:1789760776.215906:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4ef3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.215921:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760776.215925:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.215931:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4ef3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63025152->63029247] (req 63025152->63029247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.215944:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4ef3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63025152->63029247] (req 63025152->63029247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.224322:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982696 is committed 00002000:00010000:1.0:1789760776.224820:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63025152 -> 63029248 00002000:00010000:1.0:1789760776.224825:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123120 -> 123128 00010000:00010000:1.0:1789760776.224838:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4ef3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63025152->63029247] (req 63025152->63029247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.224851:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4ef3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63025152->63029247] (req 63025152->63029247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.224860:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4ef3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63025152->63029247] (req 63025152->63029247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.224867:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.224871:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4ef3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63025152->63029247] (req 63025152->63029247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.224881:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4ef3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63025152->63029247] (req 63025152->63029247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.224900:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4ef3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63025152->63029247] (req 63025152->63029247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.224909:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4ef3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63025152->63029247] (req 63025152->63029247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.225046:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554734976:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9191us (9409us total) trans 4294982696 rc 0/0 00000100:00100000:1.0:1789760776.225061:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15398 00000100:00100000:1.0:1789760776.225070:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719449871522 time 719461457898 state 0x2 ref 0 00000100:00100000:0.0:1789760776.228897:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760776.229460:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554735104 00000100:00100000:3.0:1789760776.229471:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554735104 00000100:00100000:3.0:1789760776.229536:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719449871522 time 719461457898 state 0x0 ref 1 00000100:00100000:3.0:1789760776.229736:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719461718314 time 719461457898 state 0x1 ref 1 00000100:00100000:3.0:1789760776.229771:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719461718314 time 719475389466 state 0x0 ref 1 00000100:00100000:3.0:1789760776.229784:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe1f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554735104:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760776.229829:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63029248, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e0f8 00010000:00010000:3.0:1789760776.229837:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4efa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.229853:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760776.229858:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.229865:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4efa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63029248->63033343] (req 63029248->63033343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.229883:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4efa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63029248->63033343] (req 63029248->63033343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.237087:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982697 is committed 00002000:00010000:3.0:1789760776.237258:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63029248 -> 63033344 00002000:00010000:3.0:1789760776.237264:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123128 -> 123136 00010000:00010000:3.0:1789760776.237277:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4efa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63029248->63033343] (req 63029248->63033343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.237289:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4efa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63029248->63033343] (req 63029248->63033343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.237298:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4efa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63029248->63033343] (req 63029248->63033343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.237305:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760776.237310:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4efa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63029248->63033343] (req 63029248->63033343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.237317:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4efa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63029248->63033343] (req 63029248->63033343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.237337:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4efa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63029248->63033343] (req 63029248->63033343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.237345:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4efa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63029248->63033343] (req 63029248->63033343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.237799:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe1f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554735104:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8012us (8906us total) trans 4294982697 rc 0/0 00000100:00100000:3.0:1789760776.237811:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15399 00000100:00100000:3.0:1789760776.237820:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719461718314 time 719475389466 state 0x2 ref 0 00000100:00100000:2.0:1789760776.244350:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760776.244493:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554735232 00000100:00100000:0.0:1789760776.244503:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554735232 00000100:00100000:0.0:1789760776.244565:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719461718314 time 719475389466 state 0x0 ref 1 00000100:00100000:0.0:1789760776.244580:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719475649882 time 719475389466 state 0x1 ref 1 00000100:00100000:0.0:1789760776.244601:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719475649882 time 719490219323 state 0x0 ref 1 00000100:00100000:0.0:1789760776.244612:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554735232:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760776.244661:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63033344, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e100 00010000:00010000:0.0:1789760776.244671:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4f01 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.244686:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760776.244690:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.244696:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4f01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63033344->63037439] (req 63033344->63037439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.244708:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4f01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63033344->63037439] (req 63033344->63037439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.250973:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982698 is committed 00002000:00010000:1.0:1789760776.251157:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63033344 -> 63037440 00002000:00010000:1.0:1789760776.251163:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123136 -> 123144 00010000:00010000:1.0:1789760776.251177:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4f01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63033344->63037439] (req 63033344->63037439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.251190:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4f01 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63033344->63037439] (req 63033344->63037439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.251199:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4f01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63033344->63037439] (req 63033344->63037439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.251207:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.251212:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4f01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63033344->63037439] (req 63033344->63037439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.251221:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4f01 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63033344->63037439] (req 63033344->63037439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.251240:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4f01 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63033344->63037439] (req 63033344->63037439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.251248:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c4f01 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63033344->63037439] (req 63033344->63037439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.251330:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554735232:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6716us (6987us total) trans 4294982698 rc 0/0 00000100:00100000:1.0:1789760776.251340:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15400 00000100:00100000:1.0:1789760776.251351:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719475649882 time 719490219323 state 0x2 ref 0 00000100:00100000:0.0:1789760776.255058:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760776.255235:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554735360 00000100:00100000:2.0:1789760776.255244:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554735360 00000100:00100000:2.0:1789760776.255296:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719475649882 time 719490219323 state 0x0 ref 1 00000100:00100000:2.0:1789760776.255311:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719490479739 time 719490219323 state 0x1 ref 1 00000100:00100000:2.0:1789760776.255349:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719490479739 time 719500967462 state 0x0 ref 1 00000100:00100000:2.0:1789760776.255361:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554735360:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760776.255453:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63037440, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e108 00010000:00010000:2.0:1789760776.255463:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4f08 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.255479:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760776.255484:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.255491:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4f08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63037440->63041535] (req 63037440->63041535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.255505:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4f08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63037440->63041535] (req 63037440->63041535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.260959:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982699 is committed 00002000:00010000:2.0:1789760776.261528:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63037440 -> 63041536 00002000:00010000:2.0:1789760776.261534:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123144 -> 123152 00010000:00010000:2.0:1789760776.261547:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4f08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63037440->63041535] (req 63037440->63041535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.261559:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4f08 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63037440->63041535] (req 63037440->63041535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.261570:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4f08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63037440->63041535] (req 63037440->63041535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.261578:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760776.261583:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4f08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63037440->63041535] (req 63037440->63041535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.261593:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4f08 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63037440->63041535] (req 63037440->63041535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.261613:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4f08 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63037440->63041535] (req 63037440->63041535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.261866:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4f08 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63037440->63041535] (req 63037440->63041535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.261999:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554735360:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6636us (6945us total) trans 4294982699 rc 0/0 00000100:00100000:2.0:1789760776.262009:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15401 00000100:00100000:2.0:1789760776.262018:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719490479739 time 719500967462 state 0x2 ref 0 00000100:00100000:0.0:1789760776.265589:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.265937:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554735488 00000100:00100000:1.0:1789760776.265946:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554735488 00000100:00100000:1.0:1789760776.266003:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719490479739 time 719500967462 state 0x0 ref 1 00000100:00100000:1.0:1789760776.266017:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719501227878 time 719500967462 state 0x1 ref 1 00000100:00100000:1.0:1789760776.266084:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719501227878 time 719511701172 state 0x0 ref 1 00000100:00100000:1.0:1789760776.266097:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554735488:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.266157:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63041536, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e110 00010000:00010000:1.0:1789760776.266166:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f0f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.266181:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760776.266186:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.266192:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63041536->63045631] (req 63041536->63045631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.266206:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63041536->63045631] (req 63041536->63045631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.270985:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982700 is committed 00002000:00010000:1.0:1789760776.271756:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63041536 -> 63045632 00002000:00010000:1.0:1789760776.271763:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123152 -> 123160 00010000:00010000:1.0:1789760776.271775:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63041536->63045631] (req 63041536->63045631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.272167:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f0f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63041536->63045631] (req 63041536->63045631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.272176:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f0f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63041536->63045631] (req 63041536->63045631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.272183:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.272188:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f0f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63041536->63045631] (req 63041536->63045631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.272197:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f0f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63041536->63045631] (req 63041536->63045631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.272218:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f0f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63041536->63045631] (req 63041536->63045631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.272225:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f0f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63041536->63045631] (req 63041536->63045631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.272365:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554735488:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6264us (6778us total) trans 4294982700 rc 0/0 00000100:00100000:1.0:1789760776.272409:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15402 00000100:00100000:1.0:1789760776.272420:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719501227878 time 719511701172 state 0x2 ref 0 00000100:00100000:1.0:1789760776.282300:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760776.283161:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554735616 00000100:00100000:0.0:1789760776.283172:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554735616 00000100:00100000:0.0:1789760776.283236:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719501227878 time 719511701172 state 0x0 ref 1 00000100:00100000:0.0:1789760776.283253:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719511961588 time 719511701172 state 0x1 ref 1 00000100:00100000:0.0:1789760776.283296:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719511961588 time 719528911011 state 0x0 ref 1 00000100:00100000:0.0:1789760776.283308:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554735616:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760776.283357:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63045632, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e118 00010000:00010000:0.0:1789760776.283421:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4f16 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.283438:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760776.283443:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.283450:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4f16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63045632->63049727] (req 63045632->63049727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.283472:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4f16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63045632->63049727] (req 63045632->63049727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.288661:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982701 is committed 00002000:00010000:3.0:1789760776.288830:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63045632 -> 63049728 00002000:00010000:3.0:1789760776.288835:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123160 -> 123168 00010000:00010000:3.0:1789760776.288849:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4f16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63045632->63049727] (req 63045632->63049727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.288862:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4f16 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63045632->63049727] (req 63045632->63049727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.288871:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4f16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63045632->63049727] (req 63045632->63049727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.288879:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760776.288883:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4f16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63045632->63049727] (req 63045632->63049727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.288892:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4f16 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63045632->63049727] (req 63045632->63049727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.288911:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4f16 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63045632->63049727] (req 63045632->63049727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.288920:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c4f16 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63045632->63049727] (req 63045632->63049727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.289521:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554735616:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6207us (7222us total) trans 4294982701 rc 0/0 00000100:00100000:3.0:1789760776.289532:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15403 00000100:00100000:3.0:1789760776.289544:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719511961588 time 719528911011 state 0x2 ref 0 00000100:00100000:0.0:1789760776.295630:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.295875:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554735744 00000100:00100000:1.0:1789760776.295887:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554735744 00000100:00100000:1.0:1789760776.295941:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719511961588 time 719528911011 state 0x0 ref 1 00000100:00100000:1.0:1789760776.295957:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719529171427 time 719528911011 state 0x1 ref 1 00000100:00100000:1.0:1789760776.296051:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719529171427 time 719541665324 state 0x0 ref 1 00000100:00100000:1.0:1789760776.296065:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554735744:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.296124:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63049728, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e120 00010000:00010000:1.0:1789760776.296133:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f1d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.296149:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760776.296153:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.296158:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63049728->63053823] (req 63049728->63053823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.296172:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63049728->63053823] (req 63049728->63053823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.301299:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982702 is committed 00002000:00010000:1.0:1789760776.301682:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63049728 -> 63053824 00002000:00010000:1.0:1789760776.301689:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123168 -> 123176 00010000:00010000:1.0:1789760776.301702:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63049728->63053823] (req 63049728->63053823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.301714:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f1d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63049728->63053823] (req 63049728->63053823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.301732:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63049728->63053823] (req 63049728->63053823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.301740:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.301745:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63049728->63053823] (req 63049728->63053823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.301753:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f1d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63049728->63053823] (req 63049728->63053823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.301772:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f1d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63049728->63053823] (req 63049728->63053823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.301780:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f1d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63049728->63053823] (req 63049728->63053823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.301951:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554735744:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5884us (6326us total) trans 4294982702 rc 0/0 00000100:00100000:1.0:1789760776.301962:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15404 00000100:00100000:1.0:1789760776.301971:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719529171427 time 719541665324 state 0x2 ref 0 00000100:00100000:0.0:1789760776.308902:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760776.309332:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554735872 00000100:00100000:2.0:1789760776.309344:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554735872 00000100:00100000:2.0:1789760776.309447:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719529171427 time 719541665324 state 0x0 ref 1 00000100:00100000:2.0:1789760776.309463:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719541925740 time 719541665324 state 0x1 ref 1 00000100:00100000:3.0:1789760776.309635:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719541925740 time 719555250872 state 0x0 ref 1 00000100:00100000:3.0:1789760776.309653:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554735872:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760776.309701:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63053824, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e128 00010000:00010000:3.0:1789760776.309711:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4f24 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.309737:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760776.309742:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.309749:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4f24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63053824->63057919] (req 63053824->63057919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.309763:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4f24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63053824->63057919] (req 63053824->63057919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.315927:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982703 is committed 00002000:00010000:3.0:1789760776.316261:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63053824 -> 63057920 00002000:00010000:3.0:1789760776.316267:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123176 -> 123184 00010000:00010000:3.0:1789760776.316280:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4f24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63053824->63057919] (req 63053824->63057919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.316293:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4f24 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63053824->63057919] (req 63053824->63057919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.316302:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4f24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63053824->63057919] (req 63053824->63057919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.316309:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760776.316313:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4f24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63053824->63057919] (req 63053824->63057919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.316322:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4f24 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63053824->63057919] (req 63053824->63057919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.316341:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4f24 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63053824->63057919] (req 63053824->63057919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.316348:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4f24 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63053824->63057919] (req 63053824->63057919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760776.316480:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554735872:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6827us (7582us total) trans 4294982703 rc 0/0 00000100:00100000:3.0:1789760776.316928:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15405 00000100:00100000:3.0:1789760776.316938:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719541925740 time 719555250872 state 0x2 ref 0 00000100:00100000:2.0:1789760776.321040:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.323940:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554736000 00000100:00100000:1.0:1789760776.323957:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554736000 00000100:00100000:1.0:1789760776.324020:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719541925740 time 719555250872 state 0x0 ref 1 00000100:00100000:1.0:1789760776.324035:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719555511288 time 719555250872 state 0x1 ref 1 00000100:00100000:1.0:1789760776.324115:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719555511288 time 719569732314 state 0x0 ref 1 00000100:00100000:1.0:1789760776.324130:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554736000:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.324184:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63057920, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e130 00010000:00010000:1.0:1789760776.324194:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f2b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.324210:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760776.324215:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.324221:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63057920->63062015] (req 63057920->63062015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.324236:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63057920->63062015] (req 63057920->63062015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.328735:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982704 is committed 00002000:00010000:1.0:1789760776.328866:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63057920 -> 63062016 00002000:00010000:1.0:1789760776.328871:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123184 -> 123192 00010000:00010000:1.0:1789760776.328885:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63057920->63062015] (req 63057920->63062015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.328898:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f2b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63057920->63062015] (req 63057920->63062015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.328907:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63057920->63062015] (req 63057920->63062015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.328921:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.328925:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63057920->63062015] (req 63057920->63062015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.328934:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f2b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63057920->63062015] (req 63057920->63062015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.328954:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f2b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63057920->63062015] (req 63057920->63062015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.328962:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f2b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63057920->63062015] (req 63057920->63062015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.329061:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554736000:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4929us (8026us total) trans 4294982704 rc 0/0 00000100:00100000:1.0:1789760776.329071:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15406 00000100:00100000:1.0:1789760776.329081:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719555511288 time 719569732314 state 0x2 ref 0 00000100:00100000:0.0:1789760776.338340:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760776.338520:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554736128 00000100:00100000:2.0:1789760776.338529:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554736128 00000100:00100000:2.0:1789760776.338582:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719555511288 time 719569732314 state 0x0 ref 1 00000100:00100000:2.0:1789760776.338597:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719569992730 time 719569732314 state 0x1 ref 1 00000100:00100000:2.0:1789760776.338630:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719569992730 time 719584249034 state 0x0 ref 1 00000100:00100000:2.0:1789760776.338643:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554736128:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760776.338688:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63062016, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e138 00010000:00010000:2.0:1789760776.338697:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4f32 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.338713:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760776.338717:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.338723:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4f32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63062016->63066111] (req 63062016->63066111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.338748:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4f32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63062016->63066111] (req 63062016->63066111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.343250:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982705 is committed 00002000:00010000:2.0:1789760776.343399:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63062016 -> 63066112 00002000:00010000:2.0:1789760776.343405:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123192 -> 123200 00010000:00010000:2.0:1789760776.343419:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4f32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63062016->63066111] (req 63062016->63066111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.343450:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4f32 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63062016->63066111] (req 63062016->63066111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.343460:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4f32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63062016->63066111] (req 63062016->63066111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.343468:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760776.343473:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4f32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63062016->63066111] (req 63062016->63066111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.343748:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4f32 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63062016->63066111] (req 63062016->63066111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.343779:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4f32 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63062016->63066111] (req 63062016->63066111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.343788:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c4f32 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63062016->63066111] (req 63062016->63066111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.343883:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554736128:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5238us (5550us total) trans 4294982705 rc 0/0 00000100:00100000:2.0:1789760776.343895:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15407 00000100:00100000:2.0:1789760776.343903:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719569992730 time 719584249034 state 0x2 ref 0 00000100:00100000:0.0:1789760776.348482:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.348562:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554736256 00000100:00100000:1.0:1789760776.348573:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554736256 00000100:00100000:1.0:1789760776.348628:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719569992730 time 719584249034 state 0x0 ref 1 00000100:00100000:1.0:1789760776.348641:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719584509450 time 719584249034 state 0x1 ref 1 00000100:00100000:1.0:1789760776.348672:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719584509450 time 719594290628 state 0x0 ref 1 00000100:00100000:1.0:1789760776.348687:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554736256:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.348743:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63066112, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e140 00010000:00010000:1.0:1789760776.348752:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f39 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.348769:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760776.348773:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.348779:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63066112->63070207] (req 63066112->63070207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.348802:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63066112->63070207] (req 63066112->63070207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.357688:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982706 is committed 00002000:00010000:1.0:1789760776.357830:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63066112 -> 63070208 00002000:00010000:1.0:1789760776.357836:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123200 -> 123208 00010000:00010000:1.0:1789760776.357849:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63066112->63070207] (req 63066112->63070207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.357862:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f39 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63066112->63070207] (req 63066112->63070207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.357871:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63066112->63070207] (req 63066112->63070207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.357878:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.357884:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63066112->63070207] (req 63066112->63070207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.357892:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f39 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63066112->63070207] (req 63066112->63070207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.357912:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f39 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63066112->63070207] (req 63066112->63070207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.357919:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f39 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63066112->63070207] (req 63066112->63070207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.358012:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554736256:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9322us (9534us total) trans 4294982706 rc 0/0 00000100:00100000:1.0:1789760776.358021:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15408 00000100:00100000:1.0:1789760776.358033:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719584509450 time 719594290628 state 0x2 ref 0 00000100:00100000:1.0:1789760776.366677:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760776.367635:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554736384 00000100:00100000:3.0:1789760776.367648:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554736384 00000100:00100000:3.0:1789760776.367707:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719584509450 time 719594290628 state 0x0 ref 1 00000100:00100000:3.0:1789760776.367722:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719594551044 time 719594290628 state 0x1 ref 1 00000100:00100000:3.0:1789760776.367767:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719594551044 time 719613385742 state 0x0 ref 1 00000100:00100000:3.0:1789760776.367779:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d83100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554736384:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760776.367994:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63070208, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e148 00010000:00010000:3.0:1789760776.368003:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4f40 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.368019:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760776.368024:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.368030:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4f40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63070208->63074303] (req 63070208->63074303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.368044:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4f40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63070208->63074303] (req 63070208->63074303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.381454:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982707 is committed 00002000:00010000:0.0:1789760776.381663:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63070208 -> 63074304 00002000:00010000:0.0:1789760776.381668:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123208 -> 123216 00010000:00010000:0.0:1789760776.381681:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4f40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63070208->63074303] (req 63070208->63074303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.381694:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4f40 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63070208->63074303] (req 63070208->63074303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.381708:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4f40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63070208->63074303] (req 63070208->63074303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.381714:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760776.381719:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4f40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63070208->63074303] (req 63070208->63074303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.381737:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4f40 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63070208->63074303] (req 63070208->63074303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.381756:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4f40 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63070208->63074303] (req 63070208->63074303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.381764:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c4f40 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63070208->63074303] (req 63070208->63074303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.381989:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d83100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554736384:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14207us (15317us total) trans 4294982707 rc 0/0 00000100:00100000:0.0:1789760776.382000:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15409 00000100:00100000:0.0:1789760776.382009:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719594551044 time 719613385742 state 0x2 ref 0 00000100:00100000:0.0:1789760776.388480:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760776.388564:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554736512 00000100:00100000:2.0:1789760776.388575:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554736512 00000100:00100000:2.0:1789760776.388627:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719594551044 time 719613385742 state 0x0 ref 1 00000100:00100000:2.0:1789760776.388643:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719613646158 time 719613385742 state 0x1 ref 1 00000100:00100000:2.0:1789760776.388671:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719613646158 time 719634289645 state 0x0 ref 1 00000100:00100000:2.0:1789760776.388684:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554736512:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760776.390038:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63074304, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e150 00010000:00010000:2.0:1789760776.390050:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4f47 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.390067:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760776.390072:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.390078:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4f47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63074304->63078399] (req 63074304->63078399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.390091:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4f47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63074304->63078399] (req 63074304->63078399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.401637:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982708 is committed 00002000:00010000:1.0:1789760776.403325:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63074304 -> 63078400 00002000:00010000:1.0:1789760776.403334:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123216 -> 123224 00010000:00010000:1.0:1789760776.403346:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4f47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63074304->63078399] (req 63074304->63078399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.403360:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4f47 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63074304->63078399] (req 63074304->63078399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.403370:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4f47 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63074304->63078399] (req 63074304->63078399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.403420:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.403426:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4f47 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63074304->63078399] (req 63074304->63078399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.403436:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4f47 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63074304->63078399] (req 63074304->63078399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.403458:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4f47 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63074304->63078399] (req 63074304->63078399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.403466:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4f47 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63074304->63078399] (req 63074304->63078399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.403571:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554736512:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14883us (15094us total) trans 4294982708 rc 0/0 00000100:00100000:1.0:1789760776.403582:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15410 00000100:00100000:1.0:1789760776.403591:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719613646158 time 719634289645 state 0x2 ref 0 00000100:00100000:2.0:1789760776.409112:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.409176:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554736640 00000100:00100000:1.0:1789760776.409185:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554736640 00000100:00100000:1.0:1789760776.409234:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719613646158 time 719634289645 state 0x0 ref 1 00000100:00100000:1.0:1789760776.409248:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719634550061 time 719634289645 state 0x1 ref 1 00000100:00100000:1.0:1789760776.409287:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719634550061 time 719654905771 state 0x0 ref 1 00000100:00100000:1.0:1789760776.409302:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554736640:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.409343:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63078400, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e158 00010000:00010000:1.0:1789760776.409351:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f4e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.409366:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760776.409370:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.409404:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63078400->63082495] (req 63078400->63082495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.409419:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63078400->63082495] (req 63078400->63082495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.417037:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982709 is committed 00002000:00010000:1.0:1789760776.417298:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63078400 -> 63082496 00002000:00010000:1.0:1789760776.417303:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123224 -> 123232 00010000:00010000:1.0:1789760776.417315:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63078400->63082495] (req 63078400->63082495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.417327:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f4e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63078400->63082495] (req 63078400->63082495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.417336:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63078400->63082495] (req 63078400->63082495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.417343:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.417348:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63078400->63082495] (req 63078400->63082495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.417357:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f4e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63078400->63082495] (req 63078400->63082495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.417407:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f4e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63078400->63082495] (req 63078400->63082495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.417414:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f4e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63078400->63082495] (req 63078400->63082495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.417495:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554736640:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8189us (8399us total) trans 4294982709 rc 0/0 00000100:00100000:1.0:1789760776.417505:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15411 00000100:00100000:1.0:1789760776.417514:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719634550061 time 719654905771 state 0x2 ref 0 00000100:00100000:2.0:1789760776.423289:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.423364:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554736768 00000100:00100000:1.0:1789760776.423373:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554736768 00000100:00100000:1.0:1789760776.423452:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719634550061 time 719654905771 state 0x0 ref 1 00000100:00100000:1.0:1789760776.423466:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719655166187 time 719654905771 state 0x1 ref 1 00000100:00100000:1.0:1789760776.423494:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719655166187 time 719669112909 state 0x0 ref 1 00000100:00100000:1.0:1789760776.423504:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554736768:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.423555:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63082496, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e160 00010000:00010000:1.0:1789760776.423565:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f55 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.423580:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760776.423585:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.423591:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63082496->63086591] (req 63082496->63086591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.423604:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63082496->63086591] (req 63082496->63086591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.433184:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982710 is committed 00002000:00010000:1.0:1789760776.433360:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63082496 -> 63086592 00002000:00010000:1.0:1789760776.433365:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123232 -> 123240 00010000:00010000:1.0:1789760776.433410:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63082496->63086591] (req 63082496->63086591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.433424:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f55 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63082496->63086591] (req 63082496->63086591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.433434:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63082496->63086591] (req 63082496->63086591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.433441:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.433445:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63082496->63086591] (req 63082496->63086591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.433454:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f55 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63082496->63086591] (req 63082496->63086591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.433472:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f55 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63082496->63086591] (req 63082496->63086591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.433480:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f55 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63082496->63086591] (req 63082496->63086591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.433575:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554736768:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10067us (10289us total) trans 4294982710 rc 0/0 00000100:00100000:1.0:1789760776.433586:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15412 00000100:00100000:1.0:1789760776.433595:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719655166187 time 719669112909 state 0x2 ref 0 00000100:00100000:0.0:1789760776.439637:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760776.441864:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554736896 00000100:00100000:3.0:1789760776.441879:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554736896 00000100:00100000:3.0:1789760776.442007:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719655166187 time 719669112909 state 0x0 ref 1 00000100:00100000:3.0:1789760776.442023:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719669373325 time 719669112909 state 0x1 ref 1 00000100:00100000:3.0:1789760776.442117:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719669373325 time 719687733652 state 0x0 ref 1 00000100:00100000:3.0:1789760776.442131:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554736896:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760776.442177:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63086592, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e168 00010000:00010000:3.0:1789760776.442187:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4f5c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.442201:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:3.0:1789760776.442206:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760776.442212:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4f5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63086592->63090687] (req 63086592->63090687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760776.442225:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4f5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63086592->63090687] (req 63086592->63090687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.454355:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982711 is committed 00002000:00010000:0.0:1789760776.454777:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63086592 -> 63090688 00002000:00010000:0.0:1789760776.454784:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123240 -> 123248 00010000:00010000:0.0:1789760776.454798:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4f5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63086592->63090687] (req 63086592->63090687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.454810:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4f5c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63086592->63090687] (req 63086592->63090687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.454819:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4f5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63086592->63090687] (req 63086592->63090687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.454827:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760776.454832:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4f5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63086592->63090687] (req 63086592->63090687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.454841:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4f5c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63086592->63090687] (req 63086592->63090687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.454861:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4f5c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63086592->63090687] (req 63086592->63090687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.454868:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c4f5c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63086592->63090687] (req 63086592->63090687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.454958:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554736896:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12826us (15325us total) trans 4294982711 rc 0/0 00000100:00100000:0.0:1789760776.454970:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15413 00000100:00100000:0.0:1789760776.454979:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719669373325 time 719687733652 state 0x2 ref 0 00000100:00100000:2.0:1789760776.459238:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.459336:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554737024 00000100:00100000:1.0:1789760776.459346:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554737024 00000100:00100000:1.0:1789760776.460062:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719669373325 time 719687733652 state 0x0 ref 1 00000100:00100000:1.0:1789760776.460080:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719687994068 time 719687733652 state 0x1 ref 1 00000100:00100000:1.0:1789760776.460145:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719687994068 time 719705762847 state 0x0 ref 1 00000100:00100000:1.0:1789760776.460161:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554737024:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.460209:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63090688, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e170 00010000:00010000:1.0:1789760776.460218:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f63 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.460234:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760776.460240:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.460257:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63090688->63094783] (req 63090688->63094783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.460271:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63090688->63094783] (req 63090688->63094783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.467214:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982712 is committed 00002000:00010000:1.0:1789760776.467505:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63090688 -> 63094784 00002000:00010000:1.0:1789760776.467510:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123248 -> 123256 00010000:00010000:1.0:1789760776.467524:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63090688->63094783] (req 63090688->63094783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.467537:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f63 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63090688->63094783] (req 63090688->63094783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.467546:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f63 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63090688->63094783] (req 63090688->63094783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.467554:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.467559:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f63 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63090688->63094783] (req 63090688->63094783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.467568:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f63 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63090688->63094783] (req 63090688->63094783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.467587:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f63 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63090688->63094783] (req 63090688->63094783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.467601:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f63 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63090688->63094783] (req 63090688->63094783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.467703:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554737024:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7542us (8469us total) trans 4294982712 rc 0/0 00000100:00100000:1.0:1789760776.467713:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15414 00000100:00100000:1.0:1789760776.467721:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719687994068 time 719705762847 state 0x2 ref 0 00000100:00100000:2.0:1789760776.475587:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.475774:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554737152 00000100:00100000:1.0:1789760776.475784:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554737152 00000100:00100000:1.0:1789760776.475838:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719687994068 time 719705762847 state 0x0 ref 1 00000100:00100000:1.0:1789760776.475853:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719706023263 time 719705762847 state 0x1 ref 1 00000100:00100000:1.0:1789760776.475887:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719706023263 time 719721504867 state 0x0 ref 1 00000100:00100000:1.0:1789760776.475900:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554737152:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.475950:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63094784, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e178 00010000:00010000:1.0:1789760776.475961:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f6a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.475977:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760776.475981:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.475988:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63094784->63098879] (req 63094784->63098879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.476009:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63094784->63098879] (req 63094784->63098879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.483358:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982713 is committed 00002000:00010000:1.0:1789760776.485256:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63094784 -> 63098880 00002000:00010000:1.0:1789760776.485262:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123256 -> 123264 00010000:00010000:1.0:1789760776.485274:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63094784->63098879] (req 63094784->63098879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.485283:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f6a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63094784->63098879] (req 63094784->63098879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.485289:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f6a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63094784->63098879] (req 63094784->63098879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.485953:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.485958:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f6a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63094784->63098879] (req 63094784->63098879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.485965:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f6a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63094784->63098879] (req 63094784->63098879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.485983:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f6a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63094784->63098879] (req 63094784->63098879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.485988:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f6a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63094784->63098879] (req 63094784->63098879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.486081:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554737152:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10179us (10499us total) trans 4294982713 rc 0/0 00000100:00100000:1.0:1789760776.486091:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15415 00000100:00100000:1.0:1789760776.486101:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719706023263 time 719721504867 state 0x2 ref 0 00000100:00100000:0.0:1789760776.493473:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.493931:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554737280 00000100:00100000:1.0:1789760776.493942:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554737280 00000100:00100000:1.0:1789760776.493993:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719706023263 time 719721504867 state 0x0 ref 1 00000100:00100000:1.0:1789760776.494009:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719721765283 time 719721504867 state 0x1 ref 1 00000100:00100000:1.0:1789760776.494038:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719721765283 time 719739656351 state 0x0 ref 1 00000100:00100000:1.0:1789760776.494050:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554737280:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.494096:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63098880, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e180 00010000:00010000:1.0:1789760776.494216:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f71 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.494233:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760776.494237:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.494244:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63098880->63102975] (req 63098880->63102975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.494258:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63098880->63102975] (req 63098880->63102975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.501276:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982714 is committed 00002000:00010000:0.0:1789760776.501497:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63098880 -> 63102976 00002000:00010000:0.0:1789760776.501503:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123264 -> 123272 00010000:00010000:0.0:1789760776.501516:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63098880->63102975] (req 63098880->63102975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.501530:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f71 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63098880->63102975] (req 63098880->63102975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.501540:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63098880->63102975] (req 63098880->63102975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.501548:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760776.501552:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63098880->63102975] (req 63098880->63102975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.501562:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f71 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63098880->63102975] (req 63098880->63102975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.501583:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f71 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63098880->63102975] (req 63098880->63102975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.501591:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c4f71 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63098880->63102975] (req 63098880->63102975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.501680:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554737280:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7627us (8212us total) trans 4294982714 rc 0/0 00000100:00100000:0.0:1789760776.501693:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15416 00000100:00100000:0.0:1789760776.501703:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719721765283 time 719739656351 state 0x2 ref 0 00000100:00100000:2.0:1789760776.508803:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760776.510899:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554737408 00000100:00100000:0.0:1789760776.510913:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554737408 00000100:00100000:0.0:1789760776.510974:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719721765283 time 719739656351 state 0x0 ref 1 00000100:00100000:0.0:1789760776.510990:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719739916767 time 719739656351 state 0x1 ref 1 00000100:00100000:0.0:1789760776.511021:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719739916767 time 719756639238 state 0x0 ref 1 00000100:00100000:0.0:1789760776.511032:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554737408:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760776.511096:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63102976, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e188 00010000:00010000:0.0:1789760776.511107:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f78 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.511123:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760776.511129:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.511137:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63102976->63107071] (req 63102976->63107071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.511153:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63102976->63107071] (req 63102976->63107071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.516949:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982715 is committed 00002000:00010000:0.0:1789760776.517153:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63102976 -> 63107072 00002000:00010000:0.0:1789760776.517159:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123272 -> 123280 00010000:00010000:0.0:1789760776.517172:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63102976->63107071] (req 63102976->63107071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.517182:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f78 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63102976->63107071] (req 63102976->63107071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.517191:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63102976->63107071] (req 63102976->63107071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.517198:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760776.517202:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63102976->63107071] (req 63102976->63107071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.517295:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f78 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63102976->63107071] (req 63102976->63107071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.517318:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f78 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63102976->63107071] (req 63102976->63107071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.517326:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f78 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63102976->63107071] (req 63102976->63107071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.517693:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554737408:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6658us (8895us total) trans 4294982715 rc 0/0 00000100:00100000:0.0:1789760776.517704:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15417 00000100:00100000:0.0:1789760776.517713:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719739916767 time 719756639238 state 0x2 ref 0 00000100:00100000:2.0:1789760776.524855:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.524953:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554737536 00000100:00100000:1.0:1789760776.524962:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554737536 00000100:00100000:1.0:1789760776.525016:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719739916767 time 719756639238 state 0x0 ref 1 00000100:00100000:1.0:1789760776.525030:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719756899654 time 719756639238 state 0x1 ref 1 00000100:00100000:1.0:1789760776.525062:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719756899654 time 719770680132 state 0x0 ref 1 00000100:00100000:1.0:1789760776.525075:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554737536:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.525128:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63107072, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e190 00010000:00010000:1.0:1789760776.525138:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4f7f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.525153:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760776.525157:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.525163:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4f7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63107072->63111167] (req 63107072->63111167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.525177:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4f7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63107072->63111167] (req 63107072->63111167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.530819:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982716 is committed 00002000:00010000:1.0:1789760776.530999:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63107072 -> 63111168 00002000:00010000:1.0:1789760776.531004:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123280 -> 123288 00010000:00010000:1.0:1789760776.531017:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4f7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63107072->63111167] (req 63107072->63111167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.532088:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4f7f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63107072->63111167] (req 63107072->63111167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.532098:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4f7f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63107072->63111167] (req 63107072->63111167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.532105:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.532111:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4f7f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63107072->63111167] (req 63107072->63111167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.532120:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4f7f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63107072->63111167] (req 63107072->63111167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.532141:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4f7f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63107072->63111167] (req 63107072->63111167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.532150:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4f7f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63107072->63111167] (req 63107072->63111167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.532252:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554737536:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7175us (7401us total) trans 4294982716 rc 0/0 00000100:00100000:1.0:1789760776.532263:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15418 00000100:00100000:1.0:1789760776.532272:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719756899654 time 719770680132 state 0x2 ref 0 00000100:00100000:2.0:1789760776.539610:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.539673:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554737664 00000100:00100000:1.0:1789760776.539683:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554737664 00000100:00100000:1.0:1789760776.539757:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719756899654 time 719770680132 state 0x0 ref 1 00000100:00100000:1.0:1789760776.539772:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719770940548 time 719770680132 state 0x1 ref 1 00000100:00100000:1.0:1789760776.539799:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719770940548 time 719785417208 state 0x0 ref 1 00000100:00100000:1.0:1789760776.539820:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554737664:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.539865:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63111168, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e198 00010000:00010000:1.0:1789760776.539874:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4f86 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.539891:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760776.539896:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.539902:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4f86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63111168->63115263] (req 63111168->63115263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.539915:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4f86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63111168->63115263] (req 63111168->63115263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.545349:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982717 is committed 00002000:00010000:1.0:1789760776.545817:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63111168 -> 63115264 00002000:00010000:1.0:1789760776.545823:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123288 -> 123296 00010000:00010000:1.0:1789760776.545836:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4f86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63111168->63115263] (req 63111168->63115263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.545849:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4f86 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63111168->63115263] (req 63111168->63115263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.545863:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4f86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63111168->63115263] (req 63111168->63115263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.545871:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.545876:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4f86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63111168->63115263] (req 63111168->63115263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.545885:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4f86 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63111168->63115263] (req 63111168->63115263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.545905:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4f86 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63111168->63115263] (req 63111168->63115263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.545912:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4f86 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63111168->63115263] (req 63111168->63115263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.546006:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554737664:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6183us (6399us total) trans 4294982717 rc 0/0 00000100:00100000:1.0:1789760776.546017:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15419 00000100:00100000:1.0:1789760776.546027:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719770940548 time 719785417208 state 0x2 ref 0 00000100:00100000:2.0:1789760776.554687:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760776.555042:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554737792 00000100:00100000:0.0:1789760776.555052:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554737792 00000100:00100000:0.0:1789760776.555105:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719770940548 time 719785417208 state 0x0 ref 1 00000100:00100000:0.0:1789760776.555122:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719785677624 time 719785417208 state 0x1 ref 1 00000100:00100000:0.0:1789760776.555155:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719785677624 time 719800773876 state 0x0 ref 1 00000100:00100000:0.0:1789760776.555167:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554737792:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760776.555213:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63115264, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e1a0 00010000:00010000:0.0:1789760776.555221:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f8d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.555237:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760776.555242:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.555248:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63115264->63119359] (req 63115264->63119359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.555261:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63115264->63119359] (req 63115264->63119359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.561142:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982718 is committed 00002000:00010000:0.0:1789760776.562962:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63115264 -> 63119360 00002000:00010000:0.0:1789760776.562970:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123296 -> 123304 00010000:00010000:0.0:1789760776.562983:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63115264->63119359] (req 63115264->63119359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.563000:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f8d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63115264->63119359] (req 63115264->63119359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.563010:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63115264->63119359] (req 63115264->63119359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.563017:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760776.563022:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63115264->63119359] (req 63115264->63119359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.563030:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f8d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63115264->63119359] (req 63115264->63119359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.563050:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f8d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63115264->63119359] (req 63115264->63119359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.563057:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f8d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63115264->63119359] (req 63115264->63119359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.563360:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554737792:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8188us (8685us total) trans 4294982718 rc 0/0 00000100:00100000:0.0:1789760776.563371:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15420 00000100:00100000:0.0:1789760776.563418:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719785677624 time 719800773876 state 0x2 ref 0 00000100:00100000:2.0:1789760776.571253:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760776.571511:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554737920 00000100:00100000:0.0:1789760776.571520:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554737920 00000100:00100000:0.0:1789760776.571572:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719785677624 time 719800773876 state 0x0 ref 1 00000100:00100000:0.0:1789760776.571588:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719801034292 time 719800773876 state 0x1 ref 1 00000100:00100000:0.0:1789760776.571621:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719801034292 time 719817239686 state 0x0 ref 1 00000100:00100000:0.0:1789760776.571634:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554737920:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760776.571680:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63119360, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e1a8 00010000:00010000:0.0:1789760776.571689:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f94 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.571704:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760776.571709:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.571715:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63119360->63123455] (req 63119360->63123455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.571741:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63119360->63123455] (req 63119360->63123455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.577240:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982719 is committed 00002000:00010000:0.0:1789760776.578511:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63119360 -> 63123456 00002000:00010000:0.0:1789760776.578519:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123304 -> 123312 00010000:00010000:0.0:1789760776.578532:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63119360->63123455] (req 63119360->63123455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.578544:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f94 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63119360->63123455] (req 63119360->63123455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.578560:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63119360->63123455] (req 63119360->63123455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.578567:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760776.578572:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63119360->63123455] (req 63119360->63123455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.578582:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f94 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63119360->63123455] (req 63119360->63123455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.578601:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f94 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63119360->63123455] (req 63119360->63123455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.578609:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c4f94 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63119360->63123455] (req 63119360->63123455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.578878:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554737920:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7239us (7628us total) trans 4294982719 rc 0/0 00000100:00100000:0.0:1789760776.578888:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15421 00000100:00100000:0.0:1789760776.578898:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719801034292 time 719817239686 state 0x2 ref 0 00000100:00100000:2.0:1789760776.585867:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.585967:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554738048 00000100:00100000:1.0:1789760776.585975:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554738048 00000100:00100000:1.0:1789760776.586027:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719801034292 time 719817239686 state 0x0 ref 1 00000100:00100000:1.0:1789760776.586193:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719817500102 time 719817239686 state 0x1 ref 1 00000100:00100000:1.0:1789760776.586284:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719817500102 time 719831900537 state 0x0 ref 1 00000100:00100000:1.0:1789760776.586298:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554738048:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.586345:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63123456, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e1b0 00010000:00010000:1.0:1789760776.586366:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4f9b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.586419:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760776.586424:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.586431:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4f9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63123456->63127551] (req 63123456->63127551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.586445:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4f9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63123456->63127551] (req 63123456->63127551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.592250:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982720 is committed 00002000:00010000:1.0:1789760776.592849:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63123456 -> 63127552 00002000:00010000:1.0:1789760776.592855:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123312 -> 123320 00010000:00010000:1.0:1789760776.592868:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4f9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63123456->63127551] (req 63123456->63127551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.592881:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4f9b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63123456->63127551] (req 63123456->63127551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.592891:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4f9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63123456->63127551] (req 63123456->63127551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.592905:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.592910:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4f9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63123456->63127551] (req 63123456->63127551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.592918:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4f9b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63123456->63127551] (req 63123456->63127551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.592937:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4f9b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63123456->63127551] (req 63123456->63127551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.592945:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4f9b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63123456->63127551] (req 63123456->63127551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.593254:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554738048:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6953us (7390us total) trans 4294982720 rc 0/0 00000100:00100000:1.0:1789760776.593265:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15422 00000100:00100000:1.0:1789760776.593274:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719817500102 time 719831900537 state 0x2 ref 0 00000100:00100000:2.0:1789760776.598699:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760776.598803:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554738176 00000100:00100000:0.0:1789760776.598813:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554738176 00000100:00100000:1.0:1789760776.599694:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719817500102 time 719831900537 state 0x0 ref 1 00000100:00100000:1.0:1789760776.599714:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719832160953 time 719831900537 state 0x1 ref 1 00000100:00100000:1.0:1789760776.599789:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719832160953 time 719845406745 state 0x0 ref 1 00000100:00100000:1.0:1789760776.599806:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554738176:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.599852:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63127552, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e1b8 00010000:00010000:1.0:1789760776.599863:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4fa2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.599879:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760776.599884:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.599904:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4fa2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63127552->63131647] (req 63127552->63131647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.599919:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4fa2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63127552->63131647] (req 63127552->63131647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.607579:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982721 is committed 00002000:00010000:2.0:1789760776.607891:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63127552 -> 63131648 00002000:00010000:2.0:1789760776.607897:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123320 -> 123328 00010000:00010000:2.0:1789760776.607912:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4fa2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63127552->63131647] (req 63127552->63131647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.607925:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4fa2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63127552->63131647] (req 63127552->63131647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.607933:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4fa2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63127552->63131647] (req 63127552->63131647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.607941:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760776.607946:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4fa2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63127552->63131647] (req 63127552->63131647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.607956:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4fa2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63127552->63131647] (req 63127552->63131647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.607976:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4fa2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63127552->63131647] (req 63127552->63131647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.607983:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c4fa2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63127552->63131647] (req 63127552->63131647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.608198:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554738176:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8391us (9501us total) trans 4294982721 rc 0/0 00000100:00100000:2.0:1789760776.608209:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15423 00000100:00100000:2.0:1789760776.608219:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719832160953 time 719845406745 state 0x2 ref 0 00000100:00100000:2.0:1789760776.614497:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760776.614585:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554738304 00000100:00100000:0.0:1789760776.614594:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554738304 00000100:00100000:0.0:1789760776.614648:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719832160953 time 719845406745 state 0x0 ref 1 00000100:00100000:0.0:1789760776.614663:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719845667161 time 719845406745 state 0x1 ref 1 00000100:00100000:0.0:1789760776.614695:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719845667161 time 719860313920 state 0x0 ref 1 00000100:00100000:0.0:1789760776.614709:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554738304:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760776.614775:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63131648, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e1c0 00010000:00010000:0.0:1789760776.614784:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fa9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.614799:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760776.614803:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.614810:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fa9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63131648->63135743] (req 63131648->63135743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.614825:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fa9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63131648->63135743] (req 63131648->63135743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.622956:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982722 is committed 00002000:00010000:0.0:1789760776.623080:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63131648 -> 63135744 00002000:00010000:0.0:1789760776.623085:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123328 -> 123336 00010000:00010000:0.0:1789760776.623271:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fa9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63131648->63135743] (req 63131648->63135743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.623285:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fa9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63131648->63135743] (req 63131648->63135743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.623297:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fa9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63131648->63135743] (req 63131648->63135743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.623303:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760776.623307:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fa9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63131648->63135743] (req 63131648->63135743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.623316:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fa9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63131648->63135743] (req 63131648->63135743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.623335:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fa9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63131648->63135743] (req 63131648->63135743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.623341:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fa9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63131648->63135743] (req 63131648->63135743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.624427:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554738304:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9713us (9933us total) trans 4294982722 rc 0/0 00000100:00100000:0.0:1789760776.624439:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15424 00000100:00100000:0.0:1789760776.624448:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719845667161 time 719860313920 state 0x2 ref 0 00000100:00100000:1.0:1789760776.629165:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760776.629449:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554738432 00000100:00100000:0.0:1789760776.629459:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554738432 00000100:00100000:0.0:1789760776.629520:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719845667161 time 719860313920 state 0x0 ref 1 00000100:00100000:0.0:1789760776.629535:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719860574336 time 719860313920 state 0x1 ref 1 00000100:00100000:0.0:1789760776.629565:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719860574336 time 719875183946 state 0x0 ref 1 00000100:00100000:0.0:1789760776.629577:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d83480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554738432:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760776.629620:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63135744, mtime: 0x6aad9507, atime: 0x0, ctime: 0x6aad9507, blocks: 0x1e1c8 00010000:00010000:0.0:1789760776.629629:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fb0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.629645:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760776.629649:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.629655:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63135744->63139839] (req 63135744->63139839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.629668:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63135744->63139839] (req 63135744->63139839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.641216:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982723 is committed 00002000:00010000:0.0:1789760776.641332:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63135744 -> 63139840 00002000:00010000:0.0:1789760776.641337:0:8442:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760775 -> 1789760776 00002000:00010000:0.0:1789760776.641339:0:8442:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760775 -> 1789760776 00002000:00010000:0.0:1789760776.641342:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123336 -> 123344 00010000:00010000:0.0:1789760776.641367:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63135744->63139839] (req 63135744->63139839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.641822:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fb0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63135744->63139839] (req 63135744->63139839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.641833:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fb0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63135744->63139839] (req 63135744->63139839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.641841:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760776.641846:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fb0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63135744->63139839] (req 63135744->63139839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.641855:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fb0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63135744->63139839] (req 63135744->63139839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.641875:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fb0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63135744->63139839] (req 63135744->63139839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.641883:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fb0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63135744->63139839] (req 63135744->63139839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.641982:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d83480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554738432:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12402us (12821us total) trans 4294982723 rc 0/0 00000100:00100000:0.0:1789760776.641993:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15425 00000100:00100000:0.0:1789760776.642003:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719860574336 time 719875183946 state 0x2 ref 0 00000100:00100000:3.0:1789760776.646526:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.647279:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554738560 00000100:00100000:1.0:1789760776.647290:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554738560 00000100:00100000:1.0:1789760776.647348:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719860574336 time 719875183946 state 0x0 ref 1 00000100:00100000:1.0:1789760776.647362:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719875444362 time 719875183946 state 0x1 ref 1 00000100:00100000:1.0:1789760776.647687:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719875444362 time 719893302693 state 0x0 ref 1 00000100:00100000:1.0:1789760776.647702:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f16a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554738560:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.647827:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63139840, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e1d0 00010000:00010000:1.0:1789760776.647837:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4fb7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.647854:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760776.647858:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.647864:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4fb7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63139840->63143935] (req 63139840->63143935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.647878:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4fb7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63139840->63143935] (req 63139840->63143935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.655614:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982724 is committed 00002000:00010000:1.0:1789760776.655868:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63139840 -> 63143936 00002000:00010000:1.0:1789760776.655873:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123344 -> 123352 00010000:00010000:1.0:1789760776.655887:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4fb7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63139840->63143935] (req 63139840->63143935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.655900:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4fb7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63139840->63143935] (req 63139840->63143935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.655909:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4fb7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63139840->63143935] (req 63139840->63143935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.655917:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.655921:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4fb7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63139840->63143935] (req 63139840->63143935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.655930:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4fb7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63139840->63143935] (req 63139840->63143935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.655955:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4fb7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63139840->63143935] (req 63139840->63143935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.655963:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4fb7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63139840->63143935] (req 63139840->63143935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.656062:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f16a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554738560:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8358us (9539us total) trans 4294982724 rc 0/0 00000100:00100000:1.0:1789760776.656072:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15426 00000100:00100000:1.0:1789760776.656081:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719875444362 time 719893302693 state 0x2 ref 0 00000100:00100000:2.0:1789760776.661205:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.661280:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554738688 00000100:00100000:1.0:1789760776.661289:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554738688 00000100:00100000:1.0:1789760776.661338:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719875444362 time 719893302693 state 0x0 ref 1 00000100:00100000:1.0:1789760776.661353:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719893563109 time 719893302693 state 0x1 ref 1 00000100:00100000:1.0:1789760776.661419:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719893563109 time 719907036391 state 0x0 ref 1 00000100:00100000:1.0:1789760776.661432:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554738688:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.661473:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63143936, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e1d8 00010000:00010000:1.0:1789760776.661492:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4fbe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.661506:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760776.661511:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.661518:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4fbe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63143936->63148031] (req 63143936->63148031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.661531:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4fbe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63143936->63148031] (req 63143936->63148031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.667810:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982725 is committed 00002000:00010000:1.0:1789760776.667958:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63143936 -> 63148032 00002000:00010000:1.0:1789760776.667963:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123352 -> 123360 00010000:00010000:1.0:1789760776.667976:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4fbe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63143936->63148031] (req 63143936->63148031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.667990:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4fbe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63143936->63148031] (req 63143936->63148031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.668155:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4fbe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63143936->63148031] (req 63143936->63148031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.668166:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.668171:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4fbe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63143936->63148031] (req 63143936->63148031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.668182:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4fbe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63143936->63148031] (req 63143936->63148031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.668201:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4fbe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63143936->63148031] (req 63143936->63148031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.668216:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c4fbe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63143936->63148031] (req 63143936->63148031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.668733:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554738688:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7288us (7521us total) trans 4294982725 rc 0/0 00000100:00100000:1.0:1789760776.668745:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15427 00000100:00100000:1.0:1789760776.668764:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719893563109 time 719907036391 state 0x2 ref 0 00000100:00100000:2.0:1789760776.672329:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760776.672519:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554738816 00000100:00100000:0.0:1789760776.672528:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554738816 00000100:00100000:0.0:1789760776.672581:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719893563109 time 719907036391 state 0x0 ref 1 00000100:00100000:0.0:1789760776.672595:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719907296807 time 719907036391 state 0x1 ref 1 00000100:00100000:0.0:1789760776.672625:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719907296807 time 719918242769 state 0x0 ref 1 00000100:00100000:0.0:1789760776.672639:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554738816:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760776.672687:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63148032, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e1e0 00010000:00010000:0.0:1789760776.672696:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fc5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.672712:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760776.672717:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.672734:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fc5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63148032->63152127] (req 63148032->63152127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.672748:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fc5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63148032->63152127] (req 63148032->63152127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.680468:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982726 is committed 00002000:00010000:0.0:1789760776.680669:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63148032 -> 63152128 00002000:00010000:0.0:1789760776.680674:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123360 -> 123368 00010000:00010000:0.0:1789760776.680689:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fc5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63148032->63152127] (req 63148032->63152127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.680702:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fc5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63148032->63152127] (req 63148032->63152127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.680711:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fc5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63148032->63152127] (req 63148032->63152127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.680719:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760776.680908:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fc5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63148032->63152127] (req 63148032->63152127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.680919:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fc5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63148032->63152127] (req 63148032->63152127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.680941:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fc5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63148032->63152127] (req 63148032->63152127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.680949:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fc5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63148032->63152127] (req 63148032->63152127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.682687:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554738816:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10036us (10354us total) trans 4294982726 rc 0/0 00000100:00100000:0.0:1789760776.682700:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15428 00000100:00100000:0.0:1789760776.682711:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719907296807 time 719918242769 state 0x2 ref 0 00000100:00100000:3.0:1789760776.687761:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760776.687845:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554738944 00000100:00100000:0.0:1789760776.687854:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554738944 00000100:00100000:0.0:1789760776.687903:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719907296807 time 719918242769 state 0x0 ref 1 00000100:00100000:0.0:1789760776.687916:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719918503185 time 719918242769 state 0x1 ref 1 00000100:00100000:0.0:1789760776.687946:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719918503185 time 719933565140 state 0x0 ref 1 00000100:00100000:0.0:1789760776.687957:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f15f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554738944:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760776.688004:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63152128, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e1e8 00010000:00010000:0.0:1789760776.688014:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fcc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.688030:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760776.688035:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.688041:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fcc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63152128->63156223] (req 63152128->63156223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.688054:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fcc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63152128->63156223] (req 63152128->63156223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.694138:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982727 is committed 00002000:00010000:0.0:1789760776.694258:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63152128 -> 63156224 00002000:00010000:0.0:1789760776.694263:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123368 -> 123376 00010000:00010000:0.0:1789760776.694276:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fcc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63152128->63156223] (req 63152128->63156223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.694289:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fcc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63152128->63156223] (req 63152128->63156223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.694298:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fcc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63152128->63156223] (req 63152128->63156223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.694305:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760776.694310:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fcc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63152128->63156223] (req 63152128->63156223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.694318:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fcc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63152128->63156223] (req 63152128->63156223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.694337:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fcc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63152128->63156223] (req 63152128->63156223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.694345:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c4fcc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63152128->63156223] (req 63152128->63156223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.694497:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f15f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554738944:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6522us (6725us total) trans 4294982727 rc 0/0 00000100:00100000:0.0:1789760776.694507:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15429 00000100:00100000:0.0:1789760776.694516:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719918503185 time 719933565140 state 0x2 ref 0 00000100:00100000:1.0:1789760776.698023:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760776.698272:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554739072 00000100:00100000:2.0:1789760776.698281:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554739072 00000100:00100000:2.0:1789760776.698331:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719918503185 time 719933565140 state 0x0 ref 1 00000100:00100000:2.0:1789760776.698345:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719933825556 time 719933565140 state 0x1 ref 1 00000100:00100000:0.0:1789760776.698476:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719933825556 time 719944092994 state 0x0 ref 1 00000100:00100000:0.0:1789760776.698589:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554739072:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760776.698641:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63156224, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e1f0 00010000:00010000:0.0:1789760776.698651:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4fd3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.698669:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760776.698674:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.698681:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4fd3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63156224->63160319] (req 63156224->63160319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.698695:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4fd3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63156224->63160319] (req 63156224->63160319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.703697:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982728 is committed 00002000:00010000:0.0:1789760776.703814:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63156224 -> 63160320 00002000:00010000:0.0:1789760776.703819:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123376 -> 123384 00010000:00010000:0.0:1789760776.703831:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4fd3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63156224->63160319] (req 63156224->63160319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.703844:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4fd3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63156224->63160319] (req 63156224->63160319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.703854:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4fd3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63156224->63160319] (req 63156224->63160319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.703862:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760776.703867:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4fd3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63156224->63160319] (req 63156224->63160319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.703876:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4fd3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63156224->63160319] (req 63156224->63160319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.703895:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4fd3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63156224->63160319] (req 63156224->63160319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.703904:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4fd3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63156224->63160319] (req 63156224->63160319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.703994:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554739072:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5405us (5976us total) trans 4294982728 rc 0/0 00000100:00100000:0.0:1789760776.704005:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15430 00000100:00100000:0.0:1789760776.704014:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719933825556 time 719944092994 state 0x2 ref 0 00000100:00100000:1.0:1789760776.708066:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760776.708471:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554739200 00000100:00100000:2.0:1789760776.708489:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554739200 00000100:00100000:2.0:1789760776.708541:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719933825556 time 719944092994 state 0x0 ref 1 00000100:00100000:2.0:1789760776.708555:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719944353410 time 719944092994 state 0x1 ref 1 00000100:00100000:2.0:1789760776.708670:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719944353410 time 719954286609 state 0x0 ref 1 00000100:00100000:2.0:1789760776.708685:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d80700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554739200:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760776.708740:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63160320, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e1f8 00010000:00010000:2.0:1789760776.708748:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4fda lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.708764:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760776.708768:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.708774:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4fda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63160320->63164415] (req 63160320->63164415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.708786:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4fda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63160320->63164415] (req 63160320->63164415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.714121:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982729 is committed 00002000:00010000:1.0:1789760776.714533:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63160320 -> 63164416 00002000:00010000:1.0:1789760776.714539:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123384 -> 123392 00010000:00010000:1.0:1789760776.714552:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4fda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63160320->63164415] (req 63160320->63164415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.714566:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4fda lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63160320->63164415] (req 63160320->63164415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.714576:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4fda lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63160320->63164415] (req 63160320->63164415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.714584:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.714589:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4fda lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63160320->63164415] (req 63160320->63164415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.714600:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4fda lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63160320->63164415] (req 63160320->63164415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.714619:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4fda lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63160320->63164415] (req 63160320->63164415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.714626:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c4fda lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63160320->63164415] (req 63160320->63164415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.714764:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d80700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554739200:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6077us (6701us total) trans 4294982729 rc 0/0 00000100:00100000:1.0:1789760776.714775:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15431 00000100:00100000:1.0:1789760776.714784:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719944353410 time 719954286609 state 0x2 ref 0 00000100:00100000:3.0:1789760776.719466:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760776.719682:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554739328 00000100:00100000:2.0:1789760776.719691:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554739328 00000100:00100000:2.0:1789760776.719752:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719944353410 time 719954286609 state 0x0 ref 1 00000100:00100000:2.0:1789760776.719769:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719954547025 time 719954286609 state 0x1 ref 1 00000100:00100000:2.0:1789760776.719790:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719954547025 time 719965408768 state 0x0 ref 1 00000100:00100000:2.0:1789760776.719802:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554739328:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760776.719848:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63164416, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e200 00010000:00010000:2.0:1789760776.719858:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4fe1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.719874:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:2.0:1789760776.719878:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.719885:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4fe1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63164416->63168511] (req 63164416->63168511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.719899:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4fe1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63164416->63168511] (req 63164416->63168511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760776.726107:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982730 is committed 00002000:00010000:2.0:1789760776.726801:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63164416 -> 63168512 00002000:00010000:2.0:1789760776.726808:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123392 -> 123400 00010000:00010000:2.0:1789760776.726820:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4fe1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63164416->63168511] (req 63164416->63168511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.726833:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4fe1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63164416->63168511] (req 63164416->63168511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.726842:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4fe1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63164416->63168511] (req 63164416->63168511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.726856:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760776.726861:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4fe1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63164416->63168511] (req 63164416->63168511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.726870:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4fe1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63164416->63168511] (req 63164416->63168511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.726891:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4fe1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63164416->63168511] (req 63164416->63168511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.726899:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c4fe1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63164416->63168511] (req 63164416->63168511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.727045:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554739328:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7239us (7583us total) trans 4294982730 rc 0/0 00000100:00100000:2.0:1789760776.727057:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15432 00000100:00100000:2.0:1789760776.727066:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719954547025 time 719965408768 state 0x2 ref 0 00000100:00100000:2.0:1789760776.740025:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.740132:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554739456 00000100:00100000:1.0:1789760776.740143:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554739456 00000100:00100000:1.0:1789760776.740202:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719954547025 time 719965408768 state 0x0 ref 1 00000100:00100000:1.0:1789760776.740218:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719965669184 time 719965408768 state 0x1 ref 1 00000100:00100000:1.0:1789760776.740288:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719965669184 time 719985905747 state 0x0 ref 1 00000100:00100000:1.0:1789760776.740300:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04562300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554739456:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.740347:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63168512, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e208 00010000:00010000:1.0:1789760776.740357:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4fe8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.740372:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:1.0:1789760776.740416:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.740424:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4fe8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63168512->63172607] (req 63168512->63172607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.740438:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4fe8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63168512->63172607] (req 63168512->63172607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.744645:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982731 is committed 00002000:00010000:1.0:1789760776.744756:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63168512 -> 63172608 00002000:00010000:1.0:1789760776.744760:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123400 -> 123408 00010000:00010000:1.0:1789760776.744781:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4fe8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63168512->63172607] (req 63168512->63172607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.744794:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4fe8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63168512->63172607] (req 63168512->63172607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.744804:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4fe8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63168512->63172607] (req 63168512->63172607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.744811:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.744817:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4fe8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63168512->63172607] (req 63168512->63172607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.744826:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4fe8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63168512->63172607] (req 63168512->63172607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.744844:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4fe8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63168512->63172607] (req 63168512->63172607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.744851:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4fe8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63168512->63172607] (req 63168512->63172607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.744941:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04562300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554739456:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4637us (4920us total) trans 4294982731 rc 0/0 00000100:00100000:1.0:1789760776.744953:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15433 00000100:00100000:1.0:1789760776.744963:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719965669184 time 719985905747 state 0x2 ref 0 00000100:00100000:1.0:1789760776.750255:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760776.750315:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554739584 00000100:00100000:0.0:1789760776.750325:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554739584 00000100:00100000:0.0:1789760776.750393:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719965669184 time 719985905747 state 0x0 ref 1 00000100:00100000:0.0:1789760776.750409:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719986166163 time 719985905747 state 0x1 ref 1 00000100:00100000:0.0:1789760776.750444:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719986166163 time 719996058323 state 0x0 ref 1 00000100:00100000:0.0:1789760776.750456:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d80380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554739584:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760776.750498:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63172608, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e210 00010000:00010000:0.0:1789760776.750508:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4fef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.750522:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=719s v=(0 0) 00010000:00010000:0.0:1789760776.750526:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.750531:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4fef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63172608->63176703] (req 63172608->63176703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.750545:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4fef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63172608->63176703] (req 63172608->63176703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.757040:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982732 is committed 00002000:00010000:0.0:1789760776.757177:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63172608 -> 63176704 00002000:00010000:0.0:1789760776.757183:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123408 -> 123416 00010000:00010000:0.0:1789760776.757197:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4fef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63172608->63176703] (req 63172608->63176703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.757210:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4fef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63172608->63176703] (req 63172608->63176703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.757219:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4fef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63172608->63176703] (req 63172608->63176703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.757226:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760776.757230:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4fef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63172608->63176703] (req 63172608->63176703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.757240:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4fef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63172608->63176703] (req 63172608->63176703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.757261:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4fef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63172608->63176703] (req 63172608->63176703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.757269:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c4fef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63172608->63176703] (req 63172608->63176703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.757480:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d80380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554739584:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7020us (7229us total) trans 4294982732 rc 0/0 00000100:00100000:0.0:1789760776.757490:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15434 00000100:00100000:0.0:1789760776.757499:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719986166163 time 719996058323 state 0x2 ref 0 00000100:00100000:0.0:1789760776.767815:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.767898:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554739712 00000100:00100000:1.0:1789760776.767908:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554739712 00000100:00100000:1.0:1789760776.767959:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719986166163 time 719996058323 state 0x0 ref 1 00000100:00100000:1.0:1789760776.767973:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719996318739 time 719996058323 state 0x1 ref 1 00000100:00100000:1.0:1789760776.768001:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719996318739 time 720013620153 state 0x0 ref 1 00000100:00100000:1.0:1789760776.768013:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554739712:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.768056:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63176704, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e218 00010000:00010000:1.0:1789760776.768064:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ff6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.768084:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760776.768088:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.768094:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ff6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63176704->63180799] (req 63176704->63180799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.768105:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ff6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63176704->63180799] (req 63176704->63180799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760776.772342:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982733 is committed 00002000:00010000:1.0:1789760776.772471:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63176704 -> 63180800 00002000:00010000:1.0:1789760776.772476:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123416 -> 123424 00010000:00010000:1.0:1789760776.772490:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ff6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63176704->63180799] (req 63176704->63180799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.772503:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ff6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63176704->63180799] (req 63176704->63180799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.772513:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ff6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63176704->63180799] (req 63176704->63180799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.772521:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.772526:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ff6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63176704->63180799] (req 63176704->63180799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.772536:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ff6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63176704->63180799] (req 63176704->63180799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.772560:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ff6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63176704->63180799] (req 63176704->63180799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.772568:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c4ff6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63176704->63180799] (req 63176704->63180799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.772730:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554739712:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4703us (4906us total) trans 4294982733 rc 0/0 00000100:00100000:1.0:1789760776.772741:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15435 00000100:00100000:1.0:1789760776.772751:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719996318739 time 720013620153 state 0x2 ref 0 00000100:00100000:2.0:1789760776.779622:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.779699:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554739840 00000100:00100000:1.0:1789760776.779709:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554739840 00000100:00100000:1.0:1789760776.779776:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 719996318739 time 720013620153 state 0x0 ref 1 00000100:00100000:1.0:1789760776.779791:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720013880569 time 720013620153 state 0x1 ref 1 00000100:00100000:1.0:1789760776.779820:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720013880569 time 720025438896 state 0x0 ref 1 00000100:00100000:1.0:1789760776.779832:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04560000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554739840:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.779880:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63180800, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e220 00010000:00010000:1.0:1789760776.779888:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4ffd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.779904:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760776.779909:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.779916:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4ffd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63180800->63184895] (req 63180800->63184895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.779930:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4ffd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63180800->63184895] (req 63180800->63184895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.786239:0:6508:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760776.790860:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982734 is committed 00002000:00010000:1.0:1789760776.792558:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63180800 -> 63184896 00002000:00010000:1.0:1789760776.792567:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123424 -> 123432 00010000:00010000:1.0:1789760776.792582:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4ffd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63180800->63184895] (req 63180800->63184895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.792595:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4ffd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63180800->63184895] (req 63180800->63184895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.792604:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4ffd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63180800->63184895] (req 63180800->63184895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.792611:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.792616:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4ffd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63180800->63184895] (req 63180800->63184895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.792625:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4ffd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63180800->63184895] (req 63180800->63184895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.792643:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4ffd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63180800->63184895] (req 63180800->63184895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.792655:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c4ffd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63180800->63184895] (req 63180800->63184895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.792931:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04560000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554739840:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13095us (13315us total) trans 4294982734 rc 0/0 00000100:00100000:1.0:1789760776.792941:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15436 00000100:00100000:1.0:1789760776.792950:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720013880569 time 720025438896 state 0x2 ref 0 00000100:00100000:0.0:1789760776.797264:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760776.797361:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554739968 00000100:00100000:3.0:1789760776.797371:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554739968 00000100:00100000:2.0:1789760776.801341:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720013880569 time 720025438896 state 0x0 ref 1 00000100:00100000:2.0:1789760776.801360:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720025699312 time 720025438896 state 0x1 ref 1 00000100:00100000:2.0:1789760776.801434:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720025699312 time 720047051924 state 0x0 ref 1 00000100:00100000:2.0:1789760776.801449:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb065a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554739968:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760776.801496:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63184896, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e228 00010000:00010000:2.0:1789760776.801507:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5004 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.801523:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760776.801528:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.801534:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5004 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63184896->63188991] (req 63184896->63188991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.801549:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5004 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63184896->63188991] (req 63184896->63188991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.808688:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982735 is committed 00002000:00010000:2.0:1789760776.808795:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63184896 -> 63188992 00002000:00010000:2.0:1789760776.808799:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123432 -> 123440 00010000:00010000:2.0:1789760776.808810:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5004 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63184896->63188991] (req 63184896->63188991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.808827:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5004 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63184896->63188991] (req 63184896->63188991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.808833:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5004 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63184896->63188991] (req 63184896->63188991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.808838:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760776.808842:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5004 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63184896->63188991] (req 63184896->63188991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.808848:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5004 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63184896->63188991] (req 63184896->63188991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.808862:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5004 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63184896->63188991] (req 63184896->63188991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.808867:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5004 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63184896->63188991] (req 63184896->63188991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.808958:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554739968:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7508us (11698us total) trans 4294982735 rc 0/0 00000100:00100000:2.0:1789760776.808969:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15437 00000100:00100000:2.0:1789760776.808980:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720025699312 time 720047051924 state 0x2 ref 0 00000100:00100000:0.0:1789760776.815361:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760776.816668:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554740096 00000100:00100000:2.0:1789760776.816735:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554740096 00000100:00100000:0.0:1789760776.816866:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720025699312 time 720047051924 state 0x0 ref 1 00000100:00100000:0.0:1789760776.816881:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720047312340 time 720047051924 state 0x1 ref 1 00000100:00100000:0.0:1789760776.816915:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720047312340 time 720062533928 state 0x0 ref 1 00000100:00100000:0.0:1789760776.816928:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb065a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554740096:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760776.816977:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63188992, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e230 00010000:00010000:0.0:1789760776.816986:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c500b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.817002:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760776.817006:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.817011:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c500b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63188992->63193087] (req 63188992->63193087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.817027:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c500b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63188992->63193087] (req 63188992->63193087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.821996:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982736 is committed 00002000:00010000:0.0:1789760776.822109:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63188992 -> 63193088 00002000:00010000:0.0:1789760776.822114:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123440 -> 123448 00010000:00010000:0.0:1789760776.822128:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c500b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63188992->63193087] (req 63188992->63193087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.822141:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c500b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63188992->63193087] (req 63188992->63193087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.822149:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c500b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63188992->63193087] (req 63188992->63193087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.822155:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760776.822159:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c500b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63188992->63193087] (req 63188992->63193087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.822168:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c500b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63188992->63193087] (req 63188992->63193087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.822187:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c500b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63188992->63193087] (req 63188992->63193087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.822208:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c500b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63188992->63193087] (req 63188992->63193087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.822302:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554740096:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5372us (6946us total) trans 4294982736 rc 0/0 00000100:00100000:0.0:1789760776.822312:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15438 00000100:00100000:0.0:1789760776.822321:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720047312340 time 720062533928 state 0x2 ref 0 00000100:00100000:1.0:1789760776.828118:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760776.828212:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554740224 00000100:00100000:2.0:1789760776.828223:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554740224 00000100:00100000:2.0:1789760776.828276:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720047312340 time 720062533928 state 0x0 ref 1 00000100:00100000:2.0:1789760776.828291:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720062794344 time 720062533928 state 0x1 ref 1 00000100:00100000:2.0:1789760776.828324:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720062794344 time 720073941985 state 0x0 ref 1 00000100:00100000:2.0:1789760776.828339:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554740224:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760776.828413:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63193088, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e238 00010000:00010000:2.0:1789760776.828422:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5012 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.828521:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760776.828526:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.828533:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5012 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63193088->63197183] (req 63193088->63197183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.828548:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5012 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63193088->63197183] (req 63193088->63197183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.833201:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982737 is committed 00002000:00010000:0.0:1789760776.833356:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63193088 -> 63197184 00002000:00010000:0.0:1789760776.833362:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123448 -> 123456 00010000:00010000:0.0:1789760776.833375:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5012 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63193088->63197183] (req 63193088->63197183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.833400:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5012 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63193088->63197183] (req 63193088->63197183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.833409:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5012 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63193088->63197183] (req 63193088->63197183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.833417:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760776.833421:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5012 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63193088->63197183] (req 63193088->63197183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.833430:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5012 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63193088->63197183] (req 63193088->63197183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.833448:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5012 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63193088->63197183] (req 63193088->63197183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.833456:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5012 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63193088->63197183] (req 63193088->63197183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.833778:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554740224:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5436us (5664us total) trans 4294982737 rc 0/0 00000100:00100000:0.0:1789760776.833789:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15439 00000100:00100000:0.0:1789760776.833799:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720062794344 time 720073941985 state 0x2 ref 0 00000100:00100000:0.0:1789760776.838524:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.838677:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554740352 00000100:00100000:1.0:1789760776.838684:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554740352 00000100:00100000:1.0:1789760776.838761:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720062794344 time 720073941985 state 0x0 ref 1 00000100:00100000:1.0:1789760776.838778:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720074202401 time 720073941985 state 0x1 ref 1 00000100:00100000:1.0:1789760776.838803:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720074202401 time 720084421468 state 0x0 ref 1 00000100:00100000:1.0:1789760776.838816:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb065a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554740352:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.839022:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63197184, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e240 00010000:00010000:1.0:1789760776.839032:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5019 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.839046:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760776.839049:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.839053:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5019 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63197184->63201279] (req 63197184->63201279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.839064:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5019 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63197184->63201279] (req 63197184->63201279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.843718:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982738 is committed 00002000:00010000:1.0:1789760776.843862:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63197184 -> 63201280 00002000:00010000:1.0:1789760776.843868:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123456 -> 123464 00010000:00010000:1.0:1789760776.843881:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5019 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63197184->63201279] (req 63197184->63201279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.843893:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5019 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63197184->63201279] (req 63197184->63201279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.843902:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5019 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63197184->63201279] (req 63197184->63201279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.843909:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.843913:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5019 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63197184->63201279] (req 63197184->63201279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.843922:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5019 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63197184->63201279] (req 63197184->63201279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.843942:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5019 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63197184->63201279] (req 63197184->63201279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.843950:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5019 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63197184->63201279] (req 63197184->63201279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.844082:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554740352:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5264us (5703us total) trans 4294982738 rc 0/0 00000100:00100000:1.0:1789760776.844092:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15440 00000100:00100000:1.0:1789760776.844101:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720074202401 time 720084421468 state 0x2 ref 0 00000100:00100000:0.0:1789760776.850009:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.851050:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554740480 00000100:00100000:1.0:1789760776.851061:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554740480 00000100:00100000:1.0:1789760776.851118:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720074202401 time 720084421468 state 0x0 ref 1 00000100:00100000:1.0:1789760776.851132:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720084681884 time 720084421468 state 0x1 ref 1 00000100:00100000:1.0:1789760776.851334:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720084681884 time 720096951802 state 0x0 ref 1 00000100:00100000:1.0:1789760776.851347:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb065a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554740480:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.851423:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63201280, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e248 00010000:00010000:1.0:1789760776.851432:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5020 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.851450:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760776.851454:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.851460:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5020 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63201280->63205375] (req 63201280->63205375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.851472:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5020 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63201280->63205375] (req 63201280->63205375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.858247:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982739 is committed 00002000:00010000:1.0:1789760776.858368:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63201280 -> 63205376 00002000:00010000:1.0:1789760776.858373:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123464 -> 123472 00010000:00010000:1.0:1789760776.858412:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5020 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63201280->63205375] (req 63201280->63205375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.858425:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5020 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63201280->63205375] (req 63201280->63205375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.858435:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5020 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63201280->63205375] (req 63201280->63205375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.858441:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.858447:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5020 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63201280->63205375] (req 63201280->63205375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.858457:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5020 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63201280->63205375] (req 63201280->63205375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.858478:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5020 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63201280->63205375] (req 63201280->63205375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.858487:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5020 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63201280->63205375] (req 63201280->63205375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.859316:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554740480:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7961us (9307us total) trans 4294982739 rc 0/0 00000100:00100000:1.0:1789760776.859329:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15441 00000100:00100000:1.0:1789760776.859338:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720084681884 time 720096951802 state 0x2 ref 0 00000100:00100000:2.0:1789760776.865274:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760776.865777:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554740608 00000100:00100000:0.0:1789760776.865788:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554740608 00000100:00100000:0.0:1789760776.865851:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720084681884 time 720096951802 state 0x0 ref 1 00000100:00100000:0.0:1789760776.865866:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720097212218 time 720096951802 state 0x1 ref 1 00000100:00100000:0.0:1789760776.865894:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720097212218 time 720111512316 state 0x0 ref 1 00000100:00100000:0.0:1789760776.865907:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04560a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554740608:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760776.865954:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63205376, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e250 00010000:00010000:0.0:1789760776.865963:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5027 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.865979:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760776.865983:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.865989:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5027 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63205376->63209471] (req 63205376->63209471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.866003:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5027 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63205376->63209471] (req 63205376->63209471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.871256:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982740 is committed 00002000:00010000:2.0:1789760776.871429:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63205376 -> 63209472 00002000:00010000:2.0:1789760776.871433:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123472 -> 123480 00010000:00010000:2.0:1789760776.871445:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5027 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63205376->63209471] (req 63205376->63209471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.871467:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5027 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63205376->63209471] (req 63205376->63209471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.871475:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5027 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63205376->63209471] (req 63205376->63209471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.871482:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760776.871486:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5027 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63205376->63209471] (req 63205376->63209471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.871493:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5027 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63205376->63209471] (req 63205376->63209471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.871511:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5027 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63205376->63209471] (req 63205376->63209471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.871517:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5027 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63205376->63209471] (req 63205376->63209471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.871709:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04560a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554740608:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5799us (6440us total) trans 4294982740 rc 0/0 00000100:00100000:2.0:1789760776.871731:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15442 00000100:00100000:2.0:1789760776.871743:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720097212218 time 720111512316 state 0x2 ref 0 00000100:00100000:0.0:1789760776.878041:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.878214:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554740736 00000100:00100000:1.0:1789760776.878225:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554740736 00000100:00100000:1.0:1789760776.878285:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720097212218 time 720111512316 state 0x0 ref 1 00000100:00100000:1.0:1789760776.878302:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720111772732 time 720111512316 state 0x1 ref 1 00000100:00100000:1.0:1789760776.878332:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720111772732 time 720123951261 state 0x0 ref 1 00000100:00100000:1.0:1789760776.878343:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb065a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554740736:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.878467:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63209472, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e258 00010000:00010000:1.0:1789760776.878476:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c502e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.878490:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760776.878495:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.878500:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c502e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63209472->63213567] (req 63209472->63213567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.878513:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c502e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63209472->63213567] (req 63209472->63213567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.884365:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982741 is committed 00002000:00010000:1.0:1789760776.884650:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63209472 -> 63213568 00002000:00010000:1.0:1789760776.884655:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123480 -> 123488 00010000:00010000:1.0:1789760776.884669:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c502e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63209472->63213567] (req 63209472->63213567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.884681:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c502e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63209472->63213567] (req 63209472->63213567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.884689:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c502e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63209472->63213567] (req 63209472->63213567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.884696:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.884700:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c502e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63209472->63213567] (req 63209472->63213567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.884708:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c502e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63209472->63213567] (req 63209472->63213567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.884938:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c502e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63209472->63213567] (req 63209472->63213567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.884947:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c502e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63209472->63213567] (req 63209472->63213567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.886089:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554740736:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7739us (8050us total) trans 4294982741 rc 0/0 00000100:00100000:1.0:1789760776.886108:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15443 00000100:00100000:1.0:1789760776.886125:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720111772732 time 720123951261 state 0x2 ref 0 00000100:00100000:0.0:1789760776.896168:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.896282:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554740864 00000100:00100000:1.0:1789760776.896294:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554740864 00000100:00100000:1.0:1789760776.896345:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720111772732 time 720123951261 state 0x0 ref 1 00000100:00100000:1.0:1789760776.896360:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720124211677 time 720123951261 state 0x1 ref 1 00000100:00100000:1.0:1789760776.896423:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720124211677 time 720142040589 state 0x0 ref 1 00000100:00100000:1.0:1789760776.896435:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb065a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554740864:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.896486:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63213568, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e260 00010000:00010000:1.0:1789760776.896496:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5035 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.896513:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760776.896517:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.896524:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5035 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63213568->63217663] (req 63213568->63217663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.896538:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5035 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63213568->63217663] (req 63213568->63217663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.902367:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982742 is committed 00002000:00010000:1.0:1789760776.902654:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63213568 -> 63217664 00002000:00010000:1.0:1789760776.902660:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123488 -> 123496 00010000:00010000:1.0:1789760776.902673:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5035 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63213568->63217663] (req 63213568->63217663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.902693:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5035 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63213568->63217663] (req 63213568->63217663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.902703:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5035 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63213568->63217663] (req 63213568->63217663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.902710:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.902716:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5035 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63213568->63217663] (req 63213568->63217663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.902739:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5035 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63213568->63217663] (req 63213568->63217663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.902758:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5035 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63213568->63217663] (req 63213568->63217663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.902766:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5035 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63213568->63217663] (req 63213568->63217663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.902870:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554740864:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6432us (6709us total) trans 4294982742 rc 0/0 00000100:00100000:1.0:1789760776.902883:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15444 00000100:00100000:1.0:1789760776.902893:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720124211677 time 720142040589 state 0x2 ref 0 00000100:00100000:2.0:1789760776.907241:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.907668:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554740992 00000100:00100000:1.0:1789760776.907679:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554740992 00000100:00100000:1.0:1789760776.907753:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720124211677 time 720142040589 state 0x0 ref 1 00000100:00100000:1.0:1789760776.907768:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720142301005 time 720142040589 state 0x1 ref 1 00000100:00100000:0.0:1789760776.907962:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720142301005 time 720153577907 state 0x0 ref 1 00000100:00100000:0.0:1789760776.907981:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04563800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554740992:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760776.908036:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63217664, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e268 00010000:00010000:0.0:1789760776.908047:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c503c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.908065:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760776.908069:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760776.908075:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c503c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63217664->63221759] (req 63217664->63221759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.908089:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c503c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63217664->63221759] (req 63217664->63221759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.913430:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982743 is committed 00002000:00010000:0.0:1789760776.913663:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63217664 -> 63221760 00002000:00010000:0.0:1789760776.913668:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123496 -> 123504 00010000:00010000:0.0:1789760776.913681:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c503c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63217664->63221759] (req 63217664->63221759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.913693:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c503c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63217664->63221759] (req 63217664->63221759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.913703:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c503c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63217664->63221759] (req 63217664->63221759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.913717:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760776.913732:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c503c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63217664->63221759] (req 63217664->63221759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.913742:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c503c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63217664->63221759] (req 63217664->63221759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.913761:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c503c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63217664->63221759] (req 63217664->63221759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760776.913768:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c503c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63217664->63221759] (req 63217664->63221759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760776.913898:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04563800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554740992:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5916us (6661us total) trans 4294982743 rc 0/0 00000100:00100000:0.0:1789760776.913908:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15445 00000100:00100000:0.0:1789760776.913929:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720142301005 time 720153577907 state 0x2 ref 0 00000100:00100000:0.0:1789760776.919372:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760776.919540:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554741120 00000100:00100000:2.0:1789760776.919550:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554741120 00000100:00100000:2.0:1789760776.919606:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720142301005 time 720153577907 state 0x0 ref 1 00000100:00100000:2.0:1789760776.919623:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720153838323 time 720153577907 state 0x1 ref 1 00000100:00100000:2.0:1789760776.919658:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720153838323 time 720165276482 state 0x0 ref 1 00000100:00100000:2.0:1789760776.919671:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb065a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554741120:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760776.919718:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63221760, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e270 00010000:00010000:2.0:1789760776.919737:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5043 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.919751:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760776.919757:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.919763:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5043 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63221760->63225855] (req 63221760->63225855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.919778:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5043 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63221760->63225855] (req 63221760->63225855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.924163:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982744 is committed 00002000:00010000:2.0:1789760776.924304:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63221760 -> 63225856 00002000:00010000:2.0:1789760776.924309:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123504 -> 123512 00010000:00010000:2.0:1789760776.924323:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5043 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63221760->63225855] (req 63221760->63225855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.924336:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5043 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63221760->63225855] (req 63221760->63225855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.924346:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5043 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63221760->63225855] (req 63221760->63225855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.924360:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760776.924365:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5043 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63221760->63225855] (req 63221760->63225855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.924399:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5043 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63221760->63225855] (req 63221760->63225855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.924420:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5043 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63221760->63225855] (req 63221760->63225855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.924428:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5043 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63221760->63225855] (req 63221760->63225855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.924529:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554741120:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4855us (5161us total) trans 4294982744 rc 0/0 00000100:00100000:2.0:1789760776.924539:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15446 00000100:00100000:2.0:1789760776.924549:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720153838323 time 720165276482 state 0x2 ref 0 00000100:00100000:2.0:1789760776.930370:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.930505:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554741248 00000100:00100000:1.0:1789760776.930517:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554741248 00000100:00100000:1.0:1789760776.930577:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720153838323 time 720165276482 state 0x0 ref 1 00000100:00100000:1.0:1789760776.930592:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720165536898 time 720165276482 state 0x1 ref 1 00000100:00100000:1.0:1789760776.930627:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720165536898 time 720176245723 state 0x0 ref 1 00000100:00100000:1.0:1789760776.930640:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04563480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554741248:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.930687:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63225856, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e278 00010000:00010000:1.0:1789760776.930696:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c504a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.930713:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760776.930717:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.930734:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c504a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63225856->63229951] (req 63225856->63229951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.930749:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c504a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63225856->63229951] (req 63225856->63229951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.935047:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982745 is committed 00002000:00010000:1.0:1789760776.935172:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63225856 -> 63229952 00002000:00010000:1.0:1789760776.935178:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123512 -> 123520 00010000:00010000:1.0:1789760776.935191:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c504a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63225856->63229951] (req 63225856->63229951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.935399:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c504a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63225856->63229951] (req 63225856->63229951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.935411:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c504a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63225856->63229951] (req 63225856->63229951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.935418:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.935423:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c504a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63225856->63229951] (req 63225856->63229951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.935433:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c504a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63225856->63229951] (req 63225856->63229951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.935523:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c504a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63225856->63229951] (req 63225856->63229951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.935533:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c504a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63225856->63229951] (req 63225856->63229951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.935624:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04563480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554741248:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4981us (5260us total) trans 4294982745 rc 0/0 00000100:00100000:1.0:1789760776.935804:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15447 00000100:00100000:1.0:1789760776.935813:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720165536898 time 720176245723 state 0x2 ref 0 00000100:00100000:1.0:1789760776.939795:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760776.939882:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554741376 00000100:00100000:2.0:1789760776.939893:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554741376 00000100:00100000:2.0:1789760776.939947:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720165536898 time 720176245723 state 0x0 ref 1 00000100:00100000:2.0:1789760776.939963:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720176506139 time 720176245723 state 0x1 ref 1 00000100:00100000:2.0:1789760776.940038:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720176506139 time 720185656147 state 0x0 ref 1 00000100:00100000:2.0:1789760776.940051:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554741376:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760776.940109:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63229952, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e280 00010000:00010000:2.0:1789760776.940118:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5051 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.940134:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760776.940139:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.940145:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5051 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63229952->63234047] (req 63229952->63234047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.940160:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5051 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63229952->63234047] (req 63229952->63234047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.944291:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982746 is committed 00002000:00010000:2.0:1789760776.944443:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63229952 -> 63234048 00002000:00010000:2.0:1789760776.944449:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123520 -> 123528 00010000:00010000:2.0:1789760776.944461:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5051 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63229952->63234047] (req 63229952->63234047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.944474:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5051 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63229952->63234047] (req 63229952->63234047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.944493:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5051 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63229952->63234047] (req 63229952->63234047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.944505:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760776.944510:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5051 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63229952->63234047] (req 63229952->63234047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.944519:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5051 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63229952->63234047] (req 63229952->63234047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.944538:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5051 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63229952->63234047] (req 63229952->63234047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.944545:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5051 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63229952->63234047] (req 63229952->63234047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.944632:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554741376:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4579us (4842us total) trans 4294982746 rc 0/0 00000100:00100000:2.0:1789760776.944642:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15448 00000100:00100000:2.0:1789760776.944651:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720176506139 time 720185656147 state 0x2 ref 0 00000100:00100000:0.0:1789760776.948620:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.948867:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554741504 00000100:00100000:1.0:1789760776.948877:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554741504 00000100:00100000:1.0:1789760776.948929:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720176506139 time 720185656147 state 0x0 ref 1 00000100:00100000:1.0:1789760776.948945:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720185916563 time 720185656147 state 0x1 ref 1 00000100:00100000:1.0:1789760776.948971:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720185916563 time 720194589203 state 0x0 ref 1 00000100:00100000:1.0:1789760776.948983:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554741504:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.949035:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63234048, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e288 00010000:00010000:1.0:1789760776.949044:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5058 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.949060:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760776.949064:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.949070:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5058 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63234048->63238143] (req 63234048->63238143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.949085:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5058 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63234048->63238143] (req 63234048->63238143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.954204:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982747 is committed 00002000:00010000:1.0:1789760776.954460:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63234048 -> 63238144 00002000:00010000:1.0:1789760776.954466:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123528 -> 123536 00010000:00010000:1.0:1789760776.954480:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5058 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63234048->63238143] (req 63234048->63238143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.954498:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5058 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63234048->63238143] (req 63234048->63238143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.954518:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5058 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63234048->63238143] (req 63234048->63238143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.954525:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.954530:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5058 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63234048->63238143] (req 63234048->63238143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.954538:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5058 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63234048->63238143] (req 63234048->63238143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.954557:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5058 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63234048->63238143] (req 63234048->63238143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.954566:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5058 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63234048->63238143] (req 63234048->63238143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.954660:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554741504:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5674us (6043us total) trans 4294982747 rc 0/0 00000100:00100000:1.0:1789760776.954671:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15449 00000100:00100000:1.0:1789760776.954681:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720185916563 time 720194589203 state 0x2 ref 0 00000100:00100000:0.0:1789760776.961574:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760776.961652:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554741632 00000100:00100000:2.0:1789760776.961662:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554741632 00000100:00100000:2.0:1789760776.961711:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720185916563 time 720194589203 state 0x0 ref 1 00000100:00100000:2.0:1789760776.961737:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720194849619 time 720194589203 state 0x1 ref 1 00000100:00100000:2.0:1789760776.961808:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720194849619 time 720207425344 state 0x0 ref 1 00000100:00100000:2.0:1789760776.961822:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554741632:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760776.961879:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63238144, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e290 00010000:00010000:2.0:1789760776.961891:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c505f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.961909:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760776.961914:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760776.961922:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c505f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63238144->63242239] (req 63238144->63242239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.961936:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c505f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63238144->63242239] (req 63238144->63242239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.967547:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982748 is committed 00002000:00010000:2.0:1789760776.968213:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63238144 -> 63242240 00002000:00010000:2.0:1789760776.968219:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123536 -> 123544 00010000:00010000:2.0:1789760776.968233:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c505f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63238144->63242239] (req 63238144->63242239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.968247:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c505f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63238144->63242239] (req 63238144->63242239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.968256:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c505f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63238144->63242239] (req 63238144->63242239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.968264:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760776.968271:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c505f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63238144->63242239] (req 63238144->63242239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.968463:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c505f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63238144->63242239] (req 63238144->63242239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.968551:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c505f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63238144->63242239] (req 63238144->63242239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760776.968559:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c505f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63238144->63242239] (req 63238144->63242239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760776.968694:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554741632:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6870us (7125us total) trans 4294982748 rc 0/0 00000100:00100000:2.0:1789760776.968705:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15450 00000100:00100000:2.0:1789760776.968731:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720194849619 time 720207425344 state 0x2 ref 0 00000100:00100000:2.0:1789760776.973248:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.973492:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554741760 00000100:00100000:1.0:1789760776.973503:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554741760 00000100:00100000:1.0:1789760776.973555:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720194849619 time 720207425344 state 0x0 ref 1 00000100:00100000:1.0:1789760776.973571:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720207685760 time 720207425344 state 0x1 ref 1 00000100:00100000:1.0:1789760776.973606:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720207685760 time 720219224608 state 0x0 ref 1 00000100:00100000:1.0:1789760776.973618:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554741760:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.973665:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63242240, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e298 00010000:00010000:1.0:1789760776.973673:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5066 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.973688:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760776.973692:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.973698:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5066 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63242240->63246335] (req 63242240->63246335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.973712:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5066 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63242240->63246335] (req 63242240->63246335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760776.979489:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982749 is committed 00002000:00010000:1.0:1789760776.980063:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63242240 -> 63246336 00002000:00010000:1.0:1789760776.980074:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123544 -> 123552 00010000:00010000:1.0:1789760776.980088:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5066 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63242240->63246335] (req 63242240->63246335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.980160:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5066 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63242240->63246335] (req 63242240->63246335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.980176:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5066 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63242240->63246335] (req 63242240->63246335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.980184:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.980189:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5066 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63242240->63246335] (req 63242240->63246335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.980199:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5066 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63242240->63246335] (req 63242240->63246335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.980220:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5066 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63242240->63246335] (req 63242240->63246335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.980228:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5066 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63242240->63246335] (req 63242240->63246335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.980360:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554741760:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6739us (7116us total) trans 4294982749 rc 0/0 00000100:00100000:1.0:1789760776.980370:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15451 00000100:00100000:1.0:1789760776.980432:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720207685760 time 720219224608 state 0x2 ref 0 00000100:00100000:0.0:1789760776.985546:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.986696:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554741888 00000100:00100000:1.0:1789760776.986707:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554741888 00000100:00100000:1.0:1789760776.986772:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720207685760 time 720219224608 state 0x0 ref 1 00000100:00100000:1.0:1789760776.986787:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720219485024 time 720219224608 state 0x1 ref 1 00000100:00100000:1.0:1789760776.986820:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720219485024 time 720232438740 state 0x0 ref 1 00000100:00100000:1.0:1789760776.987367:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554741888:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.987740:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63246336, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e2a0 00010000:00010000:1.0:1789760776.987750:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c506d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.987767:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760776.987772:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.987778:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c506d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63246336->63250431] (req 63246336->63250431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.987792:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c506d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63246336->63250431] (req 63246336->63250431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760776.994085:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982750 is committed 00002000:00010000:1.0:1789760776.994610:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63246336 -> 63250432 00002000:00010000:1.0:1789760776.994616:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123552 -> 123560 00010000:00010000:1.0:1789760776.994629:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c506d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63246336->63250431] (req 63246336->63250431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.994641:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c506d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63246336->63250431] (req 63246336->63250431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.994651:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c506d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63246336->63250431] (req 63246336->63250431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.994664:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760776.994669:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c506d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63246336->63250431] (req 63246336->63250431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.994679:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c506d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63246336->63250431] (req 63246336->63250431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.994697:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c506d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63246336->63250431] (req 63246336->63250431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.994705:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c506d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63246336->63250431] (req 63246336->63250431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760776.994915:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554741888:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7549us (9373us total) trans 4294982750 rc 0/0 00000100:00100000:1.0:1789760776.994926:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15452 00000100:00100000:1.0:1789760776.994935:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720219485024 time 720232438740 state 0x2 ref 0 00000100:00100000:0.0:1789760776.998942:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760776.999042:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554742016 00000100:00100000:1.0:1789760776.999052:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554742016 00000100:00100000:1.0:1789760776.999112:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720219485024 time 720232438740 state 0x0 ref 1 00000100:00100000:1.0:1789760776.999128:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720232699156 time 720232438740 state 0x1 ref 1 00000100:00100000:1.0:1789760776.999149:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720232699156 time 720244767706 state 0x0 ref 1 00000100:00100000:1.0:1789760776.999164:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554742016:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760776.999209:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63250432, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e2a8 00010000:00010000:1.0:1789760776.999218:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c5074 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.999233:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760776.999238:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760776.999243:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c5074 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63250432->63254527] (req 63250432->63254527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760776.999257:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c5074 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63250432->63254527] (req 63250432->63254527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.005733:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982751 is committed 00002000:00010000:1.0:1789760777.006044:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63250432 -> 63254528 00002000:00010000:1.0:1789760777.006049:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123560 -> 123568 00010000:00010000:1.0:1789760777.006071:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c5074 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63250432->63254527] (req 63250432->63254527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.006085:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c5074 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63250432->63254527] (req 63250432->63254527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.006103:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c5074 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63250432->63254527] (req 63250432->63254527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.006111:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655de00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760777.006117:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c5074 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63250432->63254527] (req 63250432->63254527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.006126:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c5074 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63250432->63254527] (req 63250432->63254527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.006146:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c5074 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63250432->63254527] (req 63250432->63254527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.006154:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c5074 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63250432->63254527] (req 63250432->63254527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.006247:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554742016:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7079us (7310us total) trans 4294982751 rc 0/0 00000100:00100000:1.0:1789760777.006258:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15453 00000100:00100000:1.0:1789760777.006267:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720232699156 time 720244767706 state 0x2 ref 0 00000100:00100000:2.0:1789760777.009859:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760777.009912:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554742144 00000100:00100000:0.0:1789760777.009922:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554742144 00000100:00100000:0.0:1789760777.009987:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720232699156 time 720244767706 state 0x0 ref 1 00000100:00100000:0.0:1789760777.010002:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720245028122 time 720244767706 state 0x1 ref 1 00000100:00100000:0.0:1789760777.010032:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720245028122 time 720255651043 state 0x0 ref 1 00000100:00080000:1.0:1789760777.010201:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760777.010203:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04562d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554742144:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760777.010250:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63254528, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e2b0 00010000:00010000:0.0:1789760777.010260:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c507b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.010274:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760777.010278:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.010284:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c507b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63254528->63258623] (req 63254528->63258623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.010296:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c507b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63254528->63258623] (req 63254528->63258623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.015357:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982752 is committed 00002000:00010000:0.0:1789760777.015579:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63254528 -> 63258624 00002000:00010000:0.0:1789760777.015585:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123568 -> 123576 00010000:00010000:0.0:1789760777.015599:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c507b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63254528->63258623] (req 63254528->63258623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.015617:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c507b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63254528->63258623] (req 63254528->63258623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.015626:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c507b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63254528->63258623] (req 63254528->63258623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.015633:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760777.015638:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c507b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63254528->63258623] (req 63254528->63258623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.015647:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c507b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63254528->63258623] (req 63254528->63258623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.015666:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c507b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63254528->63258623] (req 63254528->63258623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.015674:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c507b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63254528->63258623] (req 63254528->63258623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.015778:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04562d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554742144:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5574us (5923us total) trans 4294982752 rc 0/0 00000100:00100000:0.0:1789760777.015789:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15454 00000100:00100000:0.0:1789760777.015798:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720245028122 time 720255651043 state 0x2 ref 0 00000100:00100000:0.0:1789760777.023259:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.023512:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554742272 00000100:00100000:2.0:1789760777.023524:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554742272 00000100:00100000:2.0:1789760777.023579:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720245028122 time 720255651043 state 0x0 ref 1 00000100:00100000:2.0:1789760777.023596:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720255911459 time 720255651043 state 0x1 ref 1 00000100:00100000:2.0:1789760777.023633:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720255911459 time 720269250617 state 0x0 ref 1 00000100:00100000:2.0:1789760777.023646:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554742272:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.023694:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63258624, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e2b8 00010000:00010000:2.0:1789760777.023704:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5082 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.023728:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760777.023733:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.023740:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5082 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63258624->63262719] (req 63258624->63262719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.023755:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5082 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63258624->63262719] (req 63258624->63262719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.030437:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982753 is committed 00002000:00010000:2.0:1789760777.030698:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63258624 -> 63262720 00002000:00010000:2.0:1789760777.030704:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123576 -> 123584 00010000:00010000:2.0:1789760777.030717:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5082 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63258624->63262719] (req 63258624->63262719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.030740:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5082 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63258624->63262719] (req 63258624->63262719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.030749:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5082 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63258624->63262719] (req 63258624->63262719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.030762:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.030766:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5082 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63258624->63262719] (req 63258624->63262719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.030775:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5082 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63258624->63262719] (req 63258624->63262719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.030793:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5082 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63258624->63262719] (req 63258624->63262719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.030800:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5082 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63258624->63262719] (req 63258624->63262719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.030897:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554742272:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7250us (7643us total) trans 4294982753 rc 0/0 00000100:00100000:2.0:1789760777.030907:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15455 00000100:00100000:2.0:1789760777.030916:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720255911459 time 720269250617 state 0x2 ref 0 00000100:00100000:2.0:1789760777.035006:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760777.035090:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554742400 00000100:00100000:1.0:1789760777.035153:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554742400 00000100:00100000:1.0:1789760777.035204:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720255911459 time 720269250617 state 0x0 ref 1 00000100:00100000:1.0:1789760777.035223:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720269511033 time 720269250617 state 0x1 ref 1 00000100:00100000:1.0:1789760777.035256:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720269511033 time 720280874209 state 0x0 ref 1 00000100:00100000:1.0:1789760777.035268:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe1d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554742400:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760777.035312:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63262720, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e2c0 00010000:00010000:1.0:1789760777.035321:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5089 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.035336:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760777.035340:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.035346:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5089 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63262720->63266815] (req 63262720->63266815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.035360:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5089 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63262720->63266815] (req 63262720->63266815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.043609:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982754 is committed 00002000:00010000:0.0:1789760777.047031:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63262720 -> 63266816 00002000:00010000:0.0:1789760777.047041:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123584 -> 123592 00010000:00010000:0.0:1789760777.047054:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5089 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63262720->63266815] (req 63262720->63266815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.047067:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5089 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63262720->63266815] (req 63262720->63266815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.047076:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5089 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63262720->63266815] (req 63262720->63266815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.047084:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760777.047089:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5089 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63262720->63266815] (req 63262720->63266815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.047156:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5089 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63262720->63266815] (req 63262720->63266815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.047175:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5089 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63262720->63266815] (req 63262720->63266815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.047182:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5089 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63262720->63266815] (req 63262720->63266815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.047338:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe1d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554742400:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12067us (12337us total) trans 4294982754 rc 0/0 00000100:00100000:0.0:1789760777.047349:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15456 00000100:00100000:0.0:1789760777.047359:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720269511033 time 720280874209 state 0x2 ref 0 00000100:00100000:1.0:1789760777.051108:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760777.051942:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554742528 00000100:00100000:0.0:1789760777.051953:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554742528 00000100:00100000:0.0:1789760777.052004:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720269511033 time 720280874209 state 0x0 ref 1 00000100:00100000:0.0:1789760777.052019:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720281134625 time 720280874209 state 0x1 ref 1 00000100:00100000:0.0:1789760777.052051:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720281134625 time 720297669616 state 0x0 ref 1 00000100:00100000:0.0:1789760777.052063:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d81c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554742528:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760777.052220:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63266816, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e2c8 00010000:00010000:0.0:1789760777.052230:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5090 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.052245:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760777.052249:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.052254:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5090 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63266816->63270911] (req 63266816->63270911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.052265:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5090 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63266816->63270911] (req 63266816->63270911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.057592:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982755 is committed 00002000:00010000:0.0:1789760777.057817:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63266816 -> 63270912 00002000:00010000:0.0:1789760777.057823:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123592 -> 123600 00010000:00010000:0.0:1789760777.057835:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5090 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63266816->63270911] (req 63266816->63270911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.057851:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5090 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63266816->63270911] (req 63266816->63270911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.057860:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5090 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63266816->63270911] (req 63266816->63270911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.057868:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760777.057873:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5090 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63266816->63270911] (req 63266816->63270911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.057883:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5090 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63266816->63270911] (req 63266816->63270911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.057901:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5090 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63266816->63270911] (req 63266816->63270911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.057909:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5090 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63266816->63270911] (req 63266816->63270911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.058024:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d81c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554742528:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5960us (6927us total) trans 4294982755 rc 0/0 00000100:00100000:0.0:1789760777.058034:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15457 00000100:00100000:0.0:1789760777.058043:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720281134625 time 720297669616 state 0x2 ref 0 00000100:00100000:2.0:1789760777.063663:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760777.066216:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554742656 00000100:00100000:0.0:1789760777.066231:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554742656 00000100:00100000:0.0:1789760777.066291:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720281134625 time 720297669616 state 0x0 ref 1 00000100:00100000:0.0:1789760777.066307:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720297930032 time 720297669616 state 0x1 ref 1 00000100:00100000:0.0:1789760777.066444:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720297930032 time 720312060675 state 0x0 ref 1 00000100:00100000:0.0:1789760777.066458:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04563b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554742656:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760777.068198:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63270912, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e2d0 00010000:00010000:0.0:1789760777.068212:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5097 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.068227:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760777.068231:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.068237:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5097 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63270912->63275007] (req 63270912->63275007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.068252:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5097 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63270912->63275007] (req 63270912->63275007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.073207:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982756 is committed 00002000:00010000:0.0:1789760777.073538:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63270912 -> 63275008 00002000:00010000:0.0:1789760777.073543:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123600 -> 123608 00010000:00010000:0.0:1789760777.073557:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5097 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63270912->63275007] (req 63270912->63275007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.073570:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5097 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63270912->63275007] (req 63270912->63275007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.073586:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5097 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63270912->63275007] (req 63270912->63275007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.073594:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760777.073600:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5097 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63270912->63275007] (req 63270912->63275007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.073609:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5097 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63270912->63275007] (req 63270912->63275007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.073630:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5097 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63270912->63275007] (req 63270912->63275007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.073637:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5097 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63270912->63275007] (req 63270912->63275007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.073734:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04563b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554742656:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7262us (10340us total) trans 4294982756 rc 0/0 00000100:00100000:0.0:1789760777.073744:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15458 00000100:00100000:0.0:1789760777.074645:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720297930032 time 720312060675 state 0x2 ref 0 00000100:00100000:2.0:1789760777.077839:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760777.078003:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554742784 00000100:00100000:1.0:1789760777.078013:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554742784 00000100:00100000:1.0:1789760777.078064:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720297930032 time 720312060675 state 0x0 ref 1 00000100:00100000:1.0:1789760777.078079:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720312321091 time 720312060675 state 0x1 ref 1 00000100:00100000:1.0:1789760777.078115:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720312321091 time 720323733268 state 0x0 ref 1 00000100:00100000:1.0:1789760777.078125:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554742784:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760777.078170:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63275008, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e2d8 00010000:00010000:1.0:1789760777.078179:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c509e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.078198:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760777.078203:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.078208:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c509e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63275008->63279103] (req 63275008->63279103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.078221:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c509e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63275008->63279103] (req 63275008->63279103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.084950:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982757 is committed 00002000:00010000:2.0:1789760777.085123:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63275008 -> 63279104 00002000:00010000:2.0:1789760777.085129:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123608 -> 123616 00010000:00010000:2.0:1789760777.085143:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c509e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63275008->63279103] (req 63275008->63279103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.085157:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c509e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63275008->63279103] (req 63275008->63279103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.085166:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c509e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63275008->63279103] (req 63275008->63279103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.085174:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.085179:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c509e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63275008->63279103] (req 63275008->63279103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.085189:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c509e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63275008->63279103] (req 63275008->63279103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.085209:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c509e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63275008->63279103] (req 63275008->63279103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.085217:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c509e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63275008->63279103] (req 63275008->63279103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.085324:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554742784:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7194us (7488us total) trans 4294982757 rc 0/0 00000100:00100000:2.0:1789760777.085334:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15459 00000100:00100000:2.0:1789760777.085342:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720312321091 time 720323733268 state 0x2 ref 0 00000100:00100000:1.0:1789760777.094266:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.094476:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554742912 00000100:00100000:2.0:1789760777.094487:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554742912 00000100:00100000:2.0:1789760777.094548:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720312321091 time 720323733268 state 0x0 ref 1 00000100:00100000:2.0:1789760777.094564:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720323993684 time 720323733268 state 0x1 ref 1 00000100:00100000:2.0:1789760777.094587:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720323993684 time 720340206144 state 0x0 ref 1 00000100:00100000:2.0:1789760777.094600:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d81880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554742912:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.094648:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63279104, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e2e0 00010000:00010000:2.0:1789760777.094659:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50a5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.094688:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760777.094693:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.094699:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63279104->63283199] (req 63279104->63283199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.094712:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63279104->63283199] (req 63279104->63283199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.100540:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982758 is committed 00002000:00010000:2.0:1789760777.100655:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63279104 -> 63283200 00002000:00010000:2.0:1789760777.100660:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123616 -> 123624 00010000:00010000:2.0:1789760777.100673:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63279104->63283199] (req 63279104->63283199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.100692:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50a5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63279104->63283199] (req 63279104->63283199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.100702:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63279104->63283199] (req 63279104->63283199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.100709:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.100713:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63279104->63283199] (req 63279104->63283199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.100732:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50a5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63279104->63283199] (req 63279104->63283199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.100750:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50a5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63279104->63283199] (req 63279104->63283199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.100758:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50a5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63279104->63283199] (req 63279104->63283199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.100849:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d81880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554742912:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6246us (6587us total) trans 4294982758 rc 0/0 00000100:00100000:2.0:1789760777.100860:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15460 00000100:00100000:2.0:1789760777.100869:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720323993684 time 720340206144 state 0x2 ref 0 00000100:00100000:0.0:1789760777.104895:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760777.105576:0:3648:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789760777.105580:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03f16d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573914752:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760777.105593:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb04561c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3648:1876699573914624:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789760777.105644:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760777.105679:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760777.105727:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573914752 00000100:00100000:3.0:1789760777.105734:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573914752 00000100:00100000:3.0:1789760777.105784:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573914624 00000100:00100000:3.0:1789760777.105787:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573914624 00000100:00100000:3.0:1789760777.105803:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 358 00000100:00100000:3.0:1789760777.105810:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04560e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573914752:12345-0@lo:13:osp-pre-1-0.0 00000100:00080000:2.0:1789760777.105836:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00002000:00000024:3.0:1789760777.105839:0:8440:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:3.0:1789760777.105890:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04560e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573914752:12345-0@lo:13:osp-pre-1-0.0 Request processed in 80us (247us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760777.105898:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 358 00000100:00100000:3.0:1789760777.105923:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 359 00000100:00100000:3.0:1789760777.105931:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f14380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3648:x1876699573914624:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1789760777.105942:0:8440:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:2.0:1789760777.105962:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03f16d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573914752:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760777.105981:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f14380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3648:x1876699573914624:12345-0@lo:13:osp-pre-1-1.0 Request processed in 50us (301us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760777.105987:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 359 00000100:00100000:3.0:1789760777.106020:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb04561c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3648:1876699573914624:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789760777.107152:0:3646:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760777.107169:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03ebf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573914880:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789760777.107226:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760777.107266:0:7831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573914880 00000100:00100000:0.0:1789760777.107272:0:7831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573914880 00000100:00100000:0.0:1789760777.107317:0:7831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 383 00000100:00100000:0.0:1789760777.107326:0:7831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3646:x1876699573914880:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1789760777.107356:0:7831:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446227968 avail 1299988480 00000100:00100000:0.0:1789760777.107422:0:7831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3646:x1876699573914880:12345-0@lo:41:osp-pre-0-1.0 Request processed in 96us (195us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760777.107430:0:7831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 383 00000100:00100000:0.0:1789760777.108157:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03ebf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573914880:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789760777.108182:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb04561500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3646:1876699573915264:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.108219:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760777.108312:0:3647:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb04562680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3647:1876699573915392:0@lo:400:kworker.0 00000100:00100000:3.0:1789760777.108350:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760777.108392:0:3647:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb04560a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3647:1876699573915904:0@lo:400:kworker.0 00000100:00100000:3.0:1789760777.108415:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760777.108443:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573915392 00000100:00100000:3.0:1789760777.108448:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573915392 00000100:00100000:1.0:1789760777.108454:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554743040 00000100:00100000:1.0:1789760777.108462:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554743040 00000100:00100000:1.0:1789760777.108480:0:3649:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb04560380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3649:1876699573915136:0@lo:400:kworker.0 00000100:00100000:3.0:1789760777.108495:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 384 00000100:00100000:3.0:1789760777.108505:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f17800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3647:x1876699573915392:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760777.108556:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720323993684 time 720340206144 state 0x0 ref 1 00000100:00100000:3.0:1789760777.108557:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f17800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3647:x1876699573915392:12345-0@lo:400:kworker.0 Request processed in 52us (207us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760777.108572:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 384 00000100:00100000:1.0:1789760777.108579:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720340466560 time 720340206144 state 0x1 ref 1 00000100:00100000:3.0:1789760777.108596:0:15103:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573915904 00000100:00100000:3.0:1789760777.108601:0:15103:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573915904 00000100:00100000:1.0:1789760777.108606:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720340466560 time 720354223325 state 0x0 ref 1 00000100:00100000:1.0:1789760777.108618:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554743040:12345-192.168.203.22@tcp:4:dd.0 00000100:00100000:3.0:1789760777.108644:0:15103:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 294 00000100:00100000:3.0:1789760777.108654:0:15103:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f17100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3647:x1876699573915904:12345-0@lo:400:kworker.0 00002000:00010000:1.0:1789760777.108659:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63283200, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e2e8 00010000:00010000:1.0:1789760777.108669:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50ac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.108687:0:15103:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f17100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3647:x1876699573915904:12345-0@lo:400:kworker.0 Request processed in 35us (271us total) trans 0 rc 0/0 00010000:00010000:1.0:1789760777.108694:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00000100:00100000:3.0:1789760777.108694:0:15103:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 294 00010000:00010000:1.0:1789760777.108702:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.108709:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63283200->63287295] (req 63283200->63287295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760777.108715:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760777.108751:0:3647:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb04562680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3647:1876699573915392:0@lo:400:kworker.0 00000100:00100000:3.0:1789760777.108762:0:3647:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb04560a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3647:1876699573915904:0@lo:400:kworker.0 00000100:00100000:3.0:1789760777.108781:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb04562a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:5711db52-8702-4d6b-b42c-8543e9bf1cb2:3648:1876699573915008:0@lo:400:kworker.0 00000100:00100000:3.0:1789760777.108809:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1789760777.108833:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63283200->63287295] (req 63283200->63287295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.108837:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb04561c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3648:1876699573915520:0@lo:400:kworker.0 00000100:00100000:3.0:1789760777.108869:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573915008 00000100:00100000:3.0:1789760777.108872:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573915008 00000100:00100000:3.0:1789760777.108896:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 469 00000100:00100000:3.0:1789760777.108901:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc320b6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5711db52-8702-4d6b-b42c-8543e9bf1cb2+24:3648:x1876699573915008:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760777.108937:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc320b6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5711db52-8702-4d6b-b42c-8543e9bf1cb2+24:3648:x1876699573915008:12345-0@lo:400:kworker.0 Request processed in 36us (128us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760777.108943:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 469 00000100:00100000:0.0:1789760777.109565:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb04560000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573915776:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.109596:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760777.109663:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760777.111300:0:8438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573915776 00000100:00100000:3.0:1789760777.111313:0:8438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573915776 00000100:00100000:3.0:1789760777.111415:0:8438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 295 00000100:00100000:3.0:1789760777.111425:0:8438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3646:x1876699573915776:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760777.111513:0:8438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3646:x1876699573915776:12345-0@lo:400:kworker.0 Request processed in 89us (1915us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760777.111520:0:8438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 295 00000100:00100000:3.0:1789760777.111570:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb04560000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573915776:0@lo:400:kworker.0 00000100:00080000:0.0:1789760777.111982:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760777.118424:0:3649:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb04562300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3649:1876699573915648:0@lo:400:kworker.0 00000100:00100000:1.0:1789760777.118501:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760777.118519:0:3649:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb04563480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3649:1876699573916160:0@lo:400:kworker.0 00000100:00100000:1.0:1789760777.118540:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760777.118578:0:6521:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573915264 00000100:00100000:1.0:1789760777.118586:0:6521:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573915264 00000100:00100000:1.0:1789760777.118634:0:6521:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573915136 00000100:00100000:1.0:1789760777.118637:0:6521:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573915136 00000100:00100000:1.0:1789760777.118656:0:6521:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573915648 00000100:00100000:1.0:1789760777.118659:0:6521:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573915648 00000100:00100000:1.0:1789760777.118686:0:6521:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573916160 00000100:00100000:1.0:1789760777.118689:0:6521:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573916160 00000100:00100000:1.0:1789760777.118705:0:6521:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 869 00000100:00100000:1.0:1789760777.118713:0:6521:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3646:x1876699573915264:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760777.118773:0:6521:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3646:x1876699573915264:12345-0@lo:400:kworker.0 Request processed in 59us (10551us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760777.118780:0:6521:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 869 00000100:00100000:1.0:1789760777.118795:0:6521:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 870 00000100:00100000:1.0:1789760777.118801:0:6521:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d80a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3649:x1876699573915136:12345-0@lo:400:kworker.0 00000100:00080000:0.0:1789760777.118814:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:1.0:1789760777.118829:0:6521:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d80a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3649:x1876699573915136:12345-0@lo:400:kworker.0 Request processed in 28us (9167us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760777.118836:0:6521:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 870 00000100:00100000:1.0:1789760777.118848:0:6521:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 871 00000100:00100000:1.0:1789760777.118854:0:6521:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d80000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3649:x1876699573915648:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760777.118881:0:6521:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d80000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3649:x1876699573915648:12345-0@lo:400:kworker.0 Request processed in 28us (381us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760777.118887:0:6521:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 871 00000100:00100000:1.0:1789760777.118898:0:6521:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 872 00000100:00100000:1.0:1789760777.118904:0:6521:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d82680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3649:x1876699573916160:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760777.118928:0:6521:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d82680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3649:x1876699573916160:12345-0@lo:400:kworker.0 Request processed in 25us (387us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760777.118934:0:6521:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 872 00000100:00100000:1.0:1789760777.118970:0:3649:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb04563480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3649:1876699573916160:0@lo:400:kworker.0 00000100:00100000:1.0:1789760777.118994:0:3649:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb04560380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3649:1876699573915136:0@lo:400:kworker.0 00000100:00100000:1.0:1789760777.119003:0:3649:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb04562300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3649:1876699573915648:0@lo:400:kworker.0 00000100:00100000:3.0:1789760777.119543:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760777.119573:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb04563800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3648:1876699573916032:0@lo:400:kworker.0 00000100:00100000:3.0:1789760777.119605:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760777.119639:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb04562a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:5711db52-8702-4d6b-b42c-8543e9bf1cb2:3648:1876699573915008:0@lo:400:kworker.0 00000100:00100000:3.0:1789760777.119667:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb04561500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3646:1876699573915264:0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.119808:0:7470:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573915520 00000100:00100000:0.0:1789760777.119813:0:7470:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573915520 00000100:00100000:0.0:1789760777.119855:0:7470:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573916032 00000100:00100000:0.0:1789760777.119858:0:7470:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573916032 00000100:00100000:0.0:1789760777.119878:0:7470:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 873 00000100:00100000:0.0:1789760777.119894:0:7470:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f15500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3648:x1876699573915520:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.119938:0:7470:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f15500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3648:x1876699573915520:12345-0@lo:400:kworker.0 Request processed in 53us (399us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760777.119944:0:7470:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 873 00000100:00100000:0.0:1789760777.119958:0:7470:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 874 00000100:00100000:0.0:1789760777.119964:0:7470:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f14a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3648:x1876699573916032:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760777.120087:0:7470:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f14a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3648:x1876699573916032:12345-0@lo:400:kworker.0 Request processed in 118us (476us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760777.120183:0:7470:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 874 00000100:00080000:3.0:1789760777.120359:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789760777.120401:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb04561c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3648:1876699573915520:0@lo:400:kworker.0 00000100:00100000:3.0:1789760777.120411:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb04563800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3648:1876699573916032:0@lo:400:kworker.0 00000001:00080000:0.0:1789760777.120638:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982759 is committed 00002000:00010000:1.0:1789760777.120786:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63283200 -> 63287296 00002000:00010000:1.0:1789760777.120791:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123624 -> 123632 00010000:00010000:1.0:1789760777.120803:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63283200->63287295] (req 63283200->63287295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.120815:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63283200->63287295] (req 63283200->63287295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.120824:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63283200->63287295] (req 63283200->63287295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.120837:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760777.120842:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63283200->63287295] (req 63283200->63287295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.120851:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63283200->63287295] (req 63283200->63287295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.120871:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50ac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63283200->63287295] (req 63283200->63287295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.120878:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50ac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63283200->63287295] (req 63283200->63287295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.120959:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554743040:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12340us (16070us total) trans 4294982759 rc 0/0 00000100:00100000:1.0:1789760777.120970:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15461 00000100:00100000:1.0:1789760777.120979:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720340466560 time 720354223325 state 0x2 ref 0 00000100:00100000:0.0:1789760777.132235:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.132732:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554743168 00000100:00100000:2.0:1789760777.132742:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554743168 00000100:00100000:2.0:1789760777.132797:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720340466560 time 720354223325 state 0x0 ref 1 00000100:00100000:2.0:1789760777.132811:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720354483741 time 720354223325 state 0x1 ref 1 00000100:00100000:2.0:1789760777.132843:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720354483741 time 720378462063 state 0x0 ref 1 00000100:00100000:2.0:1789760777.132855:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554743168:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.132899:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63287296, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e2f0 00010000:00010000:2.0:1789760777.132908:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50b3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.132933:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760777.132938:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.132945:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63287296->63291391] (req 63287296->63291391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.132960:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63287296->63291391] (req 63287296->63291391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.139575:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982760 is committed 00002000:00010000:2.0:1789760777.141892:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63287296 -> 63291392 00002000:00010000:2.0:1789760777.141900:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123632 -> 123640 00010000:00010000:2.0:1789760777.141980:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63287296->63291391] (req 63287296->63291391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.141994:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63287296->63291391] (req 63287296->63291391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.142003:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63287296->63291391] (req 63287296->63291391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.142011:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.142016:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63287296->63291391] (req 63287296->63291391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.142025:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63287296->63291391] (req 63287296->63291391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.142043:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50b3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63287296->63291391] (req 63287296->63291391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.142052:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50b3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63287296->63291391] (req 63287296->63291391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.142587:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554743168:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9726us (10354us total) trans 4294982760 rc 0/0 00000100:00100000:2.0:1789760777.142599:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15462 00000100:00100000:2.0:1789760777.142609:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720354483741 time 720378462063 state 0x2 ref 0 00000100:00100000:3.0:1789760777.148681:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760777.148782:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554743296 00000100:00100000:1.0:1789760777.148793:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554743296 00000100:00100000:1.0:1789760777.148847:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720354483741 time 720378462063 state 0x0 ref 1 00000100:00100000:1.0:1789760777.148863:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720378722479 time 720378462063 state 0x1 ref 1 00000100:00100000:1.0:1789760777.148898:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720378722479 time 720394516387 state 0x0 ref 1 00000100:00100000:1.0:1789760777.148913:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f14700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554743296:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760777.148960:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63291392, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e2f8 00010000:00010000:1.0:1789760777.148968:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50ba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.148983:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760777.148987:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.148992:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63291392->63295487] (req 63291392->63295487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.149004:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63291392->63295487] (req 63291392->63295487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.159011:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982761 is committed 00002000:00010000:1.0:1789760777.159337:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63291392 -> 63295488 00002000:00010000:1.0:1789760777.159342:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123640 -> 123648 00010000:00010000:1.0:1789760777.159355:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63291392->63295487] (req 63291392->63295487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.159367:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50ba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63291392->63295487] (req 63291392->63295487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.159425:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63291392->63295487] (req 63291392->63295487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.159434:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760777.159439:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63291392->63295487] (req 63291392->63295487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.159448:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63291392->63295487] (req 63291392->63295487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.159470:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50ba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63291392->63295487] (req 63291392->63295487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.159478:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50ba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63291392->63295487] (req 63291392->63295487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.159569:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f14700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554743296:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10653us (10893us total) trans 4294982761 rc 0/0 00000100:00100000:1.0:1789760777.159579:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15463 00000100:00100000:1.0:1789760777.159588:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720378722479 time 720394516387 state 0x2 ref 0 00000100:00100000:0.0:1789760777.169875:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.170613:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554743424 00000100:00100000:2.0:1789760777.170625:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554743424 00000100:00100000:2.0:1789760777.170683:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720378722479 time 720394516387 state 0x0 ref 1 00000100:00100000:2.0:1789760777.170699:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720394776803 time 720394516387 state 0x1 ref 1 00000100:00100000:2.0:1789760777.170744:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720394776803 time 720416362730 state 0x0 ref 1 00000100:00100000:2.0:1789760777.170759:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554743424:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.170807:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63295488, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e300 00010000:00010000:2.0:1789760777.170816:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50c1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.170964:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760777.170969:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.170976:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63295488->63299583] (req 63295488->63299583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.170991:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63295488->63299583] (req 63295488->63299583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.176773:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982762 is committed 00002000:00010000:2.0:1789760777.176943:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63295488 -> 63299584 00002000:00010000:2.0:1789760777.176949:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123648 -> 123656 00010000:00010000:2.0:1789760777.176962:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63295488->63299583] (req 63295488->63299583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.176982:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50c1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63295488->63299583] (req 63295488->63299583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.176991:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63295488->63299583] (req 63295488->63299583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.176998:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.177002:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63295488->63299583] (req 63295488->63299583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.177011:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50c1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63295488->63299583] (req 63295488->63299583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.177030:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50c1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63295488->63299583] (req 63295488->63299583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.177037:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50c1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63295488->63299583] (req 63295488->63299583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.177242:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554743424:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6481us (7372us total) trans 4294982762 rc 0/0 00000100:00100000:2.0:1789760777.177254:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15464 00000100:00100000:2.0:1789760777.177263:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720394776803 time 720416362730 state 0x2 ref 0 00000100:00100000:0.0:1789760777.185264:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760777.185523:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554743552 00000100:00100000:1.0:1789760777.185533:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554743552 00000100:00100000:1.0:1789760777.185583:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720394776803 time 720416362730 state 0x0 ref 1 00000100:00100000:1.0:1789760777.185597:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720416623146 time 720416362730 state 0x1 ref 1 00000100:00100000:1.0:1789760777.185656:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720416623146 time 720431274312 state 0x0 ref 1 00000100:00100000:1.0:1789760777.185669:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554743552:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760777.185729:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63299584, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e308 00010000:00010000:1.0:1789760777.185740:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50c8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.185756:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760777.185761:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.185768:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63299584->63303679] (req 63299584->63303679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.185781:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63299584->63303679] (req 63299584->63303679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.190783:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982763 is committed 00002000:00010000:1.0:1789760777.190968:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63299584 -> 63303680 00002000:00010000:1.0:1789760777.190973:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123656 -> 123664 00010000:00010000:1.0:1789760777.190986:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63299584->63303679] (req 63299584->63303679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.190999:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50c8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63299584->63303679] (req 63299584->63303679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.191008:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63299584->63303679] (req 63299584->63303679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.191015:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760777.191020:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63299584->63303679] (req 63299584->63303679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.191033:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50c8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63299584->63303679] (req 63299584->63303679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.191053:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50c8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63299584->63303679] (req 63299584->63303679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.191060:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50c8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63299584->63303679] (req 63299584->63303679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.191759:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554743552:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6084us (6498us total) trans 4294982763 rc 0/0 00000100:00100000:1.0:1789760777.191769:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15465 00000100:00100000:1.0:1789760777.191777:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720416623146 time 720431274312 state 0x2 ref 0 00000100:00100000:3.0:1789760777.195598:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760777.195744:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554743680 00000100:00100000:0.0:1789760777.195755:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554743680 00000100:00100000:0.0:1789760777.195808:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720416623146 time 720431274312 state 0x0 ref 1 00000100:00100000:0.0:1789760777.195823:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720431534728 time 720431274312 state 0x1 ref 1 00000100:00100000:0.0:1789760777.195856:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720431534728 time 720441474259 state 0x0 ref 1 00000100:00100000:0.0:1789760777.195868:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f15880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554743680:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760777.195916:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63303680, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e310 00010000:00010000:0.0:1789760777.195926:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c50cf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.195942:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760777.195946:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.195953:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c50cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63303680->63307775] (req 63303680->63307775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.195972:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c50cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63303680->63307775] (req 63303680->63307775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.203438:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982764 is committed 00002000:00010000:2.0:1789760777.203641:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63303680 -> 63307776 00002000:00010000:2.0:1789760777.203647:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123664 -> 123672 00010000:00010000:2.0:1789760777.203659:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c50cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63303680->63307775] (req 63303680->63307775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.203672:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c50cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63303680->63307775] (req 63303680->63307775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.203680:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c50cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63303680->63307775] (req 63303680->63307775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.203688:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.203693:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c50cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63303680->63307775] (req 63303680->63307775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.203704:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c50cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63303680->63307775] (req 63303680->63307775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.203735:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c50cf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63303680->63307775] (req 63303680->63307775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.203744:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c50cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63303680->63307775] (req 63303680->63307775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.203822:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f15880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554743680:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7951us (8228us total) trans 4294982764 rc 0/0 00000100:00100000:2.0:1789760777.203838:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15466 00000100:00100000:2.0:1789760777.203847:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720431534728 time 720441474259 state 0x2 ref 0 00000100:00100000:1.0:1789760777.207949:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.208076:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554743808 00000100:00100000:2.0:1789760777.208087:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554743808 00000100:00100000:2.0:1789760777.208146:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720431534728 time 720441474259 state 0x0 ref 1 00000100:00100000:2.0:1789760777.208162:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720441734675 time 720441474259 state 0x1 ref 1 00000100:00100000:2.0:1789760777.208193:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720441734675 time 720453812078 state 0x0 ref 1 00000100:00100000:2.0:1789760777.208206:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d81500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554743808:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.208257:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63307776, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e318 00010000:00010000:2.0:1789760777.208266:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50d6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.208281:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760777.208285:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.208290:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63307776->63311871] (req 63307776->63311871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.208303:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63307776->63311871] (req 63307776->63311871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.218302:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982765 is committed 00002000:00010000:2.0:1789760777.218433:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63307776 -> 63311872 00002000:00010000:2.0:1789760777.218438:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123672 -> 123680 00010000:00010000:2.0:1789760777.218452:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63307776->63311871] (req 63307776->63311871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.218470:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50d6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63307776->63311871] (req 63307776->63311871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.218590:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63307776->63311871] (req 63307776->63311871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.218598:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.218602:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63307776->63311871] (req 63307776->63311871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.218612:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50d6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63307776->63311871] (req 63307776->63311871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.218632:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50d6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63307776->63311871] (req 63307776->63311871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.218640:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c50d6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63307776->63311871] (req 63307776->63311871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.218867:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d81500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554743808:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10657us (10922us total) trans 4294982765 rc 0/0 00000100:00100000:2.0:1789760777.218877:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15467 00000100:00100000:2.0:1789760777.218895:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720441734675 time 720453812078 state 0x2 ref 0 00000100:00100000:1.0:1789760777.223491:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.223567:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554743936 00000100:00100000:2.0:1789760777.223576:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554743936 00000100:00100000:2.0:1789760777.223627:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720441734675 time 720453812078 state 0x0 ref 1 00000100:00100000:2.0:1789760777.223644:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720454072494 time 720453812078 state 0x1 ref 1 00000100:00100000:2.0:1789760777.223667:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720454072494 time 720469285512 state 0x0 ref 1 00000100:00100000:2.0:1789760777.223679:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554743936:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.223744:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63311872, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e320 00010000:00010000:2.0:1789760777.223753:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c50dd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.223769:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760777.223774:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.223781:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c50dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63311872->63315967] (req 63311872->63315967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.223795:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c50dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63311872->63315967] (req 63311872->63315967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.230784:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982766 is committed 00002000:00010000:2.0:1789760777.230963:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63311872 -> 63315968 00002000:00010000:2.0:1789760777.230969:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123680 -> 123688 00010000:00010000:2.0:1789760777.230982:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c50dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63311872->63315967] (req 63311872->63315967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.230995:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c50dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63311872->63315967] (req 63311872->63315967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.231009:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c50dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63311872->63315967] (req 63311872->63315967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.231016:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.231020:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c50dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63311872->63315967] (req 63311872->63315967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.231029:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c50dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63311872->63315967] (req 63311872->63315967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.231047:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c50dd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63311872->63315967] (req 63311872->63315967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.231055:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c50dd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63311872->63315967] (req 63311872->63315967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.231319:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554743936:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7636us (7832us total) trans 4294982766 rc 0/0 00000100:00100000:2.0:1789760777.231330:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15468 00000100:00100000:2.0:1789760777.231340:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720454072494 time 720469285512 state 0x2 ref 0 00000100:00100000:3.0:1789760777.235298:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760777.235644:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554744064 00000100:00100000:1.0:1789760777.235653:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554744064 00000100:00100000:1.0:1789760777.235700:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720454072494 time 720469285512 state 0x0 ref 1 00000100:00100000:1.0:1789760777.235730:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720469545928 time 720469285512 state 0x1 ref 1 00000100:00100000:1.0:1789760777.235755:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720469545928 time 720481374508 state 0x0 ref 1 00000100:00100000:1.0:1789760777.235767:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f16680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554744064:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760777.235814:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63315968, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e328 00010000:00010000:1.0:1789760777.235824:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50e4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.235839:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760777.235844:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.235851:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63315968->63320063] (req 63315968->63320063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.235864:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63315968->63320063] (req 63315968->63320063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.244535:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982767 is committed 00002000:00010000:1.0:1789760777.244811:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63315968 -> 63320064 00002000:00010000:1.0:1789760777.244817:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123688 -> 123696 00010000:00010000:1.0:1789760777.244830:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63315968->63320063] (req 63315968->63320063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.244843:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63315968->63320063] (req 63315968->63320063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.244852:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63315968->63320063] (req 63315968->63320063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.244859:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760777.244864:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63315968->63320063] (req 63315968->63320063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.244879:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50e4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63315968->63320063] (req 63315968->63320063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.245023:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50e4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63315968->63320063] (req 63315968->63320063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.245032:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50e4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63315968->63320063] (req 63315968->63320063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.245225:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f16680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554744064:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9455us (9932us total) trans 4294982767 rc 0/0 00000100:00100000:1.0:1789760777.245236:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15469 00000100:00100000:1.0:1789760777.245245:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720469545928 time 720481374508 state 0x2 ref 0 00000100:00100000:1.0:1789760777.248768:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760777.248864:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554744192 00000100:00100000:0.0:1789760777.248874:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554744192 00000100:00100000:0.0:1789760777.248929:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720469545928 time 720481374508 state 0x0 ref 1 00000100:00100000:0.0:1789760777.248945:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720481634924 time 720481374508 state 0x1 ref 1 00000100:00100000:0.0:1789760777.248980:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720481634924 time 720494598553 state 0x0 ref 1 00000100:00100000:0.0:1789760777.248993:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d83100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554744192:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760777.249037:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63320064, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e330 00010000:00010000:0.0:1789760777.249047:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c50eb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.249062:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760777.249066:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.249072:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c50eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63320064->63324159] (req 63320064->63324159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.249085:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c50eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63320064->63324159] (req 63320064->63324159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.256149:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982768 is committed 00002000:00010000:0.0:1789760777.256610:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63320064 -> 63324160 00002000:00010000:0.0:1789760777.256615:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123696 -> 123704 00010000:00010000:0.0:1789760777.256628:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c50eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63320064->63324159] (req 63320064->63324159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.256641:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c50eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63320064->63324159] (req 63320064->63324159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.256651:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c50eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63320064->63324159] (req 63320064->63324159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.256659:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760777.256665:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c50eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63320064->63324159] (req 63320064->63324159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.256674:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c50eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63320064->63324159] (req 63320064->63324159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.256694:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c50eb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63320064->63324159] (req 63320064->63324159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.256701:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c50eb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63320064->63324159] (req 63320064->63324159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.256801:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d83100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554744192:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7806us (8039us total) trans 4294982768 rc 0/0 00000100:00100000:0.0:1789760777.256811:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15470 00000100:00100000:0.0:1789760777.256820:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720481634924 time 720494598553 state 0x2 ref 0 00000100:00100000:2.0:1789760777.261766:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760777.261864:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554744320 00000100:00100000:1.0:1789760777.261873:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554744320 00000100:00100000:1.0:1789760777.261928:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720481634924 time 720494598553 state 0x0 ref 1 00000100:00100000:1.0:1789760777.261943:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720494858969 time 720494598553 state 0x1 ref 1 00000100:00100000:1.0:1789760777.261974:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720494858969 time 720507592361 state 0x0 ref 1 00000100:00100000:1.0:1789760777.261986:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554744320:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760777.262032:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63324160, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e338 00010000:00010000:1.0:1789760777.262041:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50f2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.262057:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760777.262061:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.262068:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63324160->63328255] (req 63324160->63328255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.262081:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63324160->63328255] (req 63324160->63328255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.268920:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982769 is committed 00002000:00010000:1.0:1789760777.270120:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63324160 -> 63328256 00002000:00010000:1.0:1789760777.270127:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123704 -> 123712 00010000:00010000:1.0:1789760777.270140:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63324160->63328255] (req 63324160->63328255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.270153:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50f2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63324160->63328255] (req 63324160->63328255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.270163:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63324160->63328255] (req 63324160->63328255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.270170:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760777.270175:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63324160->63328255] (req 63324160->63328255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.270183:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50f2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63324160->63328255] (req 63324160->63328255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.270201:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50f2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63324160->63328255] (req 63324160->63328255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.270208:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c50f2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63324160->63328255] (req 63324160->63328255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.270452:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554744320:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8462us (8690us total) trans 4294982769 rc 0/0 00000100:00100000:1.0:1789760777.270463:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15471 00000100:00100000:1.0:1789760777.270472:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720494858969 time 720507592361 state 0x2 ref 0 00000100:00100000:3.0:1789760777.283609:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.284235:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554744448 00000100:00100000:2.0:1789760777.284247:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554744448 00000100:00100000:2.0:1789760777.284371:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720494858969 time 720507592361 state 0x0 ref 1 00000100:00100000:2.0:1789760777.284552:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720507852777 time 720507592361 state 0x1 ref 1 00000100:00100000:2.0:1789760777.284624:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720507852777 time 720530241116 state 0x0 ref 1 00000100:00100000:2.0:1789760777.284639:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554744448:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.284689:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63328256, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e340 00010000:00010000:2.0:1789760777.284699:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c50f9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.284714:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760777.284728:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.284734:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c50f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63328256->63332351] (req 63328256->63332351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.284749:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c50f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63328256->63332351] (req 63328256->63332351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.297754:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982770 is committed 00002000:00010000:2.0:1789760777.297875:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63328256 -> 63332352 00002000:00010000:2.0:1789760777.297880:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123712 -> 123720 00010000:00010000:2.0:1789760777.297894:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c50f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63328256->63332351] (req 63328256->63332351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.297905:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c50f9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63328256->63332351] (req 63328256->63332351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.297919:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c50f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63328256->63332351] (req 63328256->63332351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.297926:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.297931:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c50f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63328256->63332351] (req 63328256->63332351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.297940:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c50f9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63328256->63332351] (req 63328256->63332351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.297959:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c50f9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63328256->63332351] (req 63328256->63332351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.297965:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c50f9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63328256->63332351] (req 63328256->63332351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.298053:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554744448:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13414us (14447us total) trans 4294982770 rc 0/0 00000100:00100000:2.0:1789760777.298063:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15472 00000100:00100000:2.0:1789760777.298072:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720507852777 time 720530241116 state 0x2 ref 0 00000100:00100000:2.0:1789760777.302711:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760777.302962:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554744576 00000100:00100000:1.0:1789760777.302973:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554744576 00000100:00100000:1.0:1789760777.303024:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720507852777 time 720530241116 state 0x0 ref 1 00000100:00100000:1.0:1789760777.303039:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720530501532 time 720530241116 state 0x1 ref 1 00000100:00100000:1.0:1789760777.303073:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720530501532 time 720548690866 state 0x0 ref 1 00000100:00100000:1.0:1789760777.303085:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04562d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554744576:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760777.303136:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63332352, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e348 00010000:00010000:1.0:1789760777.303146:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5100 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.303161:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760777.303166:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.303173:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5100 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63332352->63336447] (req 63332352->63336447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.303186:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5100 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63332352->63336447] (req 63332352->63336447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.309819:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982771 is committed 00002000:00010000:1.0:1789760777.310050:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63332352 -> 63336448 00002000:00010000:1.0:1789760777.310055:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123720 -> 123728 00010000:00010000:1.0:1789760777.310078:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5100 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63332352->63336447] (req 63332352->63336447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.310202:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5100 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63332352->63336447] (req 63332352->63336447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.310214:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5100 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63332352->63336447] (req 63332352->63336447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.310222:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760777.310227:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5100 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63332352->63336447] (req 63332352->63336447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.310237:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5100 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63332352->63336447] (req 63332352->63336447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.310258:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5100 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63332352->63336447] (req 63332352->63336447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.310271:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5100 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63332352->63336447] (req 63332352->63336447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.310361:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04562d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554744576:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7272us (7655us total) trans 4294982771 rc 0/0 00000100:00100000:1.0:1789760777.310371:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15473 00000100:00100000:1.0:1789760777.311186:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720530501532 time 720548690866 state 0x2 ref 0 00000100:00100000:1.0:1789760777.316935:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760777.317168:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554744704 00000100:00100000:0.0:1789760777.317178:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554744704 00000100:00100000:0.0:1789760777.317232:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720530501532 time 720548690866 state 0x0 ref 1 00000100:00100000:0.0:1789760777.317249:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720548951282 time 720548690866 state 0x1 ref 1 00000100:00100000:0.0:1789760777.317282:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720548951282 time 720562900939 state 0x0 ref 1 00000100:00100000:0.0:1789760777.317295:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d83480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554744704:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760777.317340:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63336448, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e350 00010000:00010000:0.0:1789760777.317349:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5107 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.317364:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760777.317369:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.317515:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5107 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63336448->63340543] (req 63336448->63340543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.317530:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5107 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63336448->63340543] (req 63336448->63340543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.325287:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982772 is committed 00002000:00010000:0.0:1789760777.325503:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63336448 -> 63340544 00002000:00010000:0.0:1789760777.325510:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123728 -> 123736 00010000:00010000:0.0:1789760777.325525:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5107 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63336448->63340543] (req 63336448->63340543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.325538:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5107 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63336448->63340543] (req 63336448->63340543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.325547:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5107 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63336448->63340543] (req 63336448->63340543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.325554:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760777.325558:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5107 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63336448->63340543] (req 63336448->63340543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.325567:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5107 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63336448->63340543] (req 63336448->63340543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.325586:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5107 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63336448->63340543] (req 63336448->63340543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.325594:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5107 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63336448->63340543] (req 63336448->63340543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.325698:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d83480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554744704:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8398us (8878us total) trans 4294982772 rc 0/0 00000100:00100000:0.0:1789760777.325709:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15474 00000100:00100000:0.0:1789760777.325729:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720548951282 time 720562900939 state 0x2 ref 0 00000100:00100000:3.0:1789760777.330351:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.330666:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554744832 00000100:00100000:2.0:1789760777.330677:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554744832 00000100:00100000:2.0:1789760777.330747:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720548951282 time 720562900939 state 0x0 ref 1 00000100:00100000:2.0:1789760777.330765:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720563161355 time 720562900939 state 0x1 ref 1 00000100:00100000:2.0:1789760777.330839:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720563161355 time 720576456719 state 0x0 ref 1 00000100:00100000:2.0:1789760777.330851:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554744832:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.330898:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63340544, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e358 00010000:00010000:2.0:1789760777.330906:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c510e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.330921:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760777.330926:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.330932:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c510e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63340544->63344639] (req 63340544->63344639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.330946:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c510e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63340544->63344639] (req 63340544->63344639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.336346:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982773 is committed 00002000:00010000:2.0:1789760777.336726:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63340544 -> 63344640 00002000:00010000:2.0:1789760777.336732:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123736 -> 123744 00010000:00010000:2.0:1789760777.336746:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c510e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63340544->63344639] (req 63340544->63344639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.336760:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c510e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63340544->63344639] (req 63340544->63344639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.336770:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c510e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63340544->63344639] (req 63340544->63344639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.336779:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.336784:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c510e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63340544->63344639] (req 63340544->63344639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.336793:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c510e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63340544->63344639] (req 63340544->63344639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.336813:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c510e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63340544->63344639] (req 63340544->63344639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.336822:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c510e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63340544->63344639] (req 63340544->63344639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.336922:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554744832:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6067us (6574us total) trans 4294982773 rc 0/0 00000100:00100000:2.0:1789760777.336933:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15475 00000100:00100000:2.0:1789760777.336943:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720563161355 time 720576456719 state 0x2 ref 0 00000100:00100000:2.0:1789760777.341924:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760777.342010:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554744960 00000100:00100000:0.0:1789760777.342020:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554744960 00000100:00100000:0.0:1789760777.342085:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720563161355 time 720576456719 state 0x0 ref 1 00000100:00100000:0.0:1789760777.342108:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720576717135 time 720576456719 state 0x1 ref 1 00000100:00100000:0.0:1789760777.342138:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720576717135 time 720587756983 state 0x0 ref 1 00000100:00100000:0.0:1789760777.342151:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04563800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554744960:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760777.342204:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63344640, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e360 00010000:00010000:0.0:1789760777.342214:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5115 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.342230:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760777.342234:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.342240:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5115 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63344640->63348735] (req 63344640->63348735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.342259:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5115 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63344640->63348735] (req 63344640->63348735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.347822:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982774 is committed 00002000:00010000:2.0:1789760777.348001:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63344640 -> 63348736 00002000:00010000:2.0:1789760777.348007:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123744 -> 123752 00010000:00010000:2.0:1789760777.348020:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5115 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63344640->63348735] (req 63344640->63348735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.348031:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5115 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63344640->63348735] (req 63344640->63348735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.348588:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5115 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63344640->63348735] (req 63344640->63348735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.348596:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.348600:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5115 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63344640->63348735] (req 63344640->63348735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.348609:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5115 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63344640->63348735] (req 63344640->63348735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.348630:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5115 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63344640->63348735] (req 63344640->63348735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.348637:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5115 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63344640->63348735] (req 63344640->63348735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.348863:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04563800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554744960:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6709us (6942us total) trans 4294982774 rc 0/0 00000100:00100000:2.0:1789760777.348874:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15476 00000100:00100000:2.0:1789760777.348884:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720576717135 time 720587756983 state 0x2 ref 0 00000100:00100000:1.0:1789760777.354857:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.354986:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554745088 00000100:00100000:2.0:1789760777.354996:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554745088 00000100:00100000:2.0:1789760777.355170:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720576717135 time 720587756983 state 0x0 ref 1 00000100:00100000:2.0:1789760777.355187:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720588017399 time 720587756983 state 0x1 ref 1 00000100:00100000:2.0:1789760777.355259:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720588017399 time 720600876517 state 0x0 ref 1 00000100:00100000:2.0:1789760777.355273:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554745088:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.355323:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63348736, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e368 00010000:00010000:2.0:1789760777.355333:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c511c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.355349:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760777.355353:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.355360:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c511c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63348736->63352831] (req 63348736->63352831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.355402:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c511c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63348736->63352831] (req 63348736->63352831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.361886:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982775 is committed 00002000:00010000:2.0:1789760777.362283:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63348736 -> 63352832 00002000:00010000:2.0:1789760777.362289:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123752 -> 123760 00010000:00010000:2.0:1789760777.362302:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c511c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63348736->63352831] (req 63348736->63352831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.362314:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c511c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63348736->63352831] (req 63348736->63352831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.362324:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c511c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63348736->63352831] (req 63348736->63352831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.362337:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.362342:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c511c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63348736->63352831] (req 63348736->63352831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.362350:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c511c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63348736->63352831] (req 63348736->63352831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.362728:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c511c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63348736->63352831] (req 63348736->63352831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.362740:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c511c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63348736->63352831] (req 63348736->63352831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.362863:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554745088:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7587us (8009us total) trans 4294982775 rc 0/0 00000100:00100000:2.0:1789760777.362876:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15477 00000100:00100000:2.0:1789760777.362892:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720588017399 time 720600876517 state 0x2 ref 0 00000100:00100000:3.0:1789760777.368926:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.368971:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554745216 00000100:00100000:2.0:1789760777.368981:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554745216 00000100:00100000:2.0:1789760777.369154:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720588017399 time 720600876517 state 0x0 ref 1 00000100:00100000:2.0:1789760777.369173:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720601136933 time 720600876517 state 0x1 ref 1 00000100:00100000:2.0:1789760777.369202:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720601136933 time 720614821555 state 0x0 ref 1 00000100:00100000:2.0:1789760777.369218:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f14e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554745216:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.369270:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63352832, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e370 00010000:00010000:2.0:1789760777.369278:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5123 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.369293:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760777.369298:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.369303:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5123 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63352832->63356927] (req 63352832->63356927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.369316:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5123 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63352832->63356927] (req 63352832->63356927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.377322:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982776 is committed 00002000:00010000:2.0:1789760777.377536:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63352832 -> 63356928 00002000:00010000:2.0:1789760777.377541:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123760 -> 123768 00010000:00010000:2.0:1789760777.377554:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5123 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63352832->63356927] (req 63352832->63356927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.377566:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5123 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63352832->63356927] (req 63352832->63356927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.377576:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5123 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63352832->63356927] (req 63352832->63356927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.377582:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.377586:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5123 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63352832->63356927] (req 63352832->63356927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.377593:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5123 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63352832->63356927] (req 63352832->63356927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.377609:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5123 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63352832->63356927] (req 63352832->63356927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.377613:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5123 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63352832->63356927] (req 63352832->63356927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.377768:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f14e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554745216:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8547us (8847us total) trans 4294982776 rc 0/0 00000100:00100000:2.0:1789760777.377778:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15478 00000100:00100000:2.0:1789760777.377787:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720601136933 time 720614821555 state 0x2 ref 0 00000100:00100000:1.0:1789760777.383926:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.384197:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554745344 00000100:00100000:2.0:1789760777.384206:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554745344 00000100:00100000:2.0:1789760777.384259:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720601136933 time 720614821555 state 0x0 ref 1 00000100:00100000:2.0:1789760777.384274:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720615081971 time 720614821555 state 0x1 ref 1 00000100:00100000:2.0:1789760777.384305:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720615081971 time 720629923247 state 0x0 ref 1 00000100:00100000:2.0:1789760777.384317:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d80700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554745344:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.384363:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63356928, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e378 00010000:00010000:2.0:1789760777.384372:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c512a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.384473:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760777.384478:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.384485:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c512a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63356928->63361023] (req 63356928->63361023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.384498:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c512a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63356928->63361023] (req 63356928->63361023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.392047:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982777 is committed 00002000:00010000:1.0:1789760777.392172:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63356928 -> 63361024 00002000:00010000:1.0:1789760777.392178:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123768 -> 123776 00010000:00010000:1.0:1789760777.392191:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c512a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63356928->63361023] (req 63356928->63361023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.392204:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c512a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63356928->63361023] (req 63356928->63361023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.392214:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c512a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63356928->63361023] (req 63356928->63361023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.392221:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760777.392226:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c512a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63356928->63361023] (req 63356928->63361023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.392241:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c512a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63356928->63361023] (req 63356928->63361023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.392262:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c512a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63356928->63361023] (req 63356928->63361023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.392270:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c512a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63356928->63361023] (req 63356928->63361023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.392357:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d80700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554745344:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8037us (8436us total) trans 4294982777 rc 0/0 00000100:00100000:1.0:1789760777.392368:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15479 00000100:00100000:1.0:1789760777.392431:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720615081971 time 720629923247 state 0x2 ref 0 00000100:00100000:1.0:1789760777.408727:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760777.408835:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554745472 00000100:00100000:0.0:1789760777.408854:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554745472 00000100:00100000:0.0:1789760777.408916:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720615081971 time 720629923247 state 0x0 ref 1 00000100:00100000:0.0:1789760777.408935:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720630183663 time 720629923247 state 0x1 ref 1 00000100:00100000:0.0:1789760777.409023:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720630183663 time 720654639640 state 0x0 ref 1 00000100:00100000:0.0:1789760777.409039:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d80380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554745472:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760777.409087:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63361024, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e380 00010000:00010000:0.0:1789760777.409097:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5131 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.409115:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760777.409119:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.409126:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5131 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63361024->63365119] (req 63361024->63365119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.409140:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5131 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63361024->63365119] (req 63361024->63365119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.415081:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982778 is committed 00002000:00010000:2.0:1789760777.415513:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63361024 -> 63365120 00002000:00010000:2.0:1789760777.415519:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123776 -> 123784 00010000:00010000:2.0:1789760777.415533:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5131 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63361024->63365119] (req 63361024->63365119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.415546:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5131 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63361024->63365119] (req 63361024->63365119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.415556:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5131 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63361024->63365119] (req 63361024->63365119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.415564:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.415568:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5131 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63361024->63365119] (req 63361024->63365119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.415582:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5131 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63361024->63365119] (req 63361024->63365119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.415613:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5131 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63361024->63365119] (req 63361024->63365119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.415621:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5131 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63361024->63365119] (req 63361024->63365119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.415773:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d80380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554745472:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6733us (7063us total) trans 4294982778 rc 0/0 00000100:00100000:2.0:1789760777.415786:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15480 00000100:00100000:2.0:1789760777.415796:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720630183663 time 720654639640 state 0x2 ref 0 00000100:00100000:3.0:1789760777.425745:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760777.426174:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554745600 00000100:00100000:1.0:1789760777.426188:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554745600 00000100:00100000:1.0:1789760777.426248:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720630183663 time 720654639640 state 0x0 ref 1 00000100:00100000:1.0:1789760777.426263:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720654900056 time 720654639640 state 0x1 ref 1 00000100:00100000:2.0:1789760777.426603:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720654900056 time 720672218465 state 0x0 ref 1 00000100:00100000:2.0:1789760777.426622:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f15180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554745600:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.426670:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63365120, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e388 00010000:00010000:2.0:1789760777.426680:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5138 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.426697:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760777.426702:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.426709:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5138 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63365120->63369215] (req 63365120->63369215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.426735:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5138 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63365120->63369215] (req 63365120->63369215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.433832:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982779 is committed 00002000:00010000:2.0:1789760777.433955:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63365120 -> 63369216 00002000:00010000:2.0:1789760777.433961:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123784 -> 123792 00010000:00010000:2.0:1789760777.433974:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5138 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63365120->63369215] (req 63365120->63369215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.433987:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5138 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63365120->63369215] (req 63365120->63369215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.433997:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5138 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63365120->63369215] (req 63365120->63369215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.434003:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.434008:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5138 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63365120->63369215] (req 63365120->63369215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.434017:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5138 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63365120->63369215] (req 63365120->63369215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.434038:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5138 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63365120->63369215] (req 63365120->63369215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.434046:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5138 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63365120->63369215] (req 63365120->63369215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.434150:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f15180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554745600:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7529us (8427us total) trans 4294982779 rc 0/0 00000100:00100000:2.0:1789760777.434161:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15481 00000100:00100000:2.0:1789760777.434171:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720654900056 time 720672218465 state 0x2 ref 0 00000100:00100000:1.0:1789760777.438252:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.440374:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554745728 00000100:00100000:2.0:1789760777.440427:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554745728 00000100:00100000:2.0:1789760777.440485:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720654900056 time 720672218465 state 0x0 ref 1 00000100:00100000:2.0:1789760777.440500:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720672478881 time 720672218465 state 0x1 ref 1 00000100:00100000:2.0:1789760777.440532:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720672478881 time 720686150731 state 0x0 ref 1 00000100:00100000:2.0:1789760777.440546:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554745728:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.440594:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63369216, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e390 00010000:00010000:2.0:1789760777.440603:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c513f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.440619:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760777.440623:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.440629:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c513f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63369216->63373311] (req 63369216->63373311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.440644:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c513f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63369216->63373311] (req 63369216->63373311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.452612:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982780 is committed 00002000:00010000:3.0:1789760777.452805:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63369216 -> 63373312 00002000:00010000:3.0:1789760777.452810:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123792 -> 123800 00010000:00010000:3.0:1789760777.452825:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c513f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63369216->63373311] (req 63369216->63373311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.452837:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c513f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63369216->63373311] (req 63369216->63373311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.452846:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c513f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63369216->63373311] (req 63369216->63373311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.452854:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760777.452858:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c513f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63369216->63373311] (req 63369216->63373311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.452867:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c513f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63369216->63373311] (req 63369216->63373311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.452886:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c513f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63369216->63373311] (req 63369216->63373311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.452894:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c513f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63369216->63373311] (req 63369216->63373311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.453046:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554745728:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12499us (14800us total) trans 4294982780 rc 0/0 00000100:00100000:3.0:1789760777.453061:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15482 00000100:00100000:3.0:1789760777.453070:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720672478881 time 720686150731 state 0x2 ref 0 00000100:00100000:1.0:1789760777.465867:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.465964:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554745856 00000100:00100000:2.0:1789760777.465975:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554745856 00000100:00100000:2.0:1789760777.466028:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720672478881 time 720686150731 state 0x0 ref 1 00000100:00100000:2.0:1789760777.466042:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720686411147 time 720686150731 state 0x1 ref 1 00000100:00100000:2.0:1789760777.466073:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720686411147 time 720711692188 state 0x0 ref 1 00000100:00100000:2.0:1789760777.466085:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05d82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554745856:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.466446:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63373312, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e398 00010000:00010000:2.0:1789760777.466460:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5146 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.466474:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760777.466479:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.466486:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5146 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63373312->63377407] (req 63373312->63377407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.466500:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5146 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63373312->63377407] (req 63373312->63377407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.485445:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982781 is committed 00002000:00010000:1.0:1789760777.486702:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63373312 -> 63377408 00002000:00010000:1.0:1789760777.486710:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123800 -> 123808 00010000:00010000:1.0:1789760777.486737:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5146 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63373312->63377407] (req 63373312->63377407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.486751:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5146 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63373312->63377407] (req 63373312->63377407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.486760:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5146 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63373312->63377407] (req 63373312->63377407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.486766:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760777.486771:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5146 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63373312->63377407] (req 63373312->63377407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.486780:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5146 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63373312->63377407] (req 63373312->63377407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.486799:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5146 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63373312->63377407] (req 63373312->63377407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.486806:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5146 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63373312->63377407] (req 63373312->63377407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.486957:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05d82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554745856:12345-192.168.203.22@tcp:4:dd.0 Request processed in 20869us (21093us total) trans 4294982781 rc 0/0 00000100:00100000:1.0:1789760777.486969:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15483 00000100:00100000:1.0:1789760777.486978:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720686411147 time 720711692188 state 0x2 ref 0 00000100:00100000:1.0:1789760777.500144:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760777.503038:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554745984 00000100:00100000:0.0:1789760777.503053:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554745984 00000100:00100000:0.0:1789760777.503115:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720686411147 time 720711692188 state 0x0 ref 1 00000100:00100000:0.0:1789760777.503130:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720711952604 time 720711692188 state 0x1 ref 1 00000100:00100000:0.0:1789760777.503167:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720711952604 time 720748784704 state 0x0 ref 1 00000100:00100000:0.0:1789760777.503181:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554745984:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760777.503228:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63377408, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e3a0 00010000:00010000:0.0:1789760777.503237:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c514d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.503253:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760777.503257:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.503263:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c514d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63377408->63381503] (req 63377408->63381503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.503277:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c514d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63377408->63381503] (req 63377408->63381503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.513433:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982782 is committed 00002000:00010000:0.0:1789760777.514695:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63377408 -> 63381504 00002000:00010000:0.0:1789760777.514703:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123808 -> 123816 00010000:00010000:0.0:1789760777.514726:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c514d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63377408->63381503] (req 63377408->63381503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.514739:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c514d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63377408->63381503] (req 63377408->63381503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.514748:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c514d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63377408->63381503] (req 63377408->63381503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.514760:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760777.514765:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c514d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63377408->63381503] (req 63377408->63381503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.514773:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c514d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63377408->63381503] (req 63377408->63381503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.514792:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c514d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63377408->63381503] (req 63377408->63381503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.514800:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c514d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63377408->63381503] (req 63377408->63381503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.514944:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554745984:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11761us (14804us total) trans 4294982782 rc 0/0 00000100:00100000:0.0:1789760777.514955:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15484 00000100:00100000:0.0:1789760777.514964:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720711952604 time 720748784704 state 0x2 ref 0 00000100:00100000:2.0:1789760777.525264:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760777.525524:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554746112 00000100:00100000:1.0:1789760777.525534:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554746112 00000100:00100000:1.0:1789760777.525588:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720711952604 time 720748784704 state 0x0 ref 1 00000100:00100000:1.0:1789760777.525603:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720749045120 time 720748784704 state 0x1 ref 1 00000100:00100000:1.0:1789760777.525673:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720749045120 time 720771291496 state 0x0 ref 1 00000100:00100000:1.0:1789760777.525686:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554746112:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760777.525748:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63381504, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e3a8 00010000:00010000:1.0:1789760777.525758:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5154 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.525772:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760777.525776:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.525782:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5154 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63381504->63385599] (req 63381504->63385599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.525796:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5154 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63381504->63385599] (req 63381504->63385599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.530968:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982783 is committed 00002000:00010000:1.0:1789760777.531073:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63381504 -> 63385600 00002000:00010000:1.0:1789760777.531077:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123816 -> 123824 00010000:00010000:1.0:1789760777.531097:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5154 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63381504->63385599] (req 63381504->63385599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.531109:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5154 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63381504->63385599] (req 63381504->63385599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.531118:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5154 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63381504->63385599] (req 63381504->63385599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.531124:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760777.531129:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5154 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63381504->63385599] (req 63381504->63385599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.531138:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5154 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63381504->63385599] (req 63381504->63385599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.531155:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5154 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63381504->63385599] (req 63381504->63385599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.531162:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5154 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63381504->63385599] (req 63381504->63385599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.531251:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554746112:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5562us (5993us total) trans 4294982783 rc 0/0 00000100:00100000:1.0:1789760777.531261:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15485 00000100:00100000:1.0:1789760777.531270:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720749045120 time 720771291496 state 0x2 ref 0 00000100:00100000:1.0:1789760777.535899:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760777.536008:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554746240 00000100:00100000:3.0:1789760777.536018:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554746240 00000100:00100000:3.0:1789760777.536074:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720749045120 time 720771291496 state 0x0 ref 1 00000100:00100000:3.0:1789760777.536152:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720771551912 time 720771291496 state 0x1 ref 1 00000100:00100000:3.0:1789760777.536226:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720771551912 time 720781844243 state 0x0 ref 1 00000100:00100000:3.0:1789760777.536239:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554746240:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760777.536292:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63385600, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e3b0 00010000:00010000:3.0:1789760777.536302:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c515b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.536318:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:3.0:1789760777.536321:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760777.536326:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c515b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63385600->63389695] (req 63385600->63389695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.536336:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c515b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63385600->63389695] (req 63385600->63389695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760777.541575:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982784 is committed 00002000:00010000:3.0:1789760777.544848:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63385600 -> 63389696 00002000:00010000:3.0:1789760777.544858:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123824 -> 123832 00010000:00010000:3.0:1789760777.544872:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c515b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63385600->63389695] (req 63385600->63389695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.544885:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c515b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63385600->63389695] (req 63385600->63389695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.544893:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c515b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63385600->63389695] (req 63385600->63389695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.544900:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760777.544904:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c515b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63385600->63389695] (req 63385600->63389695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.544912:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c515b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63385600->63389695] (req 63385600->63389695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.544933:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c515b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63385600->63389695] (req 63385600->63389695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760777.544941:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c515b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63385600->63389695] (req 63385600->63389695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760777.545051:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554746240:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8808us (9327us total) trans 4294982784 rc 0/0 00000100:00100000:3.0:1789760777.545064:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15486 00000100:00100000:3.0:1789760777.545073:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720771551912 time 720781844243 state 0x2 ref 0 00000100:00100000:3.0:1789760777.553512:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.556451:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554746368 00000100:00100000:2.0:1789760777.556467:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554746368 00000100:00100000:2.0:1789760777.556535:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720771551912 time 720781844243 state 0x0 ref 1 00000100:00100000:2.0:1789760777.556563:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720782104659 time 720781844243 state 0x1 ref 1 00000100:00100000:2.0:1789760777.556657:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720782104659 time 720802274989 state 0x0 ref 1 00000100:00100000:2.0:1789760777.556672:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f17b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554746368:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.556739:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63389696, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e3b8 00010000:00010000:2.0:1789760777.556748:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5162 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.556763:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760777.556768:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.556774:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5162 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63389696->63393791] (req 63389696->63393791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.556787:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5162 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63389696->63393791] (req 63389696->63393791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.562433:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982785 is committed 00002000:00010000:2.0:1789760777.562550:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63389696 -> 63393792 00002000:00010000:2.0:1789760777.562555:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123832 -> 123840 00010000:00010000:2.0:1789760777.562570:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5162 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63389696->63393791] (req 63389696->63393791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.562583:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5162 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63389696->63393791] (req 63389696->63393791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.562592:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5162 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63389696->63393791] (req 63389696->63393791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.562599:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.562604:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5162 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63389696->63393791] (req 63389696->63393791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.562612:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5162 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63389696->63393791] (req 63389696->63393791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.562630:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5162 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63389696->63393791] (req 63389696->63393791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.562637:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5162 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63389696->63393791] (req 63389696->63393791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.562750:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f17b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554746368:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6077us (9242us total) trans 4294982785 rc 0/0 00000100:00100000:2.0:1789760777.562761:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15487 00000100:00100000:2.0:1789760777.562770:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720782104659 time 720802274989 state 0x2 ref 0 00000100:00100000:2.0:1789760777.569035:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760777.569209:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554746496 00000100:00100000:0.0:1789760777.569219:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554746496 00000100:00100000:0.0:1789760777.569275:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720782104659 time 720802274989 state 0x0 ref 1 00000100:00100000:0.0:1789760777.569303:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720802535405 time 720802274989 state 0x1 ref 1 00000100:00100000:0.0:1789760777.569557:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720802535405 time 720815173621 state 0x0 ref 1 00000100:00100000:0.0:1789760777.569572:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554746496:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760777.569635:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63393792, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e3c0 00010000:00010000:0.0:1789760777.569646:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5169 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.569664:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760777.569668:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.569677:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5169 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63393792->63397887] (req 63393792->63397887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.569690:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5169 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63393792->63397887] (req 63393792->63397887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.573931:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982786 is committed 00002000:00010000:0.0:1789760777.574038:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63393792 -> 63397888 00002000:00010000:0.0:1789760777.574043:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123840 -> 123848 00010000:00010000:0.0:1789760777.574056:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5169 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63393792->63397887] (req 63393792->63397887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.574068:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5169 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63393792->63397887] (req 63393792->63397887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.574077:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5169 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63393792->63397887] (req 63393792->63397887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.574084:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760777.574099:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5169 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63393792->63397887] (req 63393792->63397887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.574108:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5169 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63393792->63397887] (req 63393792->63397887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.574127:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5169 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63393792->63397887] (req 63393792->63397887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.574134:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5169 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63393792->63397887] (req 63393792->63397887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.574222:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554746496:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4648us (5191us total) trans 4294982786 rc 0/0 00000100:00100000:0.0:1789760777.574233:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15488 00000100:00100000:0.0:1789760777.574242:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720802535405 time 720815173621 state 0x2 ref 0 00000100:00100000:1.0:1789760777.580999:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.581511:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554746624 00000100:00100000:2.0:1789760777.581521:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554746624 00000100:00100000:2.0:1789760777.581573:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720802535405 time 720815173621 state 0x0 ref 1 00000100:00100000:2.0:1789760777.581643:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720815434037 time 720815173621 state 0x1 ref 1 00000100:00100000:2.0:1789760777.581713:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720815434037 time 720827330307 state 0x0 ref 1 00000100:00100000:2.0:1789760777.581738:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554746624:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.581783:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63397888, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e3c8 00010000:00010000:2.0:1789760777.581792:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5170 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.581809:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760777.581813:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.581820:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5170 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63397888->63401983] (req 63397888->63401983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.581833:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5170 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63397888->63401983] (req 63397888->63401983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.585317:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982787 is committed 00002000:00010000:2.0:1789760777.586651:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63397888 -> 63401984 00002000:00010000:2.0:1789760777.586658:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123848 -> 123856 00010000:00010000:2.0:1789760777.586673:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5170 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63397888->63401983] (req 63397888->63401983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.586694:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5170 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63397888->63401983] (req 63397888->63401983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.586703:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5170 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63397888->63401983] (req 63397888->63401983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.586711:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.586726:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5170 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63397888->63401983] (req 63397888->63401983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.586736:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5170 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63397888->63401983] (req 63397888->63401983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.586756:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5170 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63397888->63401983] (req 63397888->63401983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.586764:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5170 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63397888->63401983] (req 63397888->63401983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.586866:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554746624:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5126us (5872us total) trans 4294982787 rc 0/0 00000100:00100000:2.0:1789760777.587125:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15489 00000100:00100000:2.0:1789760777.587135:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720815434037 time 720827330307 state 0x2 ref 0 00000100:00100000:3.0:1789760777.591468:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760777.591508:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554746752 00000100:00100000:0.0:1789760777.591518:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554746752 00000100:00100000:0.0:1789760777.591572:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720815434037 time 720827330307 state 0x0 ref 1 00000100:00100000:0.0:1789760777.591587:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720827590723 time 720827330307 state 0x1 ref 1 00000100:00100000:0.0:1789760777.591614:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720827590723 time 720837232321 state 0x0 ref 1 00000100:00100000:0.0:1789760777.591625:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f16d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554746752:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760777.591671:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63401984, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e3d0 00010000:00010000:0.0:1789760777.591680:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5177 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.591696:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760777.591700:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.591706:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5177 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63401984->63406079] (req 63401984->63406079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.591734:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5177 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63401984->63406079] (req 63401984->63406079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.595766:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982788 is committed 00002000:00010000:0.0:1789760777.595881:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63401984 -> 63406080 00002000:00010000:0.0:1789760777.595886:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123856 -> 123864 00010000:00010000:0.0:1789760777.595899:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5177 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63401984->63406079] (req 63401984->63406079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.595912:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5177 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63401984->63406079] (req 63401984->63406079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.595920:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5177 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63401984->63406079] (req 63401984->63406079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.595928:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760777.595933:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5177 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63401984->63406079] (req 63401984->63406079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.595943:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5177 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63401984->63406079] (req 63401984->63406079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.595962:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5177 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63401984->63406079] (req 63401984->63406079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.595969:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5177 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63401984->63406079] (req 63401984->63406079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.596101:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f16d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554746752:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4472us (4638us total) trans 4294982788 rc 0/0 00000100:00100000:0.0:1789760777.596112:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15490 00000100:00100000:0.0:1789760777.596120:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720827590723 time 720837232321 state 0x2 ref 0 00000100:00100000:1.0:1789760777.602828:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.603197:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554746880 00000100:00100000:2.0:1789760777.603214:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554746880 00000100:00100000:2.0:1789760777.603265:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720827590723 time 720837232321 state 0x0 ref 1 00000100:00100000:2.0:1789760777.603288:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720837492737 time 720837232321 state 0x1 ref 1 00000100:00100000:2.0:1789760777.603318:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720837492737 time 720848936336 state 0x0 ref 1 00000100:00100000:2.0:1789760777.603335:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554746880:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.603414:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63406080, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e3d8 00010000:00010000:2.0:1789760777.603424:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c517e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.603438:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760777.603443:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.603449:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c517e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63406080->63410175] (req 63406080->63410175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.603463:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c517e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63406080->63410175] (req 63406080->63410175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.607645:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982789 is committed 00002000:00010000:2.0:1789760777.607947:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63406080 -> 63410176 00002000:00010000:2.0:1789760777.607952:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123864 -> 123872 00010000:00010000:2.0:1789760777.607964:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c517e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63406080->63410175] (req 63406080->63410175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.607976:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c517e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63406080->63410175] (req 63406080->63410175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.607986:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c517e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63406080->63410175] (req 63406080->63410175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.607993:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.607998:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c517e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63406080->63410175] (req 63406080->63410175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.608005:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c517e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63406080->63410175] (req 63406080->63410175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.608023:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c517e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63406080->63410175] (req 63406080->63410175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.608030:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c517e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63406080->63410175] (req 63406080->63410175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.608156:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554746880:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4819us (5333us total) trans 4294982789 rc 0/0 00000100:00100000:2.0:1789760777.608167:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15491 00000100:00100000:2.0:1789760777.608177:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720837492737 time 720848936336 state 0x2 ref 0 00000100:00100000:1.0:1789760777.612900:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.612995:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554747008 00000100:00100000:2.0:1789760777.613005:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554747008 00000100:00100000:2.0:1789760777.613057:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720837492737 time 720848936336 state 0x0 ref 1 00000100:00100000:2.0:1789760777.613073:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720849196752 time 720848936336 state 0x1 ref 1 00000100:00100000:2.0:1789760777.613150:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720849196752 time 720858767270 state 0x0 ref 1 00000100:00100000:2.0:1789760777.613163:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554747008:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.613212:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63410176, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e3e0 00010000:00010000:2.0:1789760777.613221:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5185 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.613235:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760777.613240:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.613246:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5185 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63410176->63414271] (req 63410176->63414271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.613260:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5185 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63410176->63414271] (req 63410176->63414271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.617687:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982790 is committed 00002000:00010000:2.0:1789760777.620847:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63410176 -> 63414272 00002000:00010000:2.0:1789760777.620857:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123872 -> 123880 00010000:00010000:2.0:1789760777.620869:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5185 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63410176->63414271] (req 63410176->63414271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.620882:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5185 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63410176->63414271] (req 63410176->63414271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.620891:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5185 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63410176->63414271] (req 63410176->63414271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.620899:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.620903:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5185 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63410176->63414271] (req 63410176->63414271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.620912:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5185 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63410176->63414271] (req 63410176->63414271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.620931:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5185 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63410176->63414271] (req 63410176->63414271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.620937:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5185 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63410176->63414271] (req 63410176->63414271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.621032:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554747008:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7865us (8136us total) trans 4294982790 rc 0/0 00000100:00100000:2.0:1789760777.621043:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15492 00000100:00100000:2.0:1789760777.621053:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720849196752 time 720858767270 state 0x2 ref 0 00000100:00100000:3.0:1789760777.625491:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760777.625826:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554747136 00000100:00100000:1.0:1789760777.625837:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554747136 00000100:00100000:1.0:1789760777.625891:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720849196752 time 720858767270 state 0x0 ref 1 00000100:00100000:1.0:1789760777.625907:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720859027686 time 720858767270 state 0x1 ref 1 00000100:00100000:1.0:1789760777.625938:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720859027686 time 720871556495 state 0x0 ref 1 00000100:00100000:1.0:1789760777.625950:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f16a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554747136:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760777.625994:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63414272, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e3e8 00010000:00010000:1.0:1789760777.626003:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c518c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.626019:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760777.626023:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.626029:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c518c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63414272->63418367] (req 63414272->63418367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.626043:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c518c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63414272->63418367] (req 63414272->63418367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.632574:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982791 is committed 00002000:00010000:2.0:1789760777.632745:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63414272 -> 63418368 00002000:00010000:2.0:1789760777.632750:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123880 -> 123888 00010000:00010000:2.0:1789760777.632763:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c518c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63414272->63418367] (req 63414272->63418367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.632775:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c518c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63414272->63418367] (req 63414272->63418367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.632784:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c518c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63414272->63418367] (req 63414272->63418367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.632791:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.632797:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c518c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63414272->63418367] (req 63414272->63418367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.632811:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c518c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63414272->63418367] (req 63414272->63418367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.632830:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c518c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63414272->63418367] (req 63414272->63418367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.632838:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c518c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63414272->63418367] (req 63414272->63418367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.632934:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f16a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554747136:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6980us (7457us total) trans 4294982791 rc 0/0 00000100:00100000:2.0:1789760777.632943:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15493 00000100:00100000:2.0:1789760777.632952:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720859027686 time 720871556495 state 0x2 ref 0 00000100:00100000:0.0:1789760777.640529:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760777.640609:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554747264 00000100:00100000:3.0:1789760777.640618:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554747264 00000100:00100000:3.0:1789760777.640738:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720859027686 time 720871556495 state 0x0 ref 1 00000100:00100000:3.0:1789760777.640756:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720871816911 time 720871556495 state 0x1 ref 1 00000100:00100000:1.0:1789760777.640897:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720871816911 time 720886513474 state 0x0 ref 1 00000100:00100000:1.0:1789760777.640914:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554747264:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760777.640963:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63418368, mtime: 0x6aad9508, atime: 0x0, ctime: 0x6aad9508, blocks: 0x1e3f0 00010000:00010000:1.0:1789760777.640972:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5193 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.640989:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760777.640993:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.640998:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5193 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63418368->63422463] (req 63418368->63422463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.641009:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5193 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63418368->63422463] (req 63418368->63422463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.646048:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982792 is committed 00002000:00010000:1.0:1789760777.646164:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63418368 -> 63422464 00002000:00010000:1.0:1789760777.646169:0:8443:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760776 -> 1789760777 00002000:00010000:1.0:1789760777.646172:0:8443:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760776 -> 1789760777 00002000:00010000:1.0:1789760777.646175:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123888 -> 123896 00010000:00010000:1.0:1789760777.646188:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5193 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63418368->63422463] (req 63418368->63422463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.646202:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5193 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63418368->63422463] (req 63418368->63422463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.646215:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5193 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63418368->63422463] (req 63418368->63422463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.646223:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760777.646228:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5193 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63418368->63422463] (req 63418368->63422463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.646236:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5193 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63418368->63422463] (req 63418368->63422463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.646256:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5193 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63418368->63422463] (req 63418368->63422463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.646264:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5193 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63418368->63422463] (req 63418368->63422463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.646441:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554747264:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5524us (5913us total) trans 4294982792 rc 0/0 00000100:00100000:1.0:1789760777.646451:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15494 00000100:00100000:1.0:1789760777.646460:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720871816911 time 720886513474 state 0x2 ref 0 00000100:00100000:1.0:1789760777.653260:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.653825:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554747392 00000100:00100000:2.0:1789760777.653835:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554747392 00000100:00100000:2.0:1789760777.653888:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720871816911 time 720886513474 state 0x0 ref 1 00000100:00100000:2.0:1789760777.653904:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720886773890 time 720886513474 state 0x1 ref 1 00000100:00100000:2.0:1789760777.653938:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720886773890 time 720899556477 state 0x0 ref 1 00000100:00100000:2.0:1789760777.653951:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554747392:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.653997:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63422464, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e3f8 00010000:00010000:2.0:1789760777.654007:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c519a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.654023:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760777.654028:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.654034:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c519a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63422464->63426559] (req 63422464->63426559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.654047:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c519a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63422464->63426559] (req 63422464->63426559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.660227:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982793 is committed 00002000:00010000:2.0:1789760777.660491:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63422464 -> 63426560 00002000:00010000:2.0:1789760777.660496:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123896 -> 123904 00010000:00010000:2.0:1789760777.660509:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c519a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63422464->63426559] (req 63422464->63426559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.660522:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c519a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63422464->63426559] (req 63422464->63426559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.660531:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c519a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63422464->63426559] (req 63422464->63426559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.660538:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.660543:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c519a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63422464->63426559] (req 63422464->63426559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.660552:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c519a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63422464->63426559] (req 63422464->63426559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.660576:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c519a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63422464->63426559] (req 63422464->63426559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.660584:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c519a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63422464->63426559] (req 63422464->63426559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.660709:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554747392:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6755us (7453us total) trans 4294982793 rc 0/0 00000100:00100000:2.0:1789760777.660729:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15495 00000100:00100000:2.0:1789760777.660738:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720886773890 time 720899556477 state 0x2 ref 0 00000100:00100000:3.0:1789760777.667763:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.667843:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554747520 00000100:00100000:2.0:1789760777.667853:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554747520 00000100:00100000:2.0:1789760777.667907:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720886773890 time 720899556477 state 0x0 ref 1 00000100:00100000:2.0:1789760777.667922:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720899816893 time 720899556477 state 0x1 ref 1 00000100:00100000:2.0:1789760777.667957:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720899816893 time 720913575296 state 0x0 ref 1 00000100:00100000:2.0:1789760777.667970:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f15f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554747520:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.668012:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63426560, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e400 00010000:00010000:2.0:1789760777.668021:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51a1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.668036:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760777.668040:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.668046:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63426560->63430655] (req 63426560->63430655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.668057:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63426560->63430655] (req 63426560->63430655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.674891:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982794 is committed 00002000:00010000:2.0:1789760777.675002:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63426560 -> 63430656 00002000:00010000:2.0:1789760777.675008:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123904 -> 123912 00010000:00010000:2.0:1789760777.675021:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63426560->63430655] (req 63426560->63430655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.675034:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51a1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63426560->63430655] (req 63426560->63430655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.675043:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63426560->63430655] (req 63426560->63430655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.675051:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.675056:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63426560->63430655] (req 63426560->63430655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.675065:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51a1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63426560->63430655] (req 63426560->63430655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.675084:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51a1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63426560->63430655] (req 63426560->63430655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.675098:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51a1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63426560->63430655] (req 63426560->63430655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.675195:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f15f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554747520:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7222us (7437us total) trans 4294982794 rc 0/0 00000100:00100000:2.0:1789760777.675206:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15496 00000100:00100000:2.0:1789760777.675215:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720899816893 time 720913575296 state 0x2 ref 0 00000100:00100000:0.0:1789760777.680735:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760777.681901:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554747648 00000100:00100000:1.0:1789760777.681914:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554747648 00000100:00100000:1.0:1789760777.681971:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720899816893 time 720913575296 state 0x0 ref 1 00000100:00100000:1.0:1789760777.681986:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720913835712 time 720913575296 state 0x1 ref 1 00000100:00100000:1.0:1789760777.682116:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720913835712 time 720927733108 state 0x0 ref 1 00000100:00100000:1.0:1789760777.682131:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554747648:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760777.682177:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63430656, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e408 00010000:00010000:1.0:1789760777.682186:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c51a8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.682218:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760777.682222:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.682229:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c51a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63430656->63434751] (req 63430656->63434751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.682242:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c51a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63430656->63434751] (req 63430656->63434751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.688189:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982795 is committed 00002000:00010000:1.0:1789760777.688482:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63430656 -> 63434752 00002000:00010000:1.0:1789760777.688487:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123912 -> 123920 00010000:00010000:1.0:1789760777.688501:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c51a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63430656->63434751] (req 63430656->63434751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.688513:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c51a8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63430656->63434751] (req 63430656->63434751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.688523:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c51a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63430656->63434751] (req 63430656->63434751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.688530:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760777.688536:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c51a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63430656->63434751] (req 63430656->63434751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.688544:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c51a8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63430656->63434751] (req 63430656->63434751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.688568:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c51a8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63430656->63434751] (req 63430656->63434751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.688575:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c51a8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63430656->63434751] (req 63430656->63434751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.688735:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554747648:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6603us (8013us total) trans 4294982795 rc 0/0 00000100:00100000:1.0:1789760777.688747:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15497 00000100:00100000:1.0:1789760777.688755:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720913835712 time 720927733108 state 0x2 ref 0 00000100:00100000:1.0:1789760777.695491:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760777.695576:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554747776 00000100:00100000:0.0:1789760777.695586:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554747776 00000100:00100000:0.0:1789760777.695640:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720913835712 time 720927733108 state 0x0 ref 1 00000100:00100000:0.0:1789760777.695656:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720927993524 time 720927733108 state 0x1 ref 1 00000100:00100000:0.0:1789760777.695696:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720927993524 time 720941314077 state 0x0 ref 1 00000100:00100000:0.0:1789760777.695709:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554747776:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760777.695766:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63434752, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e410 00010000:00010000:0.0:1789760777.695776:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c51af lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.695792:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:0.0:1789760777.695797:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.695803:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c51af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63434752->63438847] (req 63434752->63438847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.695818:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c51af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63434752->63438847] (req 63434752->63438847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.702426:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982796 is committed 00002000:00010000:0.0:1789760777.702884:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63434752 -> 63438848 00002000:00010000:0.0:1789760777.702889:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123920 -> 123928 00010000:00010000:0.0:1789760777.702902:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c51af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63434752->63438847] (req 63434752->63438847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.702915:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c51af lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63434752->63438847] (req 63434752->63438847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.702923:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c51af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63434752->63438847] (req 63434752->63438847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.702931:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760777.702936:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c51af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63434752->63438847] (req 63434752->63438847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.702952:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c51af lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63434752->63438847] (req 63434752->63438847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.702971:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c51af lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63434752->63438847] (req 63434752->63438847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.702979:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c51af lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63434752->63438847] (req 63434752->63438847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.703076:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554747776:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7365us (7599us total) trans 4294982796 rc 0/0 00000100:00100000:0.0:1789760777.703087:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15498 00000100:00100000:0.0:1789760777.703104:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720927993524 time 720941314077 state 0x2 ref 0 00000100:00100000:3.0:1789760777.714205:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.714321:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554747904 00000100:00100000:2.0:1789760777.714331:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554747904 00000100:00100000:2.0:1789760777.714435:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720927993524 time 720941314077 state 0x0 ref 1 00000100:00100000:2.0:1789760777.714450:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720941574493 time 720941314077 state 0x1 ref 1 00000100:00100000:1.0:1789760777.714676:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720941574493 time 720960281707 state 0x0 ref 1 00000100:00100000:1.0:1789760777.714692:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554747904:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760777.714752:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63438848, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e418 00010000:00010000:1.0:1789760777.714761:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c51b6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.714778:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:1.0:1789760777.714782:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.714789:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c51b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63438848->63442943] (req 63438848->63442943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.714803:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c51b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63438848->63442943] (req 63438848->63442943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.722685:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982797 is committed 00002000:00010000:1.0:1789760777.722965:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63438848 -> 63442944 00002000:00010000:1.0:1789760777.722971:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123928 -> 123936 00010000:00010000:1.0:1789760777.722984:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c51b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63438848->63442943] (req 63438848->63442943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.722996:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c51b6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63438848->63442943] (req 63438848->63442943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.723004:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c51b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63438848->63442943] (req 63438848->63442943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.723011:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760777.723016:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c51b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63438848->63442943] (req 63438848->63442943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.723025:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c51b6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63438848->63442943] (req 63438848->63442943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.723044:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c51b6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63438848->63442943] (req 63438848->63442943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.723052:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c51b6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63438848->63442943] (req 63438848->63442943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.723156:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554747904:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8462us (8965us total) trans 4294982797 rc 0/0 00000100:00100000:1.0:1789760777.723168:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15499 00000100:00100000:1.0:1789760777.723178:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720941574493 time 720960281707 state 0x2 ref 0 00000100:00100000:1.0:1789760777.730422:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.730516:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554748032 00000100:00100000:2.0:1789760777.730525:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554748032 00000100:00100000:2.0:1789760777.730577:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720941574493 time 720960281707 state 0x0 ref 1 00000100:00100000:2.0:1789760777.730592:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720960542123 time 720960281707 state 0x1 ref 1 00000100:00100000:2.0:1789760777.730613:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720960542123 time 720976231956 state 0x0 ref 1 00000100:00100000:2.0:1789760777.730629:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554748032:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.730672:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63442944, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e420 00010000:00010000:2.0:1789760777.730681:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51bd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.730697:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760777.730702:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.730707:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63442944->63447039] (req 63442944->63447039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.730737:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63442944->63447039] (req 63442944->63447039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.738201:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982798 is committed 00002000:00010000:2.0:1789760777.738372:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63442944 -> 63447040 00002000:00010000:2.0:1789760777.738403:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123936 -> 123944 00010000:00010000:2.0:1789760777.738418:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63442944->63447039] (req 63442944->63447039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.738430:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51bd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63442944->63447039] (req 63442944->63447039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.738439:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63442944->63447039] (req 63442944->63447039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.738447:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.738452:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63442944->63447039] (req 63442944->63447039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.738461:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63442944->63447039] (req 63442944->63447039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.738487:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51bd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63442944->63447039] (req 63442944->63447039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.738495:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51bd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63442944->63447039] (req 63442944->63447039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.738588:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554748032:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7955us (8213us total) trans 4294982798 rc 0/0 00000100:00100000:2.0:1789760777.738598:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15500 00000100:00100000:2.0:1789760777.738608:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720960542123 time 720976231956 state 0x2 ref 0 00000100:00100000:3.0:1789760777.741899:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.742223:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554748160 00000100:00100000:2.0:1789760777.742233:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554748160 00000100:00100000:2.0:1789760777.742290:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720960542123 time 720976231956 state 0x0 ref 1 00000100:00100000:2.0:1789760777.742303:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720976492372 time 720976231956 state 0x1 ref 1 00000100:00100000:2.0:1789760777.742338:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720976492372 time 720987956177 state 0x0 ref 1 00000100:00100000:2.0:1789760777.742351:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06401180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554748160:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.742431:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63447040, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e428 00010000:00010000:2.0:1789760777.742440:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51c4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.742455:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=720s v=(0 0) 00010000:00010000:2.0:1789760777.742460:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.742466:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63447040->63451135] (req 63447040->63451135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.742489:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63447040->63451135] (req 63447040->63451135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.752355:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982799 is committed 00002000:00010000:2.0:1789760777.752540:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63447040 -> 63451136 00002000:00010000:2.0:1789760777.752545:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123944 -> 123952 00010000:00010000:2.0:1789760777.752560:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63447040->63451135] (req 63447040->63451135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.752573:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63447040->63451135] (req 63447040->63451135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.752583:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63447040->63451135] (req 63447040->63451135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.752590:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.752595:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63447040->63451135] (req 63447040->63451135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.752604:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63447040->63451135] (req 63447040->63451135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.752626:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51c4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63447040->63451135] (req 63447040->63451135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.752640:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63447040->63451135] (req 63447040->63451135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.752744:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06401180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554748160:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10390us (10860us total) trans 4294982799 rc 0/0 00000100:00100000:2.0:1789760777.752754:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15501 00000100:00100000:2.0:1789760777.752763:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720976492372 time 720987956177 state 0x2 ref 0 00000100:00100000:3.0:1789760777.759133:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760777.759203:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554748288 00000100:00100000:0.0:1789760777.759222:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554748288 00000100:00100000:0.0:1789760777.759271:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720976492372 time 720987956177 state 0x0 ref 1 00000100:00100000:0.0:1789760777.759286:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720988216593 time 720987956177 state 0x1 ref 1 00000100:00100000:0.0:1789760777.759319:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720988216593 time 721004937236 state 0x0 ref 1 00000100:00100000:0.0:1789760777.759332:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06401c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554748288:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760777.759407:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63451136, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e430 00010000:00010000:0.0:1789760777.759417:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c51cb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.759432:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760777.759437:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.759443:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c51cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63451136->63455231] (req 63451136->63455231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.759459:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c51cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63451136->63455231] (req 63451136->63455231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.765017:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982800 is committed 00002000:00010000:2.0:1789760777.765314:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63451136 -> 63455232 00002000:00010000:2.0:1789760777.765319:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123952 -> 123960 00010000:00010000:2.0:1789760777.765332:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c51cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63451136->63455231] (req 63451136->63455231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.765344:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c51cb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63451136->63455231] (req 63451136->63455231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.765353:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c51cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63451136->63455231] (req 63451136->63455231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.765361:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.765366:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c51cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63451136->63455231] (req 63451136->63455231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.765403:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c51cb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63451136->63455231] (req 63451136->63455231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.765431:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c51cb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63451136->63455231] (req 63451136->63455231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.765439:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c51cb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63451136->63455231] (req 63451136->63455231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.765564:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06401c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554748288:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6230us (6435us total) trans 4294982800 rc 0/0 00000100:00100000:2.0:1789760777.765575:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15502 00000100:00100000:2.0:1789760777.765585:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720988216593 time 721004937236 state 0x2 ref 0 00000100:00100000:1.0:1789760777.775979:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.776139:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554748416 00000100:00100000:2.0:1789760777.776149:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554748416 00000100:00100000:2.0:1789760777.776203:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 720988216593 time 721004937236 state 0x0 ref 1 00000100:00100000:2.0:1789760777.776219:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721005197652 time 721004937236 state 0x1 ref 1 00000100:00100000:2.0:1789760777.776250:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721005197652 time 721021868412 state 0x0 ref 1 00000100:00100000:2.0:1789760777.776262:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554748416:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.776309:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63455232, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e438 00010000:00010000:2.0:1789760777.776318:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51d2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.776334:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760777.776339:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.776346:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63455232->63459327] (req 63455232->63459327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.776359:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63455232->63459327] (req 63455232->63459327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.784725:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982801 is committed 00002000:00010000:2.0:1789760777.786594:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63455232 -> 63459328 00002000:00010000:2.0:1789760777.786601:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123960 -> 123968 00010000:00010000:2.0:1789760777.786615:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63455232->63459327] (req 63455232->63459327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.786628:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63455232->63459327] (req 63455232->63459327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.786637:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63455232->63459327] (req 63455232->63459327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.786645:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.786651:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63455232->63459327] (req 63455232->63459327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.786660:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63455232->63459327] (req 63455232->63459327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.786690:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51d2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63455232->63459327] (req 63455232->63459327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.787257:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51d2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63455232->63459327] (req 63455232->63459327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.787362:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554748416:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11097us (11388us total) trans 4294982801 rc 0/0 00000100:00100000:2.0:1789760777.787373:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15503 00000100:00100000:2.0:1789760777.787426:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721005197652 time 721021868412 state 0x2 ref 0 00000100:00100000:3.0:1789760777.794420:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.794529:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554748544 00000100:00100000:2.0:1789760777.794540:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554748544 00000100:00100000:2.0:1789760777.794594:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721005197652 time 721021868412 state 0x0 ref 1 00000100:00100000:2.0:1789760777.794611:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721022128828 time 721021868412 state 0x1 ref 1 00000100:00100000:2.0:1789760777.794646:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721022128828 time 721040265180 state 0x0 ref 1 00000100:00100000:2.0:1789760777.794659:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06402d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554748544:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.794727:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63459328, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e440 00010000:00010000:2.0:1789760777.794737:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51d9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.794760:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760777.794765:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.794771:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63459328->63463423] (req 63459328->63463423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.794789:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63459328->63463423] (req 63459328->63463423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.804026:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982802 is committed 00002000:00010000:2.0:1789760777.804190:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63459328 -> 63463424 00002000:00010000:2.0:1789760777.804196:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123968 -> 123976 00010000:00010000:2.0:1789760777.804208:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63459328->63463423] (req 63459328->63463423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.804221:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51d9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63459328->63463423] (req 63459328->63463423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.804231:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63459328->63463423] (req 63459328->63463423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.804239:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.804244:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63459328->63463423] (req 63459328->63463423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.804253:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51d9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63459328->63463423] (req 63459328->63463423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.804271:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51d9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63459328->63463423] (req 63459328->63463423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.804279:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51d9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63459328->63463423] (req 63459328->63463423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.804495:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06402d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554748544:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9833us (10121us total) trans 4294982802 rc 0/0 00000100:00100000:2.0:1789760777.804507:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15504 00000100:00100000:2.0:1789760777.804517:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721022128828 time 721040265180 state 0x2 ref 0 00010000:00010000:0.0:1789760777.812628:0:8453:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760777.814536:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760777.815867:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554748672 00000100:00100000:1.0:1789760777.815882:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554748672 00000100:00100000:0.0:1789760777.816971:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721022128828 time 721040265180 state 0x0 ref 1 00000100:00100000:0.0:1789760777.816990:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721040525596 time 721040265180 state 0x1 ref 1 00000100:00100000:0.0:1789760777.817076:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721040525596 time 721062694208 state 0x0 ref 1 00000100:00100000:0.0:1789760777.817103:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06400000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554748672:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760777.817154:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63463424, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e448 00010000:00010000:0.0:1789760777.817164:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c51e0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.817180:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760777.817186:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.817193:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c51e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63463424->63467519] (req 63463424->63467519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.817208:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c51e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63463424->63467519] (req 63463424->63467519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.824258:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982803 is committed 00002000:00010000:2.0:1789760777.825232:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63463424 -> 63467520 00002000:00010000:2.0:1789760777.825240:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123976 -> 123984 00010000:00010000:2.0:1789760777.825255:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c51e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63463424->63467519] (req 63463424->63467519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.825269:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c51e0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63463424->63467519] (req 63463424->63467519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.825278:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c51e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63463424->63467519] (req 63463424->63467519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.825285:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.825290:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c51e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63463424->63467519] (req 63463424->63467519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.825298:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c51e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63463424->63467519] (req 63463424->63467519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.825317:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c51e0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63463424->63467519] (req 63463424->63467519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.825331:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c51e0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63463424->63467519] (req 63463424->63467519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.825667:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06400000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554748672:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8574us (11135us total) trans 4294982803 rc 0/0 00000100:00100000:2.0:1789760777.825678:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15505 00000100:00100000:2.0:1789760777.825687:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721040525596 time 721062694208 state 0x2 ref 0 00000100:00100000:0.0:1789760777.830704:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760777.830797:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554748800 00000100:00100000:1.0:1789760777.830807:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554748800 00000100:00100000:1.0:1789760777.830862:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721040525596 time 721062694208 state 0x0 ref 1 00000100:00100000:1.0:1789760777.830875:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721062954624 time 721062694208 state 0x1 ref 1 00000100:00100000:1.0:1789760777.830912:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721062954624 time 721076530001 state 0x0 ref 1 00000100:00100000:1.0:1789760777.830924:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554748800:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760777.830973:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63467520, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e450 00010000:00010000:1.0:1789760777.830982:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c51e7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.830997:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760777.831001:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.831007:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c51e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63467520->63471615] (req 63467520->63471615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.831025:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c51e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63467520->63471615] (req 63467520->63471615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.836138:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982804 is committed 00002000:00010000:1.0:1789760777.836352:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63467520 -> 63471616 00002000:00010000:1.0:1789760777.836358:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123984 -> 123992 00010000:00010000:1.0:1789760777.836372:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c51e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63467520->63471615] (req 63467520->63471615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.836448:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c51e7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63467520->63471615] (req 63467520->63471615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.836458:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c51e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63467520->63471615] (req 63467520->63471615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.836466:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760777.836471:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c51e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63467520->63471615] (req 63467520->63471615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.836480:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c51e7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63467520->63471615] (req 63467520->63471615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.836500:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c51e7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63467520->63471615] (req 63467520->63471615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.836507:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c51e7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63467520->63471615] (req 63467520->63471615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.836633:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554748800:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5706us (5935us total) trans 4294982804 rc 0/0 00000100:00100000:1.0:1789760777.836644:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15506 00000100:00100000:1.0:1789760777.836653:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721062954624 time 721076530001 state 0x2 ref 0 00000100:00100000:3.0:1789760777.840511:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.840846:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554748928 00000100:00100000:2.0:1789760777.840856:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554748928 00000100:00100000:2.0:1789760777.840908:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721062954624 time 721076530001 state 0x0 ref 1 00000100:00100000:2.0:1789760777.840922:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721076790417 time 721076530001 state 0x1 ref 1 00000100:00100000:2.0:1789760777.840997:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721076790417 time 721086613957 state 0x0 ref 1 00000100:00100000:2.0:1789760777.841012:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06403100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554748928:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.841063:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63471616, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e458 00010000:00010000:2.0:1789760777.841072:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51ee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.841087:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760777.841149:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.841156:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63471616->63475711] (req 63471616->63475711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.841263:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63471616->63475711] (req 63471616->63475711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.846307:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982805 is committed 00002000:00010000:2.0:1789760777.846528:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63471616 -> 63475712 00002000:00010000:2.0:1789760777.846533:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 123992 -> 124000 00010000:00010000:2.0:1789760777.846547:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63471616->63475711] (req 63471616->63475711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.846558:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63471616->63475711] (req 63471616->63475711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.846566:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63471616->63475711] (req 63471616->63475711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.846573:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.846578:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63471616->63475711] (req 63471616->63475711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.846588:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63471616->63475711] (req 63471616->63475711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.846609:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51ee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63471616->63475711] (req 63471616->63475711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.846616:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c51ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63471616->63475711] (req 63471616->63475711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.846708:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06403100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554748928:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5695us (6202us total) trans 4294982805 rc 0/0 00000100:00100000:2.0:1789760777.846728:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15507 00000100:00100000:2.0:1789760777.846737:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721076790417 time 721086613957 state 0x2 ref 0 00000100:00100000:3.0:1789760777.851639:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.851725:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554749056 00000100:00100000:2.0:1789760777.851735:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554749056 00000100:00100000:2.0:1789760777.851787:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721076790417 time 721086613957 state 0x0 ref 1 00000100:00100000:2.0:1789760777.851801:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721086874373 time 721086613957 state 0x1 ref 1 00000100:00100000:2.0:1789760777.851836:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721086874373 time 721097454081 state 0x0 ref 1 00000100:00100000:2.0:1789760777.851847:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06400a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554749056:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.851893:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63475712, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e460 00010000:00010000:2.0:1789760777.851902:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51f5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.851931:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760777.851936:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.851942:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63475712->63479807] (req 63475712->63479807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.851954:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63475712->63479807] (req 63475712->63479807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.858317:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982806 is committed 00002000:00010000:2.0:1789760777.858492:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63475712 -> 63479808 00002000:00010000:2.0:1789760777.858498:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124000 -> 124008 00010000:00010000:2.0:1789760777.858510:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63475712->63479807] (req 63475712->63479807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.858522:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63475712->63479807] (req 63475712->63479807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.858532:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63475712->63479807] (req 63475712->63479807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.858538:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.858544:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63475712->63479807] (req 63475712->63479807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.858553:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63475712->63479807] (req 63475712->63479807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.858577:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51f5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63475712->63479807] (req 63475712->63479807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.858585:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51f5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63475712->63479807] (req 63475712->63479807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.858680:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06400a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554749056:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6829us (7046us total) trans 4294982806 rc 0/0 00000100:00100000:2.0:1789760777.858691:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15508 00000100:00100000:2.0:1789760777.858700:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721086874373 time 721097454081 state 0x2 ref 0 00000100:00100000:0.0:1789760777.866031:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.866349:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554749184 00000100:00100000:2.0:1789760777.866359:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554749184 00000100:00100000:2.0:1789760777.866605:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721086874373 time 721097454081 state 0x0 ref 1 00000100:00100000:2.0:1789760777.866623:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721097714497 time 721097454081 state 0x1 ref 1 00000100:00100000:2.0:1789760777.866697:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721097714497 time 721112314611 state 0x0 ref 1 00000100:00100000:2.0:1789760777.866710:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554749184:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.866764:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63479808, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e468 00010000:00010000:2.0:1789760777.866773:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51fc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.866790:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760777.866794:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.866801:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63479808->63483903] (req 63479808->63483903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.866817:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63479808->63483903] (req 63479808->63483903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.872732:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982807 is committed 00002000:00010000:2.0:1789760777.873262:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63479808 -> 63483904 00002000:00010000:2.0:1789760777.873269:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124008 -> 124016 00010000:00010000:2.0:1789760777.873282:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63479808->63483903] (req 63479808->63483903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.873294:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51fc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63479808->63483903] (req 63479808->63483903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.873302:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63479808->63483903] (req 63479808->63483903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.873309:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.873314:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63479808->63483903] (req 63479808->63483903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.873323:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51fc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63479808->63483903] (req 63479808->63483903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.873342:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51fc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63479808->63483903] (req 63479808->63483903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.873350:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c51fc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63479808->63483903] (req 63479808->63483903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.873706:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554749184:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6992us (7680us total) trans 4294982807 rc 0/0 00000100:00100000:2.0:1789760777.873726:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15509 00000100:00100000:2.0:1789760777.873737:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721097714497 time 721112314611 state 0x2 ref 0 00000100:00100000:3.0:1789760777.879584:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760777.880605:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554749312 00000100:00100000:0.0:1789760777.880617:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554749312 00000100:00100000:0.0:1789760777.880675:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721097714497 time 721112314611 state 0x0 ref 1 00000100:00100000:0.0:1789760777.880701:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721112575027 time 721112314611 state 0x1 ref 1 00000100:00100000:0.0:1789760777.880744:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721112575027 time 721126362306 state 0x0 ref 1 00000100:00100000:0.0:1789760777.880758:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06402680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554749312:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760777.880804:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63483904, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e470 00010000:00010000:0.0:1789760777.880814:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5203 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.880830:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760777.880834:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.880841:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5203 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63483904->63487999] (req 63483904->63487999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.880853:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5203 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63483904->63487999] (req 63483904->63487999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.885940:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982808 is committed 00002000:00010000:0.0:1789760777.886051:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63483904 -> 63488000 00002000:00010000:0.0:1789760777.886057:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124016 -> 124024 00010000:00010000:0.0:1789760777.886069:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5203 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63483904->63487999] (req 63483904->63487999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.886083:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5203 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63483904->63487999] (req 63483904->63487999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.886093:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5203 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63483904->63487999] (req 63483904->63487999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.886101:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760777.886106:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5203 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63483904->63487999] (req 63483904->63487999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.886116:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5203 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63483904->63487999] (req 63483904->63487999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.886138:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5203 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63483904->63487999] (req 63483904->63487999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.886144:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5203 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63483904->63487999] (req 63483904->63487999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.886245:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06402680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554749312:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5485us (6668us total) trans 4294982808 rc 0/0 00000100:00100000:0.0:1789760777.886256:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15510 00000100:00100000:0.0:1789760777.886266:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721112575027 time 721126362306 state 0x2 ref 0 00000100:00100000:3.0:1789760777.891485:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760777.892267:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554749440 00000100:00100000:1.0:1789760777.892277:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554749440 00000100:00100000:1.0:1789760777.892333:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721112575027 time 721126362306 state 0x0 ref 1 00000100:00100000:1.0:1789760777.892347:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721126622722 time 721126362306 state 0x1 ref 1 00000100:00100000:1.0:1789760777.892693:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721126622722 time 721138309468 state 0x0 ref 1 00000100:00100000:1.0:1789760777.892709:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06401500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554749440:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760777.892769:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63488000, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e478 00010000:00010000:1.0:1789760777.892779:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c520a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.892795:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760777.892799:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.892806:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c520a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63488000->63492095] (req 63488000->63492095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.892818:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c520a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63488000->63492095] (req 63488000->63492095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.899360:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982809 is committed 00002000:00010000:1.0:1789760777.902033:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63488000 -> 63492096 00002000:00010000:1.0:1789760777.902040:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124024 -> 124032 00010000:00010000:1.0:1789760777.902051:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c520a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63488000->63492095] (req 63488000->63492095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.902060:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c520a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63488000->63492095] (req 63488000->63492095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.902066:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c520a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63488000->63492095] (req 63488000->63492095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.902070:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760777.902074:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c520a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63488000->63492095] (req 63488000->63492095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.902080:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c520a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63488000->63492095] (req 63488000->63492095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.902094:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c520a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63488000->63492095] (req 63488000->63492095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.902098:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c520a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63488000->63492095] (req 63488000->63492095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.902274:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06401500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554749440:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9555us (10793us total) trans 4294982809 rc 0/0 00000100:00100000:1.0:1789760777.902286:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15511 00000100:00100000:1.0:1789760777.902295:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721126622722 time 721138309468 state 0x2 ref 0 00000100:00100000:3.0:1789760777.911475:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.911614:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554749568 00000100:00100000:2.0:1789760777.911624:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554749568 00000100:00100000:2.0:1789760777.911679:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721126622722 time 721138309468 state 0x0 ref 1 00000100:00100000:2.0:1789760777.911695:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721138569884 time 721138309468 state 0x1 ref 1 00000100:00100000:2.0:1789760777.911789:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721138569884 time 721157406587 state 0x0 ref 1 00000100:00100000:2.0:1789760777.911803:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06401880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554749568:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.911848:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63492096, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e480 00010000:00010000:2.0:1789760777.911857:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5211 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.911873:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760777.911878:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.911884:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5211 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63492096->63496191] (req 63492096->63496191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.911898:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5211 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63492096->63496191] (req 63492096->63496191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.916442:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982810 is committed 00002000:00010000:2.0:1789760777.916603:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63492096 -> 63496192 00002000:00010000:2.0:1789760777.916608:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124032 -> 124040 00010000:00010000:2.0:1789760777.916621:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5211 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63492096->63496191] (req 63492096->63496191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.916640:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5211 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63492096->63496191] (req 63492096->63496191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.916648:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5211 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63492096->63496191] (req 63492096->63496191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.916656:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.916661:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5211 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63492096->63496191] (req 63492096->63496191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.916670:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5211 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63492096->63496191] (req 63492096->63496191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.916689:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5211 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63492096->63496191] (req 63492096->63496191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.916767:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5211 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63492096->63496191] (req 63492096->63496191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.917115:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06401880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554749568:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5309us (5643us total) trans 4294982810 rc 0/0 00000100:00100000:2.0:1789760777.917126:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15512 00000100:00100000:2.0:1789760777.917135:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721138569884 time 721157406587 state 0x2 ref 0 00000100:00100000:3.0:1789760777.926421:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760777.926591:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554749696 00000100:00100000:0.0:1789760777.926602:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554749696 00000100:00100000:0.0:1789760777.926660:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721138569884 time 721157406587 state 0x0 ref 1 00000100:00100000:0.0:1789760777.926676:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721157667003 time 721157406587 state 0x1 ref 1 00000100:00100000:0.0:1789760777.926723:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721157667003 time 721172331126 state 0x0 ref 1 00000100:00100000:0.0:1789760777.926736:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe1d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554749696:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760777.926796:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63496192, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e488 00010000:00010000:0.0:1789760777.926805:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5218 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.926821:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760777.926826:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.926832:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5218 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63496192->63500287] (req 63496192->63500287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.926845:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5218 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63496192->63500287] (req 63496192->63500287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.931077:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982811 is committed 00002000:00010000:0.0:1789760777.931204:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63496192 -> 63500288 00002000:00010000:0.0:1789760777.931209:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124040 -> 124048 00010000:00010000:0.0:1789760777.931223:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5218 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63496192->63500287] (req 63496192->63500287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.931236:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5218 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63496192->63500287] (req 63496192->63500287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.931245:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5218 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63496192->63500287] (req 63496192->63500287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.931252:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760777.931256:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5218 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63496192->63500287] (req 63496192->63500287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.931263:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5218 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63496192->63500287] (req 63496192->63500287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.931283:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5218 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63496192->63500287] (req 63496192->63500287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.931289:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5218 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63496192->63500287] (req 63496192->63500287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.931418:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe1d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554749696:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4643us (6004us total) trans 4294982811 rc 0/0 00000100:00100000:0.0:1789760777.931428:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15513 00000100:00100000:0.0:1789760777.931436:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721157667003 time 721172331126 state 0x2 ref 0 00000100:00100000:3.0:1789760777.936734:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760777.936808:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554749824 00000100:00100000:0.0:1789760777.936817:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554749824 00000100:00100000:0.0:1789760777.936870:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721157667003 time 721172331126 state 0x0 ref 1 00000100:00100000:0.0:1789760777.936888:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721172591542 time 721172331126 state 0x1 ref 1 00000100:00100000:0.0:1789760777.936923:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721172591542 time 721182541234 state 0x0 ref 1 00000100:00100000:0.0:1789760777.936936:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06401f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554749824:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760777.936984:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63500288, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e490 00010000:00010000:0.0:1789760777.936994:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c521f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.937009:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760777.937014:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.937020:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c521f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63500288->63504383] (req 63500288->63504383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.937033:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c521f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63500288->63504383] (req 63500288->63504383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.940878:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982812 is committed 00002000:00010000:0.0:1789760777.940993:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63500288 -> 63504384 00002000:00010000:0.0:1789760777.940998:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124048 -> 124056 00010000:00010000:0.0:1789760777.941011:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c521f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63500288->63504383] (req 63500288->63504383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.941030:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c521f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63500288->63504383] (req 63500288->63504383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.941039:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c521f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63500288->63504383] (req 63500288->63504383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.941048:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760777.941053:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c521f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63500288->63504383] (req 63500288->63504383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.941061:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c521f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63500288->63504383] (req 63500288->63504383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.941081:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c521f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63500288->63504383] (req 63500288->63504383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.941089:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c521f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63500288->63504383] (req 63500288->63504383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.941185:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06401f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554749824:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4246us (4454us total) trans 4294982812 rc 0/0 00000100:00100000:0.0:1789760777.941196:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15514 00000100:00100000:0.0:1789760777.941206:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721172591542 time 721182541234 state 0x2 ref 0 00000100:00100000:3.0:1789760777.947973:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760777.949222:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554749952 00000100:00100000:1.0:1789760777.949235:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554749952 00000100:00100000:1.0:1789760777.949290:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721172591542 time 721182541234 state 0x0 ref 1 00000100:00100000:1.0:1789760777.949305:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721182801650 time 721182541234 state 0x1 ref 1 00000100:00100000:1.0:1789760777.949327:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721182801650 time 721194945815 state 0x0 ref 1 00000100:00100000:1.0:1789760777.949340:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06403480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554749952:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760777.949418:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63504384, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e498 00010000:00010000:1.0:1789760777.949427:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5226 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.949444:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760777.949449:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760777.949455:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5226 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63504384->63508479] (req 63504384->63508479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.949469:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5226 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63504384->63508479] (req 63504384->63508479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.955370:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982813 is committed 00002000:00010000:1.0:1789760777.957476:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63504384 -> 63508480 00002000:00010000:1.0:1789760777.957486:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124056 -> 124064 00010000:00010000:1.0:1789760777.957500:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5226 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63504384->63508479] (req 63504384->63508479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.957513:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5226 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63504384->63508479] (req 63504384->63508479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.957522:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5226 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63504384->63508479] (req 63504384->63508479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.957529:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760777.957534:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5226 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63504384->63508479] (req 63504384->63508479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.957542:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5226 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63504384->63508479] (req 63504384->63508479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.957561:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5226 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63504384->63508479] (req 63504384->63508479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.957568:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5226 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63504384->63508479] (req 63504384->63508479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.957668:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06403480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554749952:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8325us (9702us total) trans 4294982813 rc 0/0 00000100:00100000:1.0:1789760777.957680:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15515 00000100:00100000:1.0:1789760777.957689:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721182801650 time 721194945815 state 0x2 ref 0 00000100:00100000:3.0:1789760777.964149:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760777.964762:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554750080 00000100:00100000:2.0:1789760777.964774:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554750080 00000100:00100000:2.0:1789760777.964840:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721182801650 time 721194945815 state 0x0 ref 1 00000100:00100000:2.0:1789760777.964856:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721195206231 time 721194945815 state 0x1 ref 1 00000100:00100000:2.0:1789760777.964893:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721195206231 time 721210511571 state 0x0 ref 1 00000100:00100000:2.0:1789760777.964906:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06402a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554750080:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760777.964954:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63508480, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e4a0 00010000:00010000:2.0:1789760777.964963:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c522d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.964979:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760777.964984:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760777.964990:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c522d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63508480->63512575] (req 63508480->63512575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.965004:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c522d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63508480->63512575] (req 63508480->63512575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760777.972556:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982814 is committed 00002000:00010000:2.0:1789760777.973011:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63508480 -> 63512576 00002000:00010000:2.0:1789760777.973018:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124064 -> 124072 00010000:00010000:2.0:1789760777.973030:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c522d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63508480->63512575] (req 63508480->63512575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.973043:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c522d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63508480->63512575] (req 63508480->63512575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.973058:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c522d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63508480->63512575] (req 63508480->63512575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.973066:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760777.973070:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c522d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63508480->63512575] (req 63508480->63512575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.973079:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c522d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63508480->63512575] (req 63508480->63512575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.973099:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c522d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63508480->63512575] (req 63508480->63512575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760777.973107:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c522d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63508480->63512575] (req 63508480->63512575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760777.973206:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06402a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554750080:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8297us (9062us total) trans 4294982814 rc 0/0 00000100:00100000:2.0:1789760777.973217:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15516 00000100:00100000:2.0:1789760777.973228:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721195206231 time 721210511571 state 0x2 ref 0 00000100:00100000:3.0:1789760777.980079:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760777.980678:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554750208 00000100:00100000:0.0:1789760777.980690:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554750208 00000100:00100000:0.0:1789760777.980759:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721195206231 time 721210511571 state 0x0 ref 1 00000100:00100000:0.0:1789760777.980774:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721210771987 time 721210511571 state 0x1 ref 1 00000100:00100000:0.0:1789760777.980901:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721210771987 time 721226518506 state 0x0 ref 1 00000100:00100000:0.0:1789760777.980925:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06400e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554750208:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760777.980980:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63512576, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e4a8 00010000:00010000:0.0:1789760777.980991:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5234 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.981007:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760777.981011:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.981017:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5234 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63512576->63516671] (req 63512576->63516671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.981030:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5234 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63512576->63516671] (req 63512576->63516671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760777.986262:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982815 is committed 00002000:00010000:0.0:1789760777.986802:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63512576 -> 63516672 00002000:00010000:0.0:1789760777.986809:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124072 -> 124080 00010000:00010000:0.0:1789760777.986821:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5234 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63512576->63516671] (req 63512576->63516671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.986835:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5234 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63512576->63516671] (req 63512576->63516671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.986850:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5234 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63512576->63516671] (req 63512576->63516671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.986857:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760777.986862:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5234 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63512576->63516671] (req 63512576->63516671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.986871:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5234 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63512576->63516671] (req 63512576->63516671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.986891:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5234 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63512576->63516671] (req 63512576->63516671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.986899:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5234 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63512576->63516671] (req 63512576->63516671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760777.986990:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06400e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554750208:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6073us (6914us total) trans 4294982815 rc 0/0 00000100:00100000:0.0:1789760777.987001:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15517 00000100:00100000:0.0:1789760777.987010:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721210771987 time 721226518506 state 0x2 ref 0 00000100:00100000:3.0:1789760777.993118:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760777.993508:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554750336 00000100:00100000:0.0:1789760777.993517:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554750336 00000100:00100000:0.0:1789760777.993566:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721210771987 time 721226518506 state 0x0 ref 1 00000100:00100000:0.0:1789760777.993579:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721226778922 time 721226518506 state 0x1 ref 1 00000100:00100000:0.0:1789760777.993648:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721226778922 time 721239267054 state 0x0 ref 1 00000100:00100000:0.0:1789760777.993659:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06403b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554750336:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760777.993700:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63516672, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e4b0 00010000:00010000:0.0:1789760777.993707:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c523b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.993738:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760777.993746:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760777.993760:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c523b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63516672->63520767] (req 63516672->63520767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760777.993775:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c523b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63516672->63520767] (req 63516672->63520767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760777.998656:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982816 is committed 00002000:00010000:1.0:1789760777.998873:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63516672 -> 63520768 00002000:00010000:1.0:1789760777.998879:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124080 -> 124088 00010000:00010000:1.0:1789760777.998891:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c523b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63516672->63520767] (req 63516672->63520767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.998905:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c523b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63516672->63520767] (req 63516672->63520767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.998915:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c523b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63516672->63520767] (req 63516672->63520767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.998923:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760777.998928:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c523b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63516672->63520767] (req 63516672->63520767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.998937:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c523b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63516672->63520767] (req 63516672->63520767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.998962:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c523b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63516672->63520767] (req 63516672->63520767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760777.998970:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c523b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63516672->63520767] (req 63516672->63520767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760777.999064:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06403b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554750336:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5401us (5950us total) trans 4294982816 rc 0/0 00000100:00100000:1.0:1789760777.999075:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15518 00000100:00100000:1.0:1789760777.999085:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721226778922 time 721239267054 state 0x2 ref 0 00000100:00100000:3.0:1789760778.001840:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760778.001933:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554750464 00000100:00100000:1.0:1789760778.001943:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554750464 00000100:00100000:1.0:1789760778.002002:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721226778922 time 721239267054 state 0x0 ref 1 00000100:00100000:1.0:1789760778.002014:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721239527470 time 721239267054 state 0x1 ref 1 00000100:00100000:1.0:1789760778.002044:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721239527470 time 721247662079 state 0x0 ref 1 00000100:00100000:1.0:1789760778.002056:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06402300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554750464:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760778.002106:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63520768, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e4b8 00010000:00010000:1.0:1789760778.002115:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5242 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.002130:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760778.002134:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.002140:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5242 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63520768->63524863] (req 63520768->63524863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.002154:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5242 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63520768->63524863] (req 63520768->63524863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.010802:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982817 is committed 00002000:00010000:1.0:1789760778.011118:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63520768 -> 63524864 00002000:00010000:1.0:1789760778.011123:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124088 -> 124096 00010000:00010000:1.0:1789760778.011136:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5242 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63520768->63524863] (req 63520768->63524863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.011149:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5242 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63520768->63524863] (req 63520768->63524863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.011159:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5242 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63520768->63524863] (req 63520768->63524863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.011165:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760778.011170:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5242 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63520768->63524863] (req 63520768->63524863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.011178:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5242 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63520768->63524863] (req 63520768->63524863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.011206:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5242 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63520768->63524863] (req 63520768->63524863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.011214:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5242 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63520768->63524863] (req 63520768->63524863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.011356:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06402300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554750464:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9297us (9520us total) trans 4294982817 rc 0/0 00000100:00100000:1.0:1789760778.011367:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15519 00000100:00100000:1.0:1789760778.011424:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721239527470 time 721247662079 state 0x2 ref 0 00000100:00100000:3.0:1789760778.017361:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760778.017530:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554750592 00000100:00100000:2.0:1789760778.017539:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554750592 00000100:00100000:2.0:1789760778.017588:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721239527470 time 721247662079 state 0x0 ref 1 00000100:00100000:2.0:1789760778.017603:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721247922495 time 721247662079 state 0x1 ref 1 00000100:00100000:2.0:1789760778.017672:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721247922495 time 721263289018 state 0x0 ref 1 00000100:00100000:2.0:1789760778.017685:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06403800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554750592:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760778.017741:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63524864, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e4c0 00010000:00010000:2.0:1789760778.017750:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5249 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.017767:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760778.017780:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.017787:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5249 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63524864->63528959] (req 63524864->63528959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.017802:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5249 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63524864->63528959] (req 63524864->63528959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.023810:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982818 is committed 00002000:00010000:2.0:1789760778.023946:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63524864 -> 63528960 00002000:00010000:2.0:1789760778.023952:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124096 -> 124104 00010000:00010000:2.0:1789760778.023965:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5249 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63524864->63528959] (req 63524864->63528959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.023978:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5249 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63524864->63528959] (req 63524864->63528959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.023988:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5249 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63524864->63528959] (req 63524864->63528959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.024000:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760778.024005:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5249 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63524864->63528959] (req 63524864->63528959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.024014:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5249 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63524864->63528959] (req 63524864->63528959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.024034:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5249 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63524864->63528959] (req 63524864->63528959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.024042:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5249 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63524864->63528959] (req 63524864->63528959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.024260:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06403800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554750592:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6572us (6903us total) trans 4294982818 rc 0/0 00000100:00100000:2.0:1789760778.024271:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15520 00000100:00100000:2.0:1789760778.024281:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721247922495 time 721263289018 state 0x2 ref 0 00000100:00100000:3.0:1789760778.031244:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760778.031340:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554750720 00000100:00100000:0.0:1789760778.031350:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554750720 00000100:00100000:0.0:1789760778.031453:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721247922495 time 721263289018 state 0x0 ref 1 00000100:00100000:0.0:1789760778.031468:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721263549434 time 721263289018 state 0x1 ref 1 00000100:00100000:0.0:1789760778.031497:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721263549434 time 721277115351 state 0x0 ref 1 00000100:00100000:0.0:1789760778.031513:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06401180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554750720:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760778.031558:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63528960, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e4c8 00010000:00010000:0.0:1789760778.031567:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5250 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.031593:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760778.031599:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.031607:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5250 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63528960->63533055] (req 63528960->63533055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.031620:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5250 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63528960->63533055] (req 63528960->63533055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.038829:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982819 is committed 00002000:00010000:0.0:1789760778.038977:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63528960 -> 63533056 00002000:00010000:0.0:1789760778.038982:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124104 -> 124112 00010000:00010000:0.0:1789760778.038995:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5250 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63528960->63533055] (req 63528960->63533055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.039008:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5250 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63528960->63533055] (req 63528960->63533055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.039017:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5250 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63528960->63533055] (req 63528960->63533055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.039025:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760778.039030:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5250 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63528960->63533055] (req 63528960->63533055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.039040:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5250 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63528960->63533055] (req 63528960->63533055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.039058:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5250 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63528960->63533055] (req 63528960->63533055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.039066:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5250 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63528960->63533055] (req 63528960->63533055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.039239:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06401180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554750720:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7722us (7997us total) trans 4294982819 rc 0/0 00000100:00100000:0.0:1789760778.039250:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15521 00000100:00100000:0.0:1789760778.039258:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721263549434 time 721277115351 state 0x2 ref 0 00000100:00100000:3.0:1789760778.043881:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760778.043975:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554750848 00000100:00100000:1.0:1789760778.043985:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554750848 00000100:00100000:1.0:1789760778.044035:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721263549434 time 721277115351 state 0x0 ref 1 00000100:00100000:1.0:1789760778.044050:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721277375767 time 721277115351 state 0x1 ref 1 00000100:00100000:1.0:1789760778.044084:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721277375767 time 721289701840 state 0x0 ref 1 00000100:00100000:1.0:1789760778.044106:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06401c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554750848:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760778.044152:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63533056, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e4d0 00010000:00010000:1.0:1789760778.044161:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5257 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.044178:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760778.044182:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.044189:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5257 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63533056->63537151] (req 63533056->63537151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.044369:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5257 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63533056->63537151] (req 63533056->63537151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.051309:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982820 is committed 00002000:00010000:1.0:1789760778.051537:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63533056 -> 63537152 00002000:00010000:1.0:1789760778.051542:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124112 -> 124120 00010000:00010000:1.0:1789760778.051554:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5257 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63533056->63537151] (req 63533056->63537151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.051567:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5257 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63533056->63537151] (req 63533056->63537151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.051575:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5257 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63533056->63537151] (req 63533056->63537151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.051582:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760778.051586:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5257 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63533056->63537151] (req 63533056->63537151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.051594:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5257 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63533056->63537151] (req 63533056->63537151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.051613:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5257 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63533056->63537151] (req 63533056->63537151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.051621:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5257 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63533056->63537151] (req 63533056->63537151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.051907:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06401c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554750848:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7798us (8029us total) trans 4294982820 rc 0/0 00000100:00100000:1.0:1789760778.052017:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15522 00000100:00100000:1.0:1789760778.052027:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721277375767 time 721289701840 state 0x2 ref 0 00000100:00100000:3.0:1789760778.055419:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760778.055478:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554750976 00000100:00100000:1.0:1789760778.055488:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554750976 00000100:00100000:1.0:1789760778.055541:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721277375767 time 721289701840 state 0x0 ref 1 00000100:00100000:1.0:1789760778.055556:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721289962256 time 721289701840 state 0x1 ref 1 00000100:00100000:1.0:1789760778.055582:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721289962256 time 721301200460 state 0x0 ref 1 00000100:00100000:1.0:1789760778.055595:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06402d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554750976:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760778.055640:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63537152, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e4d8 00010000:00010000:1.0:1789760778.055649:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c525e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.055664:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760778.055669:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.055675:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c525e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63537152->63541247] (req 63537152->63541247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.055689:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c525e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63537152->63541247] (req 63537152->63541247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.060665:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982821 is committed 00002000:00010000:1.0:1789760778.061024:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63537152 -> 63541248 00002000:00010000:1.0:1789760778.061029:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124120 -> 124128 00010000:00010000:1.0:1789760778.061043:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c525e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63537152->63541247] (req 63537152->63541247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.061061:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c525e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63537152->63541247] (req 63537152->63541247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.061069:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c525e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63537152->63541247] (req 63537152->63541247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.061077:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760778.061082:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c525e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63537152->63541247] (req 63537152->63541247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.061144:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c525e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63537152->63541247] (req 63537152->63541247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.061164:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c525e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63537152->63541247] (req 63537152->63541247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.061171:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c525e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63537152->63541247] (req 63537152->63541247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.061263:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06402d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554750976:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5666us (5881us total) trans 4294982821 rc 0/0 00000100:00100000:1.0:1789760778.061273:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15523 00000100:00100000:1.0:1789760778.061281:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721289962256 time 721301200460 state 0x2 ref 0 00000100:00100000:3.0:1789760778.065551:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760778.066290:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554751104 00000100:00100000:0.0:1789760778.066301:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554751104 00000100:00100000:0.0:1789760778.066355:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721289962256 time 721301200460 state 0x0 ref 1 00000100:00100000:0.0:1789760778.066371:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721301460876 time 721301200460 state 0x1 ref 1 00000100:00100000:0.0:1789760778.066493:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721301460876 time 721312110973 state 0x0 ref 1 00000100:00100000:0.0:1789760778.066506:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06400000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554751104:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760778.066550:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63541248, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e4e0 00010000:00010000:0.0:1789760778.066559:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5265 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.066574:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760778.066578:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.066584:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5265 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63541248->63545343] (req 63541248->63545343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.066597:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5265 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63541248->63545343] (req 63541248->63545343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.071838:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982822 is committed 00002000:00010000:2.0:1789760778.072014:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63541248 -> 63545344 00002000:00010000:2.0:1789760778.072020:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124128 -> 124136 00010000:00010000:2.0:1789760778.072033:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5265 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63541248->63545343] (req 63541248->63545343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.072046:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5265 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63541248->63545343] (req 63541248->63545343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.072056:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5265 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63541248->63545343] (req 63541248->63545343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.072063:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760778.072068:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5265 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63541248->63545343] (req 63541248->63545343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.072077:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5265 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63541248->63545343] (req 63541248->63545343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.072152:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5265 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63541248->63545343] (req 63541248->63545343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.072162:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5265 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63541248->63545343] (req 63541248->63545343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.072299:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06400000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554751104:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5790us (6753us total) trans 4294982822 rc 0/0 00000100:00100000:2.0:1789760778.072310:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15524 00000100:00100000:2.0:1789760778.072319:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721301460876 time 721312110973 state 0x2 ref 0 00000100:00100000:3.0:1789760778.076955:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760778.078095:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554751232 00000100:00100000:0.0:1789760778.078107:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554751232 00000100:00100000:0.0:1789760778.078166:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721301460876 time 721312110973 state 0x0 ref 1 00000100:00100000:0.0:1789760778.078182:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721312371389 time 721312110973 state 0x1 ref 1 00000100:00100000:0.0:1789760778.078216:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721312371389 time 721323835067 state 0x0 ref 1 00000100:00100000:0.0:1789760778.078228:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06403100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554751232:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760778.078272:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63545344, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e4e8 00010000:00010000:0.0:1789760778.078281:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c526c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.078303:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760778.078307:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.078313:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c526c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63545344->63549439] (req 63545344->63549439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.078327:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c526c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63545344->63549439] (req 63545344->63549439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.086593:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982823 is committed 00002000:00010000:3.0:1789760778.089236:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63545344 -> 63549440 00002000:00010000:3.0:1789760778.089246:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124136 -> 124144 00010000:00010000:3.0:1789760778.089258:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c526c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63545344->63549439] (req 63545344->63549439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.089277:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c526c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63545344->63549439] (req 63545344->63549439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.089286:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c526c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63545344->63549439] (req 63545344->63549439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.089293:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760778.089297:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c526c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63545344->63549439] (req 63545344->63549439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.089307:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c526c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63545344->63549439] (req 63545344->63549439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.089326:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c526c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63545344->63549439] (req 63545344->63549439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.089334:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c526c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63545344->63549439] (req 63545344->63549439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.089473:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06403100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554751232:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11243us (12523us total) trans 4294982823 rc 0/0 00000100:00100000:3.0:1789760778.089485:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15525 00000100:00100000:3.0:1789760778.089494:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721312371389 time 721323835067 state 0x2 ref 0 00000100:00100000:3.0:1789760778.094263:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760778.094521:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554751360 00000100:00100000:1.0:1789760778.094531:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554751360 00000100:00100000:1.0:1789760778.094579:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721312371389 time 721323835067 state 0x0 ref 1 00000100:00100000:1.0:1789760778.094593:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721324095483 time 721323835067 state 0x1 ref 1 00000100:00100000:1.0:1789760778.094622:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721324095483 time 721340241153 state 0x0 ref 1 00000100:00100000:1.0:1789760778.094634:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06400a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554751360:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760778.094679:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63549440, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e4f0 00010000:00010000:1.0:1789760778.094688:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5273 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.094702:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760778.094707:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.094722:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5273 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63549440->63553535] (req 63549440->63553535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.094735:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5273 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63549440->63553535] (req 63549440->63553535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.103446:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982824 is committed 00002000:00010000:1.0:1789760778.103627:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63549440 -> 63553536 00002000:00010000:1.0:1789760778.103633:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124144 -> 124152 00010000:00010000:1.0:1789760778.103646:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5273 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63549440->63553535] (req 63549440->63553535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.103658:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5273 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63549440->63553535] (req 63549440->63553535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.103673:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5273 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63549440->63553535] (req 63549440->63553535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.103681:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760778.103687:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5273 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63549440->63553535] (req 63549440->63553535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.103696:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5273 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63549440->63553535] (req 63549440->63553535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.103729:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5273 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63549440->63553535] (req 63549440->63553535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.103738:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5273 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63549440->63553535] (req 63549440->63553535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.103832:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06400a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554751360:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9195us (9574us total) trans 4294982824 rc 0/0 00000100:00100000:1.0:1789760778.103843:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15526 00000100:00100000:1.0:1789760778.103852:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721324095483 time 721340241153 state 0x2 ref 0 00000100:00100000:0.0:1789760778.108524:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760778.108862:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554751488 00000100:00100000:1.0:1789760778.108872:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554751488 00000100:00100000:1.0:1789760778.108924:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721324095483 time 721340241153 state 0x0 ref 1 00000100:00100000:1.0:1789760778.108939:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721340501569 time 721340241153 state 0x1 ref 1 00000100:00100000:1.0:1789760778.108974:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721340501569 time 721354593374 state 0x0 ref 1 00000100:00100000:1.0:1789760778.108985:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554751488:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760778.109027:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63553536, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e4f8 00010000:00010000:1.0:1789760778.109036:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c527a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.109052:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760778.109056:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.109061:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c527a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63553536->63557631] (req 63553536->63557631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.109073:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c527a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63553536->63557631] (req 63553536->63557631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.116496:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982825 is committed 00002000:00010000:1.0:1789760778.116617:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63553536 -> 63557632 00002000:00010000:1.0:1789760778.116622:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124152 -> 124160 00010000:00010000:1.0:1789760778.116633:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c527a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63553536->63557631] (req 63553536->63557631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.116645:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c527a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63553536->63557631] (req 63553536->63557631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.116653:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c527a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63553536->63557631] (req 63553536->63557631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.116665:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760778.116670:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c527a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63553536->63557631] (req 63553536->63557631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.116678:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c527a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63553536->63557631] (req 63553536->63557631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.116696:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c527a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63553536->63557631] (req 63553536->63557631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.116703:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c527a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63553536->63557631] (req 63553536->63557631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.116990:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554751488:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8000us (8468us total) trans 4294982825 rc 0/0 00000100:00100000:1.0:1789760778.117001:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15527 00000100:00100000:1.0:1789760778.117011:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721340501569 time 721354593374 state 0x2 ref 0 00000100:00100000:3.0:1789760778.124274:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760778.124427:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554751616 00000100:00100000:2.0:1789760778.124437:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554751616 00000100:00100000:2.0:1789760778.124489:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721340501569 time 721354593374 state 0x0 ref 1 00000100:00100000:2.0:1789760778.124504:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721354853790 time 721354593374 state 0x1 ref 1 00000100:00100000:2.0:1789760778.124538:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721354853790 time 721370156978 state 0x0 ref 1 00000100:00100000:2.0:1789760778.124550:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06402680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554751616:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760778.124607:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63557632, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e500 00010000:00010000:2.0:1789760778.124617:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5281 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.124633:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760778.124638:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.124645:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5281 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63557632->63561727] (req 63557632->63561727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.124660:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5281 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63557632->63561727] (req 63557632->63561727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.131308:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982826 is committed 00002000:00010000:2.0:1789760778.131433:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63557632 -> 63561728 00002000:00010000:2.0:1789760778.131438:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124160 -> 124168 00010000:00010000:2.0:1789760778.131451:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5281 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63557632->63561727] (req 63557632->63561727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.131463:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5281 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63557632->63561727] (req 63557632->63561727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.131473:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5281 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63557632->63561727] (req 63557632->63561727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.131486:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760778.131491:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5281 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63557632->63561727] (req 63557632->63561727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.131500:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5281 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63557632->63561727] (req 63557632->63561727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.131517:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5281 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63557632->63561727] (req 63557632->63561727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.131524:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5281 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63557632->63561727] (req 63557632->63561727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.131599:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06402680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554751616:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7046us (7330us total) trans 4294982826 rc 0/0 00000100:00100000:2.0:1789760778.131607:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15528 00000100:00100000:2.0:1789760778.131617:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721354853790 time 721370156978 state 0x2 ref 0 00000100:00100000:3.0:1789760778.135977:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760778.136271:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554751744 00000100:00100000:0.0:1789760778.136355:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554751744 00000100:00100000:0.0:1789760778.136529:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721354853790 time 721370156978 state 0x0 ref 1 00000100:00100000:0.0:1789760778.136545:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721370417394 time 721370156978 state 0x1 ref 1 00000100:00100000:0.0:1789760778.136570:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721370417394 time 721382188871 state 0x0 ref 1 00000100:00100000:0.0:1789760778.136585:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06401500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554751744:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760778.136633:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63561728, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e508 00010000:00010000:0.0:1789760778.136642:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5288 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.136657:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760778.136661:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.136668:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5288 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63561728->63565823] (req 63561728->63565823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.136681:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5288 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63561728->63565823] (req 63561728->63565823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.143701:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982827 is committed 00002000:00010000:0.0:1789760778.144043:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63561728 -> 63565824 00002000:00010000:0.0:1789760778.144048:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124168 -> 124176 00010000:00010000:0.0:1789760778.144061:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5288 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63561728->63565823] (req 63561728->63565823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.144074:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5288 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63561728->63565823] (req 63561728->63565823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.144084:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5288 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63561728->63565823] (req 63561728->63565823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.144149:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760778.144155:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5288 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63561728->63565823] (req 63561728->63565823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.144165:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5288 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63561728->63565823] (req 63561728->63565823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.144186:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5288 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63561728->63565823] (req 63561728->63565823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.144198:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5288 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63561728->63565823] (req 63561728->63565823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.144473:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06401500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554751744:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7880us (8495us total) trans 4294982827 rc 0/0 00000100:00100000:0.0:1789760778.144484:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15529 00000100:00100000:0.0:1789760778.144494:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721370417394 time 721382188871 state 0x2 ref 0 00000100:00100000:0.0:1789760778.149319:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760778.149405:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554751872 00000100:00100000:1.0:1789760778.149414:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554751872 00000100:00100000:1.0:1789760778.149466:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721370417394 time 721382188871 state 0x0 ref 1 00000100:00100000:1.0:1789760778.149481:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721382449287 time 721382188871 state 0x1 ref 1 00000100:00100000:1.0:1789760778.149553:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721382449287 time 721395170683 state 0x0 ref 1 00000100:00100000:1.0:1789760778.149565:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554751872:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760778.149608:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63565824, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e510 00010000:00010000:1.0:1789760778.149617:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c528f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.149633:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760778.149636:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.149640:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c528f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63565824->63569919] (req 63565824->63569919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.149651:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c528f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63565824->63569919] (req 63565824->63569919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.155633:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982828 is committed 00002000:00010000:1.0:1789760778.156069:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63565824 -> 63569920 00002000:00010000:1.0:1789760778.156074:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124176 -> 124184 00010000:00010000:1.0:1789760778.156088:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c528f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63565824->63569919] (req 63565824->63569919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.156101:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c528f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63565824->63569919] (req 63565824->63569919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.156110:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c528f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63565824->63569919] (req 63565824->63569919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.156117:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760778.156122:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c528f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63565824->63569919] (req 63565824->63569919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.156132:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c528f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63565824->63569919] (req 63565824->63569919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.156151:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c528f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63565824->63569919] (req 63565824->63569919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.156159:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c528f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63565824->63569919] (req 63565824->63569919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.156264:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554751872:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6697us (6950us total) trans 4294982828 rc 0/0 00000100:00100000:1.0:1789760778.156276:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15530 00000100:00100000:1.0:1789760778.157210:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721382449287 time 721395170683 state 0x2 ref 0 00000100:00100000:3.0:1789760778.166218:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760778.166619:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554752000 00000100:00100000:1.0:1789760778.166628:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554752000 00000100:00100000:0.0:1789760778.166953:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721382449287 time 721395170683 state 0x0 ref 1 00000100:00100000:0.0:1789760778.166967:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721395431099 time 721395170683 state 0x1 ref 1 00000100:00100000:0.0:1789760778.167035:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721395431099 time 721412653100 state 0x0 ref 1 00000100:00100000:0.0:1789760778.167048:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06401880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554752000:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760778.167107:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63569920, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e518 00010000:00010000:0.0:1789760778.167115:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5296 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.167138:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760778.167143:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.167148:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5296 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63569920->63574015] (req 63569920->63574015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.167165:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5296 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63569920->63574015] (req 63569920->63574015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.171784:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982829 is committed 00002000:00010000:0.0:1789760778.172709:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63569920 -> 63574016 00002000:00010000:0.0:1789760778.172725:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124184 -> 124192 00010000:00010000:0.0:1789760778.172738:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5296 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63569920->63574015] (req 63569920->63574015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.172750:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5296 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63569920->63574015] (req 63569920->63574015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.172760:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5296 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63569920->63574015] (req 63569920->63574015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.172766:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760778.172771:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5296 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63569920->63574015] (req 63569920->63574015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.172780:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5296 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63569920->63574015] (req 63569920->63574015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.172800:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5296 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63569920->63574015] (req 63569920->63574015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.172813:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5296 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63569920->63574015] (req 63569920->63574015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.172907:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06401880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554752000:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5856us (6691us total) trans 4294982829 rc 0/0 00000100:00100000:0.0:1789760778.172918:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15531 00000100:00100000:0.0:1789760778.172926:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721395431099 time 721412653100 state 0x2 ref 0 00000100:00100000:3.0:1789760778.181580:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760778.181735:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554752128 00000100:00100000:2.0:1789760778.181745:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554752128 00000100:00100000:2.0:1789760778.181797:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721395431099 time 721412653100 state 0x0 ref 1 00000100:00100000:2.0:1789760778.181812:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721412913516 time 721412653100 state 0x1 ref 1 00000100:00100000:2.0:1789760778.181843:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721412913516 time 721427462071 state 0x0 ref 1 00000100:00100000:2.0:1789760778.181856:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e57800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554752128:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760778.181902:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63574016, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e520 00010000:00010000:2.0:1789760778.181912:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c529d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.181926:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760778.181930:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.181937:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c529d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63574016->63578111] (req 63574016->63578111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.181951:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c529d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63574016->63578111] (req 63574016->63578111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.186648:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982830 is committed 00002000:00010000:0.0:1789760778.186785:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63574016 -> 63578112 00002000:00010000:0.0:1789760778.186790:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124192 -> 124200 00010000:00010000:0.0:1789760778.186804:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c529d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63574016->63578111] (req 63574016->63578111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.186827:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c529d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63574016->63578111] (req 63574016->63578111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.186835:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c529d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63574016->63578111] (req 63574016->63578111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.186843:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760778.186848:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c529d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63574016->63578111] (req 63574016->63578111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.186857:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c529d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63574016->63578111] (req 63574016->63578111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.186877:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c529d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63574016->63578111] (req 63574016->63578111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.186891:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c529d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63574016->63578111] (req 63574016->63578111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.186984:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e57800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554752128:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5125us (5410us total) trans 4294982830 rc 0/0 00000100:00100000:0.0:1789760778.186995:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15532 00000100:00100000:0.0:1789760778.187005:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721412913516 time 721427462071 state 0x2 ref 0 00000100:00100000:3.0:1789760778.197067:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760778.197284:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554752256 00000100:00100000:0.0:1789760778.197293:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554752256 00000100:00100000:0.0:1789760778.197348:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721412913516 time 721427462071 state 0x0 ref 1 00000100:00100000:0.0:1789760778.197364:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721427722487 time 721427462071 state 0x1 ref 1 00000100:00100000:0.0:1789760778.197427:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721427722487 time 721443045430 state 0x0 ref 1 00000100:00100000:0.0:1789760778.197440:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554752256:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760778.197485:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63578112, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e528 00010000:00010000:0.0:1789760778.197496:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c52a4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.197513:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760778.197517:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.197523:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c52a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63578112->63582207] (req 63578112->63582207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.197543:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c52a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63578112->63582207] (req 63578112->63582207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.202196:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982831 is committed 00002000:00010000:3.0:1789760778.202320:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63578112 -> 63582208 00002000:00010000:3.0:1789760778.202325:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124200 -> 124208 00010000:00010000:3.0:1789760778.202339:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c52a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63578112->63582207] (req 63578112->63582207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.202351:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c52a4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63578112->63582207] (req 63578112->63582207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.202359:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c52a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63578112->63582207] (req 63578112->63582207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.202367:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760778.202372:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c52a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63578112->63582207] (req 63578112->63582207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.202404:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c52a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63578112->63582207] (req 63578112->63582207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.202423:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c52a4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63578112->63582207] (req 63578112->63582207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.202430:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c52a4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63578112->63582207] (req 63578112->63582207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.202520:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554752256:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5076us (5457us total) trans 4294982831 rc 0/0 00000100:00100000:3.0:1789760778.202531:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15533 00000100:00100000:3.0:1789760778.202540:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721427722487 time 721443045430 state 0x2 ref 0 00000100:00100000:3.0:1789760778.207617:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760778.207853:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554752384 00000100:00100000:1.0:1789760778.207863:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554752384 00000100:00100000:1.0:1789760778.207913:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721427722487 time 721443045430 state 0x0 ref 1 00000100:00100000:1.0:1789760778.207928:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721443305846 time 721443045430 state 0x1 ref 1 00000100:00100000:1.0:1789760778.207960:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721443305846 time 721453577918 state 0x0 ref 1 00000100:00100000:1.0:1789760778.207976:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e54a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554752384:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760778.208026:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63582208, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e530 00010000:00010000:1.0:1789760778.208034:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c52ab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.208052:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760778.208056:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.208063:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c52ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63582208->63586303] (req 63582208->63586303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.208076:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c52ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63582208->63586303] (req 63582208->63586303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.213639:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982832 is committed 00002000:00010000:0.0:1789760778.217163:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63582208 -> 63586304 00002000:00010000:0.0:1789760778.217173:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124208 -> 124216 00010000:00010000:0.0:1789760778.217187:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c52ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63582208->63586303] (req 63582208->63586303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.217200:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c52ab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63582208->63586303] (req 63582208->63586303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.217208:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c52ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63582208->63586303] (req 63582208->63586303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.217216:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760778.217220:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c52ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63582208->63586303] (req 63582208->63586303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.217228:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c52ab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63582208->63586303] (req 63582208->63586303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.217637:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c52ab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63582208->63586303] (req 63582208->63586303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.217648:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c52ab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63582208->63586303] (req 63582208->63586303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.217808:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554752384:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9828us (10193us total) trans 4294982832 rc 0/0 00000100:00100000:0.0:1789760778.217820:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15534 00000100:00100000:0.0:1789760778.217829:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721443305846 time 721453577918 state 0x2 ref 0 00000100:00100000:3.0:1789760778.221028:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760778.221288:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554752512 00000100:00100000:2.0:1789760778.221297:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554752512 00000100:00100000:2.0:1789760778.221352:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721443305846 time 721453577918 state 0x0 ref 1 00000100:00100000:2.0:1789760778.221367:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721453838334 time 721453577918 state 0x1 ref 1 00000100:00100000:2.0:1789760778.221420:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721453838334 time 721467038226 state 0x0 ref 1 00000100:00100000:2.0:1789760778.221443:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554752512:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760778.221498:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63586304, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e538 00010000:00010000:2.0:1789760778.221507:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c52b2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.221522:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760778.221527:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.221533:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c52b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63586304->63590399] (req 63586304->63590399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.221546:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c52b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63586304->63590399] (req 63586304->63590399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.228924:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982833 is committed 00002000:00010000:2.0:1789760778.229043:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63586304 -> 63590400 00002000:00010000:2.0:1789760778.229048:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124216 -> 124224 00010000:00010000:2.0:1789760778.229062:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c52b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63586304->63590399] (req 63586304->63590399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.229075:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c52b2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63586304->63590399] (req 63586304->63590399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.229096:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c52b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63586304->63590399] (req 63586304->63590399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.229105:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760778.229109:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c52b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63586304->63590399] (req 63586304->63590399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.229124:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c52b2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63586304->63590399] (req 63586304->63590399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.229144:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c52b2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63586304->63590399] (req 63586304->63590399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.229152:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c52b2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63586304->63590399] (req 63586304->63590399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.229416:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554752512:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7935us (8357us total) trans 4294982833 rc 0/0 00000100:00100000:2.0:1789760778.229428:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15535 00000100:00100000:2.0:1789760778.229437:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721453838334 time 721467038226 state 0x2 ref 0 00000100:00100000:3.0:1789760778.234455:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760778.234486:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554752640 00000100:00100000:0.0:1789760778.234496:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554752640 00000100:00100000:0.0:1789760778.234547:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721453838334 time 721467038226 state 0x0 ref 1 00000100:00100000:0.0:1789760778.234563:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721467298642 time 721467038226 state 0x1 ref 1 00000100:00100000:0.0:1789760778.234597:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721467298642 time 721480215031 state 0x0 ref 1 00000100:00100000:0.0:1789760778.234610:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e56680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554752640:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760778.234655:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63590400, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e540 00010000:00010000:0.0:1789760778.234663:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52b9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.234678:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760778.234682:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.234687:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63590400->63594495] (req 63590400->63594495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.234701:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63590400->63594495] (req 63590400->63594495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.242656:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982834 is committed 00002000:00010000:0.0:1789760778.242954:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63590400 -> 63594496 00002000:00010000:0.0:1789760778.242960:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124224 -> 124232 00010000:00010000:0.0:1789760778.242974:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63590400->63594495] (req 63590400->63594495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.242988:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63590400->63594495] (req 63590400->63594495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.242997:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63590400->63594495] (req 63590400->63594495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.243004:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760778.243009:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63590400->63594495] (req 63590400->63594495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.243017:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52b9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63590400->63594495] (req 63590400->63594495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.243034:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52b9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63590400->63594495] (req 63590400->63594495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.243041:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52b9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63590400->63594495] (req 63590400->63594495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.243490:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e56680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554752640:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8877us (9038us total) trans 4294982834 rc 0/0 00000100:00100000:0.0:1789760778.243501:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15536 00000100:00100000:0.0:1789760778.243510:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721467298642 time 721480215031 state 0x2 ref 0 00000100:00100000:3.0:1789760778.249225:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760778.249677:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554752768 00000100:00100000:1.0:1789760778.249687:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554752768 00000100:00100000:1.0:1789760778.249756:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721467298642 time 721480215031 state 0x0 ref 1 00000100:00100000:1.0:1789760778.249772:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721480475447 time 721480215031 state 0x1 ref 1 00000100:00100000:1.0:1789760778.249809:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721480475447 time 721495427550 state 0x0 ref 1 00000100:00100000:1.0:1789760778.249822:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e54380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554752768:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760778.249875:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63594496, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e548 00010000:00010000:1.0:1789760778.249885:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c52c0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.249902:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760778.249907:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.249913:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c52c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63594496->63598591] (req 63594496->63598591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.249928:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c52c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63594496->63598591] (req 63594496->63598591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.255150:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982835 is committed 00002000:00010000:1.0:1789760778.255267:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63594496 -> 63598592 00002000:00010000:1.0:1789760778.255272:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124232 -> 124240 00010000:00010000:1.0:1789760778.255284:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c52c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63594496->63598591] (req 63594496->63598591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.255297:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c52c0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63594496->63598591] (req 63594496->63598591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.255305:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c52c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63594496->63598591] (req 63594496->63598591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.255312:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760778.255317:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c52c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63594496->63598591] (req 63594496->63598591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.255325:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c52c0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63594496->63598591] (req 63594496->63598591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.255348:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c52c0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63594496->63598591] (req 63594496->63598591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.255355:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c52c0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63594496->63598591] (req 63594496->63598591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.255674:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554752768:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5848us (6452us total) trans 4294982835 rc 0/0 00000100:00100000:1.0:1789760778.255686:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15537 00000100:00100000:1.0:1789760778.255696:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721480475447 time 721495427550 state 0x2 ref 0 00000100:00100000:1.0:1789760778.261824:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760778.261930:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554752896 00000100:00100000:0.0:1789760778.261940:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554752896 00000100:00100000:0.0:1789760778.261995:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721480475447 time 721495427550 state 0x0 ref 1 00000100:00100000:0.0:1789760778.262011:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721495687966 time 721495427550 state 0x1 ref 1 00000100:00100000:0.0:1789760778.262045:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721495687966 time 721507663330 state 0x0 ref 1 00000100:00100000:0.0:1789760778.262057:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554752896:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760778.262116:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63598592, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e550 00010000:00010000:0.0:1789760778.262126:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52c7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.262141:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760778.262146:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.262152:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63598592->63602687] (req 63598592->63602687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.262165:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63598592->63602687] (req 63598592->63602687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.269651:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982836 is committed 00002000:00010000:0.0:1789760778.269955:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63598592 -> 63602688 00002000:00010000:0.0:1789760778.269961:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124240 -> 124248 00010000:00010000:0.0:1789760778.269974:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63598592->63602687] (req 63598592->63602687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.269987:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63598592->63602687] (req 63598592->63602687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.269996:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63598592->63602687] (req 63598592->63602687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.270003:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760778.270009:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63598592->63602687] (req 63598592->63602687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.270017:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63598592->63602687] (req 63598592->63602687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.270035:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52c7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63598592->63602687] (req 63598592->63602687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.270042:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52c7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63598592->63602687] (req 63598592->63602687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.270146:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554752896:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8085us (8326us total) trans 4294982836 rc 0/0 00000100:00100000:0.0:1789760778.270156:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15538 00000100:00100000:0.0:1789760778.270165:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721495687966 time 721507663330 state 0x2 ref 0 00000100:00100000:3.0:1789760778.278190:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760778.278255:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554753024 00000100:00100000:2.0:1789760778.278265:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554753024 00000100:00100000:2.0:1789760778.278318:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721495687966 time 721507663330 state 0x0 ref 1 00000100:00100000:2.0:1789760778.278335:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721507923746 time 721507663330 state 0x1 ref 1 00000100:00100000:2.0:1789760778.278368:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721507923746 time 721523986122 state 0x0 ref 1 00000100:00100000:2.0:1789760778.278425:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e57100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554753024:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760778.278558:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63602688, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e558 00010000:00010000:2.0:1789760778.278568:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c52ce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.278584:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760778.278589:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.278595:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c52ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63602688->63606783] (req 63602688->63606783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.278609:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c52ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63602688->63606783] (req 63602688->63606783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.284581:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982837 is committed 00002000:00010000:0.0:1789760778.284717:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63602688 -> 63606784 00002000:00010000:0.0:1789760778.284723:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124248 -> 124256 00010000:00010000:0.0:1789760778.284735:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c52ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63602688->63606783] (req 63602688->63606783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.284748:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c52ce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63602688->63606783] (req 63602688->63606783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.284757:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c52ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63602688->63606783] (req 63602688->63606783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.284765:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760778.284777:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c52ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63602688->63606783] (req 63602688->63606783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.284786:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c52ce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63602688->63606783] (req 63602688->63606783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.284804:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c52ce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63602688->63606783] (req 63602688->63606783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.284812:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c52ce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63602688->63606783] (req 63602688->63606783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.284904:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e57100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554753024:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6520us (6719us total) trans 4294982837 rc 0/0 00000100:00100000:0.0:1789760778.284919:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15539 00000100:00100000:0.0:1789760778.284929:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721507923746 time 721523986122 state 0x2 ref 0 00000100:00100000:3.0:1789760778.288932:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760778.289079:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554753152 00000100:00100000:2.0:1789760778.289097:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554753152 00000100:00100000:2.0:1789760778.289152:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721507923746 time 721523986122 state 0x0 ref 1 00000100:00100000:2.0:1789760778.289168:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721524246538 time 721523986122 state 0x1 ref 1 00000100:00100000:2.0:1789760778.289198:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721524246538 time 721534816024 state 0x0 ref 1 00000100:00100000:2.0:1789760778.289210:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554753152:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760778.289263:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63606784, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e560 00010000:00010000:2.0:1789760778.289273:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c52d5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.289294:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760778.289298:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.289305:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c52d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63606784->63610879] (req 63606784->63610879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.289318:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c52d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63606784->63610879] (req 63606784->63610879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.294785:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982838 is committed 00002000:00010000:0.0:1789760778.295008:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63606784 -> 63610880 00002000:00010000:0.0:1789760778.295014:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124256 -> 124264 00010000:00010000:0.0:1789760778.295026:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c52d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63606784->63610879] (req 63606784->63610879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.295159:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c52d5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63606784->63610879] (req 63606784->63610879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.295169:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c52d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63606784->63610879] (req 63606784->63610879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.295178:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760778.295184:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c52d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63606784->63610879] (req 63606784->63610879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.295193:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c52d5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63606784->63610879] (req 63606784->63610879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.295214:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c52d5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63606784->63610879] (req 63606784->63610879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.295222:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c52d5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63606784->63610879] (req 63606784->63610879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.295518:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554753152:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6303us (6588us total) trans 4294982838 rc 0/0 00000100:00100000:0.0:1789760778.295529:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15540 00000100:00100000:0.0:1789760778.295538:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721524246538 time 721534816024 state 0x2 ref 0 00000100:00100000:1.0:1789760778.302546:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760778.302787:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554753280 00000100:00100000:3.0:1789760778.302796:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554753280 00000100:00100000:3.0:1789760778.302863:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721524246538 time 721534816024 state 0x0 ref 1 00000100:00100000:3.0:1789760778.302880:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721535076440 time 721534816024 state 0x1 ref 1 00000100:00100000:3.0:1789760778.302912:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721535076440 time 721548530313 state 0x0 ref 1 00000100:00100000:3.0:1789760778.302924:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554753280:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760778.302972:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63610880, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e568 00010000:00010000:3.0:1789760778.302982:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c52dc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.302998:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760778.303003:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.303010:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c52dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63610880->63614975] (req 63610880->63614975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.303024:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c52dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63610880->63614975] (req 63610880->63614975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.308883:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982839 is committed 00002000:00010000:3.0:1789760778.309233:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63610880 -> 63614976 00002000:00010000:3.0:1789760778.309239:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124264 -> 124272 00010000:00010000:3.0:1789760778.309253:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c52dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63610880->63614975] (req 63610880->63614975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.309266:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c52dc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63610880->63614975] (req 63610880->63614975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.309275:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c52dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63610880->63614975] (req 63610880->63614975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.309283:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760778.309288:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c52dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63610880->63614975] (req 63610880->63614975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.309297:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c52dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63610880->63614975] (req 63610880->63614975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.309318:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c52dc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63610880->63614975] (req 63610880->63614975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.309332:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c52dc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63610880->63614975] (req 63610880->63614975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.309986:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554753280:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7054us (7439us total) trans 4294982839 rc 0/0 00000100:00100000:3.0:1789760778.309998:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15541 00000100:00100000:3.0:1789760778.310008:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721535076440 time 721548530313 state 0x2 ref 0 00000100:00100000:3.0:1789760778.313931:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760778.314023:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554753408 00000100:00100000:0.0:1789760778.314031:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554753408 00000100:00100000:0.0:1789760778.314081:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721535076440 time 721548530313 state 0x0 ref 1 00000100:00100000:0.0:1789760778.314113:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721548790729 time 721548530313 state 0x1 ref 1 00000100:00100000:0.0:1789760778.314148:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721548790729 time 721559763154 state 0x0 ref 1 00000100:00100000:0.0:1789760778.314161:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e54e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554753408:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760778.314209:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63614976, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e570 00010000:00010000:0.0:1789760778.314218:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52e3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.314233:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760778.314238:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.314244:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63614976->63619071] (req 63614976->63619071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.314259:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63614976->63619071] (req 63614976->63619071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.320934:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982840 is committed 00002000:00010000:0.0:1789760778.324282:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63614976 -> 63619072 00002000:00010000:0.0:1789760778.324291:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124272 -> 124280 00010000:00010000:0.0:1789760778.324304:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63614976->63619071] (req 63614976->63619071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.324317:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52e3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63614976->63619071] (req 63614976->63619071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.324325:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63614976->63619071] (req 63614976->63619071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.324331:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760778.324336:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63614976->63619071] (req 63614976->63619071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.324343:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52e3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63614976->63619071] (req 63614976->63619071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.324361:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52e3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63614976->63619071] (req 63614976->63619071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.324367:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52e3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63614976->63619071] (req 63614976->63619071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.324630:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554753408:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10466us (10703us total) trans 4294982840 rc 0/0 00000100:00100000:0.0:1789760778.324642:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15542 00000100:00100000:0.0:1789760778.324651:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721548790729 time 721559763154 state 0x2 ref 0 00000100:00100000:1.0:1789760778.329976:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760778.330070:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554753536 00000100:00100000:0.0:1789760778.330082:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554753536 00000100:00100000:0.0:1789760778.330211:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721548790729 time 721559763154 state 0x0 ref 1 00000100:00100000:0.0:1789760778.330227:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721560023570 time 721559763154 state 0x1 ref 1 00000100:00100000:0.0:1789760778.330262:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721560023570 time 721575880246 state 0x0 ref 1 00000100:00100000:0.0:1789760778.330274:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554753536:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760778.330320:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63619072, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e578 00010000:00010000:0.0:1789760778.330331:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c52ea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.330349:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760778.330354:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.330360:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c52ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63619072->63623167] (req 63619072->63623167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.330396:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c52ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63619072->63623167] (req 63619072->63623167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.336986:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982841 is committed 00002000:00010000:1.0:1789760778.338033:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63619072 -> 63623168 00002000:00010000:1.0:1789760778.338041:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124280 -> 124288 00010000:00010000:1.0:1789760778.338054:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c52ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63619072->63623167] (req 63619072->63623167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.338068:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c52ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63619072->63623167] (req 63619072->63623167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.338077:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c52ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63619072->63623167] (req 63619072->63623167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.338230:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760778.338235:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c52ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63619072->63623167] (req 63619072->63623167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.338245:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c52ea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63619072->63623167] (req 63619072->63623167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.338266:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c52ea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63619072->63623167] (req 63619072->63623167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.338275:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c52ea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63619072->63623167] (req 63619072->63623167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.338576:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554753536:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8299us (8603us total) trans 4294982841 rc 0/0 00000100:00100000:1.0:1789760778.338587:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15543 00000100:00100000:1.0:1789760778.338597:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721560023570 time 721575880246 state 0x2 ref 0 00000100:00100000:1.0:1789760778.342307:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760778.342595:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554753664 00000100:00100000:2.0:1789760778.342604:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554753664 00000100:00100000:2.0:1789760778.342652:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721560023570 time 721575880246 state 0x0 ref 1 00000100:00100000:2.0:1789760778.342668:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721576140662 time 721575880246 state 0x1 ref 1 00000100:00100000:2.0:1789760778.342701:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721576140662 time 721588318809 state 0x0 ref 1 00000100:00100000:2.0:1789760778.342722:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554753664:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760778.342767:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63623168, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e580 00010000:00010000:2.0:1789760778.342777:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c52f1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.342792:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760778.342797:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.342804:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c52f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63623168->63627263] (req 63623168->63627263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.342817:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c52f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63623168->63627263] (req 63623168->63627263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.347429:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982842 is committed 00002000:00010000:2.0:1789760778.347594:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63623168 -> 63627264 00002000:00010000:2.0:1789760778.347600:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124288 -> 124296 00010000:00010000:2.0:1789760778.347614:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c52f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63623168->63627263] (req 63623168->63627263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.347627:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c52f1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63623168->63627263] (req 63623168->63627263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.347636:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c52f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63623168->63627263] (req 63623168->63627263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.347643:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760778.347648:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c52f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63623168->63627263] (req 63623168->63627263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.347657:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c52f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63623168->63627263] (req 63623168->63627263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.347678:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c52f1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63623168->63627263] (req 63623168->63627263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.347691:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c52f1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63623168->63627263] (req 63623168->63627263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.347842:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554753664:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5126us (5538us total) trans 4294982842 rc 0/0 00000100:00100000:2.0:1789760778.347853:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15544 00000100:00100000:2.0:1789760778.347862:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721576140662 time 721588318809 state 0x2 ref 0 00000100:00100000:3.0:1789760778.355598:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760778.355788:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554753792 00000100:00100000:1.0:1789760778.355798:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554753792 00000100:00100000:1.0:1789760778.355850:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721576140662 time 721588318809 state 0x0 ref 1 00000100:00100000:1.0:1789760778.355865:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721588579225 time 721588318809 state 0x1 ref 1 00000100:00100000:1.0:1789760778.355953:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721588579225 time 721601570086 state 0x0 ref 1 00000100:00100000:1.0:1789760778.355967:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554753792:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760778.356015:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63627264, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e588 00010000:00010000:1.0:1789760778.356023:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c52f8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.356040:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760778.356044:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.356050:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c52f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63627264->63631359] (req 63627264->63631359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.356062:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c52f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63627264->63631359] (req 63627264->63631359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.361534:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982843 is committed 00002000:00010000:1.0:1789760778.362060:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63627264 -> 63631360 00002000:00010000:1.0:1789760778.362065:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124296 -> 124304 00010000:00010000:1.0:1789760778.362077:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c52f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63627264->63631359] (req 63627264->63631359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.362091:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c52f8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63627264->63631359] (req 63627264->63631359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.362100:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c52f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63627264->63631359] (req 63627264->63631359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.362107:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760778.362112:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c52f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63627264->63631359] (req 63627264->63631359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.362122:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c52f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63627264->63631359] (req 63627264->63631359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.362140:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c52f8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63627264->63631359] (req 63627264->63631359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.362147:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c52f8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63627264->63631359] (req 63627264->63631359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.362241:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554753792:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6272us (6655us total) trans 4294982843 rc 0/0 00000100:00100000:1.0:1789760778.362252:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15545 00000100:00100000:1.0:1789760778.362262:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721588579225 time 721601570086 state 0x2 ref 0 00000100:00100000:1.0:1789760778.365602:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760778.365894:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554753920 00000100:00100000:0.0:1789760778.365905:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554753920 00000100:00100000:0.0:1789760778.366029:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721588579225 time 721601570086 state 0x0 ref 1 00000100:00100000:0.0:1789760778.366045:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721601830502 time 721601570086 state 0x1 ref 1 00000100:00100000:0.0:1789760778.366079:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721601830502 time 721611697216 state 0x0 ref 1 00000100:00100000:0.0:1789760778.366151:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554753920:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760778.366198:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63631360, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e590 00010000:00010000:0.0:1789760778.366207:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52ff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.366223:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760778.366227:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.366233:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63631360->63635455] (req 63631360->63635455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.366252:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63631360->63635455] (req 63631360->63635455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.372558:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982844 is committed 00002000:00010000:0.0:1789760778.372666:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63631360 -> 63635456 00002000:00010000:0.0:1789760778.372671:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124304 -> 124312 00010000:00010000:0.0:1789760778.372684:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63631360->63635455] (req 63631360->63635455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.372698:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63631360->63635455] (req 63631360->63635455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.372881:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63631360->63635455] (req 63631360->63635455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.372890:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760778.372895:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63631360->63635455] (req 63631360->63635455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.372905:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63631360->63635455] (req 63631360->63635455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.372924:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52ff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63631360->63635455] (req 63631360->63635455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.372933:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c52ff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63631360->63635455] (req 63631360->63635455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.373032:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554753920:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6938us (7433us total) trans 4294982844 rc 0/0 00000100:00100000:0.0:1789760778.373044:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15546 00000100:00100000:0.0:1789760778.373053:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721601830502 time 721611697216 state 0x2 ref 0 00000100:00100000:3.0:1789760778.377012:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760778.377163:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554754048 00000100:00100000:2.0:1789760778.377172:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554754048 00000100:00100000:2.0:1789760778.377224:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721601830502 time 721611697216 state 0x0 ref 1 00000100:00100000:2.0:1789760778.377240:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721611957632 time 721611697216 state 0x1 ref 1 00000100:00100000:2.0:1789760778.377275:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721611957632 time 721622893623 state 0x0 ref 1 00000100:00100000:2.0:1789760778.377286:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e56d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554754048:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760778.377335:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63635456, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e598 00010000:00010000:2.0:1789760778.377344:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5306 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.377359:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760778.377363:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.377370:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5306 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63635456->63639551] (req 63635456->63639551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.377892:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5306 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63635456->63639551] (req 63635456->63639551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.383098:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982845 is committed 00002000:00010000:1.0:1789760778.383220:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63635456 -> 63639552 00002000:00010000:1.0:1789760778.383225:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124312 -> 124320 00010000:00010000:1.0:1789760778.383239:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5306 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63635456->63639551] (req 63635456->63639551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.383251:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5306 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63635456->63639551] (req 63635456->63639551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.383260:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5306 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63635456->63639551] (req 63635456->63639551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.383268:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760778.383351:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5306 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63635456->63639551] (req 63635456->63639551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.383361:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5306 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63635456->63639551] (req 63635456->63639551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.383414:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5306 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63635456->63639551] (req 63635456->63639551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.383423:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5306 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63635456->63639551] (req 63635456->63639551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.383630:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e56d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554754048:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6340us (6621us total) trans 4294982845 rc 0/0 00000100:00100000:1.0:1789760778.383641:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15547 00000100:00100000:1.0:1789760778.383650:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721611957632 time 721622893623 state 0x2 ref 0 00000100:00100000:2.0:1789760778.390643:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760778.393066:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554754176 00000100:00100000:0.0:1789760778.393082:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554754176 00000100:00100000:0.0:1789760778.393143:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721611957632 time 721622893623 state 0x0 ref 1 00000100:00100000:0.0:1789760778.393160:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721623154039 time 721622893623 state 0x1 ref 1 00000100:00100000:0.0:1789760778.393197:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721623154039 time 721638814895 state 0x0 ref 1 00000100:00100000:0.0:1789760778.393327:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04562a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554754176:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760778.393399:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63639552, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e5a0 00010000:00010000:0.0:1789760778.393409:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c530d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.393426:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760778.393430:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.393437:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c530d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63639552->63643647] (req 63639552->63643647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.393452:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c530d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63639552->63643647] (req 63639552->63643647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.399091:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982846 is committed 00002000:00010000:0.0:1789760778.402096:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63639552 -> 63643648 00002000:00010000:0.0:1789760778.402178:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124320 -> 124328 00010000:00010000:0.0:1789760778.402195:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c530d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63639552->63643647] (req 63639552->63643647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.402208:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c530d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63639552->63643647] (req 63639552->63643647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.402272:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c530d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63639552->63643647] (req 63639552->63643647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.402286:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760778.402291:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c530d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63639552->63643647] (req 63639552->63643647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.402301:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c530d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63639552->63643647] (req 63639552->63643647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.402322:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c530d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63639552->63643647] (req 63639552->63643647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.402330:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c530d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63639552->63643647] (req 63639552->63643647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.402577:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04562a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554754176:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9364us (11938us total) trans 4294982846 rc 0/0 00000100:00100000:0.0:1789760778.402588:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15548 00000100:00100000:0.0:1789760778.402597:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721623154039 time 721638814895 state 0x2 ref 0 00000100:00100000:3.0:1789760778.406999:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760778.407272:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554754304 00000100:00100000:1.0:1789760778.407283:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554754304 00000100:00100000:1.0:1789760778.407334:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721623154039 time 721638814895 state 0x0 ref 1 00000100:00100000:1.0:1789760778.407350:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721639075311 time 721638814895 state 0x1 ref 1 00000100:00100000:1.0:1789760778.407410:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721639075311 time 721653027840 state 0x0 ref 1 00000100:00100000:1.0:1789760778.407428:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e54000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554754304:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760778.407472:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63643648, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e5a8 00010000:00010000:1.0:1789760778.407482:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5314 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.407498:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760778.407502:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.407508:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5314 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63643648->63647743] (req 63643648->63647743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.407522:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5314 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63643648->63647743] (req 63643648->63647743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.413103:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982847 is committed 00002000:00010000:1.0:1789760778.413606:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63643648 -> 63647744 00002000:00010000:1.0:1789760778.413611:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124328 -> 124336 00010000:00010000:1.0:1789760778.413624:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5314 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63643648->63647743] (req 63643648->63647743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.413636:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5314 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63643648->63647743] (req 63643648->63647743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.413644:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5314 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63643648->63647743] (req 63643648->63647743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.413651:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760778.413656:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5314 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63643648->63647743] (req 63643648->63647743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.413665:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5314 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63643648->63647743] (req 63643648->63647743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.413683:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5314 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63643648->63647743] (req 63643648->63647743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.413691:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5314 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63643648->63647743] (req 63643648->63647743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.413963:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554754304:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6533us (6966us total) trans 4294982847 rc 0/0 00000100:00100000:1.0:1789760778.413975:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15549 00000100:00100000:1.0:1789760778.413984:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721639075311 time 721653027840 state 0x2 ref 0 00000100:00100000:3.0:1789760778.423616:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760778.425485:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554754432 00000100:00100000:1.0:1789760778.425499:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554754432 00000100:00100000:1.0:1789760778.425559:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721639075311 time 721653027840 state 0x0 ref 1 00000100:00100000:1.0:1789760778.425574:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721653288256 time 721653027840 state 0x1 ref 1 00000100:00100000:1.0:1789760778.425971:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721653288256 time 721671587041 state 0x0 ref 1 00000100:00100000:1.0:1789760778.425987:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e56300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554754432:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760778.426039:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63647744, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e5b0 00010000:00010000:1.0:1789760778.426049:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c531b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.426064:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760778.426068:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.426073:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c531b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63647744->63651839] (req 63647744->63651839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.426086:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c531b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63647744->63651839] (req 63647744->63651839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.433234:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982848 is committed 00002000:00010000:1.0:1789760778.433602:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63647744 -> 63651840 00002000:00010000:1.0:1789760778.433607:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124336 -> 124344 00010000:00010000:1.0:1789760778.433621:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c531b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63647744->63651839] (req 63647744->63651839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.433636:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c531b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63647744->63651839] (req 63647744->63651839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.433644:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c531b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63647744->63651839] (req 63647744->63651839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.433651:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760778.433656:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c531b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63647744->63651839] (req 63647744->63651839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.433665:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c531b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63647744->63651839] (req 63647744->63651839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.433686:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c531b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63647744->63651839] (req 63647744->63651839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.433693:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c531b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63647744->63651839] (req 63647744->63651839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.434081:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e56300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554754432:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8092us (10468us total) trans 4294982848 rc 0/0 00000100:00100000:1.0:1789760778.434093:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15550 00000100:00100000:1.0:1789760778.434110:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721653288256 time 721671587041 state 0x2 ref 0 00000100:00100000:3.0:1789760778.438213:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760778.438527:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554754560 00000100:00100000:2.0:1789760778.438537:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554754560 00000100:00100000:2.0:1789760778.438583:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721653288256 time 721671587041 state 0x0 ref 1 00000100:00100000:2.0:1789760778.438598:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721671847457 time 721671587041 state 0x1 ref 1 00000100:00100000:2.0:1789760778.438651:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721671847457 time 721684269886 state 0x0 ref 1 00000100:00100000:2.0:1789760778.438663:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554754560:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760778.438719:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63651840, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e5b8 00010000:00010000:2.0:1789760778.438728:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5322 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.438744:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760778.438748:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.438754:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5322 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63651840->63655935] (req 63651840->63655935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.438768:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5322 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63651840->63655935] (req 63651840->63655935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.444123:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982849 is committed 00002000:00010000:2.0:1789760778.444224:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63651840 -> 63655936 00002000:00010000:2.0:1789760778.444228:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124344 -> 124352 00010000:00010000:2.0:1789760778.444239:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5322 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63651840->63655935] (req 63651840->63655935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.444249:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5322 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63651840->63655935] (req 63651840->63655935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.444255:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5322 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63651840->63655935] (req 63651840->63655935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.444259:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760778.444349:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5322 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63651840->63655935] (req 63651840->63655935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.444357:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5322 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63651840->63655935] (req 63651840->63655935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.444373:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5322 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63651840->63655935] (req 63651840->63655935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.444405:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5322 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63651840->63655935] (req 63651840->63655935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.444498:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554754560:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5831us (6288us total) trans 4294982849 rc 0/0 00000100:00100000:2.0:1789760778.444518:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15551 00000100:00100000:2.0:1789760778.444528:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721671847457 time 721684269886 state 0x2 ref 0 00000100:00100000:2.0:1789760778.449686:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760778.449853:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554754688 00000100:00100000:0.0:1789760778.449863:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554754688 00000100:00100000:0.0:1789760778.449915:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721671847457 time 721684269886 state 0x0 ref 1 00000100:00100000:0.0:1789760778.449930:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721684530302 time 721684269886 state 0x1 ref 1 00000100:00100000:0.0:1789760778.449962:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721684530302 time 721695580879 state 0x0 ref 1 00000100:00100000:0.0:1789760778.449975:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04562300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554754688:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760778.450090:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63655936, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e5c0 00010000:00010000:0.0:1789760778.450154:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5329 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.450171:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760778.450175:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.450182:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5329 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63655936->63660031] (req 63655936->63660031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.450194:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5329 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63655936->63660031] (req 63655936->63660031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.454662:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982850 is committed 00002000:00010000:0.0:1789760778.455360:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63655936 -> 63660032 00002000:00010000:0.0:1789760778.455366:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124352 -> 124360 00010000:00010000:0.0:1789760778.455513:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5329 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63655936->63660031] (req 63655936->63660031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.455977:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5329 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63655936->63660031] (req 63655936->63660031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.456055:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5329 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63655936->63660031] (req 63655936->63660031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.456070:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760778.456075:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5329 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63655936->63660031] (req 63655936->63660031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.456084:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5329 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63655936->63660031] (req 63655936->63660031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.456155:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5329 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63655936->63660031] (req 63655936->63660031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.456164:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5329 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63655936->63660031] (req 63655936->63660031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.456325:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04562300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554754688:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6346us (6643us total) trans 4294982850 rc 0/0 00000100:00100000:0.0:1789760778.456336:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15552 00000100:00100000:0.0:1789760778.456346:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721684530302 time 721695580879 state 0x2 ref 0 00000100:00100000:3.0:1789760778.464525:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760778.465058:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554754816 00000100:00100000:1.0:1789760778.465068:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554754816 00000100:00100000:1.0:1789760778.465137:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721684530302 time 721695580879 state 0x0 ref 1 00000100:00100000:1.0:1789760778.465153:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721695841295 time 721695580879 state 0x1 ref 1 00000100:00100000:1.0:1789760778.465187:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721695841295 time 721710804934 state 0x0 ref 1 00000100:00100000:1.0:1789760778.465200:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e56a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554754816:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760778.465244:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63660032, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e5c8 00010000:00010000:1.0:1789760778.465254:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5330 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.465269:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760778.465274:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.465280:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5330 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63660032->63664127] (req 63660032->63664127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.465294:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5330 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63660032->63664127] (req 63660032->63664127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.473570:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982851 is committed 00002000:00010000:1.0:1789760778.473707:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63660032 -> 63664128 00002000:00010000:1.0:1789760778.473726:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124360 -> 124368 00010000:00010000:1.0:1789760778.473746:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5330 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63660032->63664127] (req 63660032->63664127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.473759:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5330 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63660032->63664127] (req 63660032->63664127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.473768:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5330 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63660032->63664127] (req 63660032->63664127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.473775:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760778.473781:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5330 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63660032->63664127] (req 63660032->63664127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.473790:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5330 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63660032->63664127] (req 63660032->63664127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.473809:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5330 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63660032->63664127] (req 63660032->63664127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.473834:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5330 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63660032->63664127] (req 63660032->63664127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.473925:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e56a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554754816:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8723us (9404us total) trans 4294982851 rc 0/0 00000100:00100000:1.0:1789760778.473935:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15553 00000100:00100000:1.0:1789760778.473945:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721695841295 time 721710804934 state 0x2 ref 0 00000100:00100000:3.0:1789760778.479600:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760778.480730:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554754944 00000100:00100000:1.0:1789760778.480741:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554754944 00000100:00100000:1.0:1789760778.480796:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721695841295 time 721710804934 state 0x0 ref 1 00000100:00100000:1.0:1789760778.480811:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721711065350 time 721710804934 state 0x1 ref 1 00000100:00100000:1.0:1789760778.480880:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721711065350 time 721726498657 state 0x0 ref 1 00000100:00100000:1.0:1789760778.480893:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554754944:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760778.480936:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63664128, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e5d0 00010000:00010000:1.0:1789760778.480945:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5337 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.480960:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760778.480965:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.480970:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5337 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63664128->63668223] (req 63664128->63668223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.480998:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5337 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63664128->63668223] (req 63664128->63668223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.485187:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982852 is committed 00002000:00010000:1.0:1789760778.485297:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63664128 -> 63668224 00002000:00010000:1.0:1789760778.485302:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124368 -> 124376 00010000:00010000:1.0:1789760778.485314:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5337 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63664128->63668223] (req 63664128->63668223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.485333:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5337 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63664128->63668223] (req 63664128->63668223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.485343:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5337 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63664128->63668223] (req 63664128->63668223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.485351:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760778.485357:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5337 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63664128->63668223] (req 63664128->63668223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.485366:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5337 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63664128->63668223] (req 63664128->63668223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.485577:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5337 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63664128->63668223] (req 63664128->63668223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.485587:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5337 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63664128->63668223] (req 63664128->63668223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.485730:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554754944:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4834us (6135us total) trans 4294982852 rc 0/0 00000100:00100000:1.0:1789760778.485742:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15554 00000100:00100000:1.0:1789760778.485751:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721711065350 time 721726498657 state 0x2 ref 0 00000100:00100000:2.0:1789760778.490112:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760778.490247:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554755072 00000100:00100000:1.0:1789760778.490257:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554755072 00000100:00100000:1.0:1789760778.490305:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721711065350 time 721726498657 state 0x0 ref 1 00000100:00100000:1.0:1789760778.490319:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721726759073 time 721726498657 state 0x1 ref 1 00000100:00100000:1.0:1789760778.492995:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721726759073 time 721738603289 state 0x0 ref 1 00000100:00100000:0.0:1789760778.495024:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04560380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554755072:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760778.495092:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63668224, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e5d8 00010000:00010000:0.0:1789760778.495102:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c533e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.495120:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760778.495125:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.495132:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c533e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63668224->63672319] (req 63668224->63672319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.495141:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00010000:00010000:0.0:1789760778.495154:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c533e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63668224->63672319] (req 63668224->63672319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.495224:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760778.495422:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760778.495452:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760778.495554:0:7470:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554755328 00000100:00100000:1.0:1789760778.495561:0:7470:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554755328 00000100:00100000:1.0:1789760778.495614:0:7470:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554755584 00000100:00100000:1.0:1789760778.495617:0:7470:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554755584 00000100:00100000:1.0:1789760778.495635:0:7470:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 875 00000100:00100000:1.0:1789760778.495642:0:7470:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04563480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+8:2411:x1876699554755328:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:1.0:1789760778.495718:0:7470:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04563480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+8:2411:x1876699554755328:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 66us (571us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760778.495725:0:7470:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 875 00000100:00100000:1.0:1789760778.495741:0:7470:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 876 00000100:00100000:1.0:1789760778.495747:0:7470:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04560000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2411:x1876699554755584:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:1.0:1789760778.495769:0:7470:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04560000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2411:x1876699554755584:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 22us (546us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760778.495775:0:7470:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 876 00000100:00100000:1.0:1789760778.495795:0:6514:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554755200 00000100:00100000:1.0:1789760778.495798:0:6514:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554755200 00000100:00100000:1.0:1789760778.495821:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 470 00000100:00100000:1.0:1789760778.495827:0:6514:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc320b6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9765eb0b-55b9-40b6-bfdf-ffee156c3226+13:2411:x1876699554755200:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:1.0:1789760778.495846:0:6514:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc320b6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9765eb0b-55b9-40b6-bfdf-ffee156c3226+13:2411:x1876699554755200:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 21us (427us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760778.495852:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 470 00000100:00080000:1.0:1789760778.495875:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789760778.496140:0:11089:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554755456 00000100:00100000:1.0:1789760778.496146:0:11089:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554755456 00000100:00100000:2.0:1789760778.496170:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760778.496195:0:11089:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554755712 00000100:00100000:1.0:1789760778.496200:0:11089:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554755712 00000100:00100000:1.0:1789760778.496222:0:11089:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 877 00000100:00100000:1.0:1789760778.496230:0:11089:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04560a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554755456:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:1.0:1789760778.496273:0:11089:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04560a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554755456:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 44us (821us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760778.496281:0:11089:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 877 00000100:00100000:1.0:1789760778.496297:0:11089:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 878 00000100:00100000:1.0:1789760778.496303:0:11089:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04562680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2411:x1876699554755712:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:1.0:1789760778.496334:0:11089:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04562680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2411:x1876699554755712:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 31us (165us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760778.496339:0:11089:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 878 00000100:00080000:0.0:1789760778.496664:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000001:00080000:0.0:1789760778.500002:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982853 is committed 00002000:00010000:2.0:1789760778.500214:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63668224 -> 63672320 00002000:00010000:2.0:1789760778.500220:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124376 -> 124384 00010000:00010000:2.0:1789760778.500233:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c533e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63668224->63672319] (req 63668224->63672319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.500247:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c533e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63668224->63672319] (req 63668224->63672319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.500257:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c533e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63668224->63672319] (req 63668224->63672319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.500265:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760778.500270:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c533e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63668224->63672319] (req 63668224->63672319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.500280:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c533e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63668224->63672319] (req 63668224->63672319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.500779:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c533e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63668224->63672319] (req 63668224->63672319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.500791:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c533e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63668224->63672319] (req 63668224->63672319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.500915:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04560380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554755072:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6129us (10807us total) trans 4294982853 rc 0/0 00000100:00100000:2.0:1789760778.500925:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15555 00000100:00100000:2.0:1789760778.500934:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721726759073 time 721738603289 state 0x2 ref 0 00000100:00100000:2.0:1789760778.507053:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760778.508936:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554755840 00000100:00100000:0.0:1789760778.508949:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554755840 00000100:00100000:0.0:1789760778.509004:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721726759073 time 721738603289 state 0x0 ref 1 00000100:00100000:0.0:1789760778.509226:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721738863705 time 721738603289 state 0x1 ref 1 00000100:00100000:0.0:1789760778.509299:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721738863705 time 721754917196 state 0x0 ref 1 00000100:00100000:0.0:1789760778.509312:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04563b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554755840:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760778.509360:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63672320, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e5e0 00010000:00010000:0.0:1789760778.509371:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5345 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.509413:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760778.509417:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.509424:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5345 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63672320->63676415] (req 63672320->63676415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.509439:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5345 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63672320->63676415] (req 63672320->63676415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.517357:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982854 is committed 00002000:00010000:2.0:1789760778.517490:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63672320 -> 63676416 00002000:00010000:2.0:1789760778.517494:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124384 -> 124392 00010000:00010000:2.0:1789760778.517507:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5345 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63672320->63676415] (req 63672320->63676415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.517521:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5345 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63672320->63676415] (req 63672320->63676415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.517530:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5345 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63672320->63676415] (req 63672320->63676415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.517543:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760778.517547:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5345 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63672320->63676415] (req 63672320->63676415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.517556:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5345 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63672320->63676415] (req 63672320->63676415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.517576:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5345 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63672320->63676415] (req 63672320->63676415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.517584:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5345 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63672320->63676415] (req 63672320->63676415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.517677:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04563b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554755840:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8363us (10629us total) trans 4294982854 rc 0/0 00000100:00100000:2.0:1789760778.517687:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15556 00000100:00100000:2.0:1789760778.517697:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721738863705 time 721754917196 state 0x2 ref 0 00000100:00100000:0.0:1789760778.524636:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760778.524841:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554755968 00000100:00100000:3.0:1789760778.524850:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554755968 00000100:00100000:3.0:1789760778.524901:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721738863705 time 721754917196 state 0x0 ref 1 00000100:00100000:3.0:1789760778.524917:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721755177612 time 721754917196 state 0x1 ref 1 00000100:00100000:3.0:1789760778.524977:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721755177612 time 721770594739 state 0x0 ref 1 00000100:00100000:3.0:1789760778.524990:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554755968:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760778.525052:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63676416, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e5e8 00010000:00010000:3.0:1789760778.525062:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c534c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.525079:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760778.525083:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.525089:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c534c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63676416->63680511] (req 63676416->63680511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.525105:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c534c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63676416->63680511] (req 63676416->63680511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.532010:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982855 is committed 00002000:00010000:3.0:1789760778.532186:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63676416 -> 63680512 00002000:00010000:3.0:1789760778.532193:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124392 -> 124400 00010000:00010000:3.0:1789760778.532211:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c534c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63676416->63680511] (req 63676416->63680511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.532226:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c534c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63676416->63680511] (req 63676416->63680511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.532238:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c534c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63676416->63680511] (req 63676416->63680511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.532246:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760778.532251:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c534c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63676416->63680511] (req 63676416->63680511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.532267:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c534c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63676416->63680511] (req 63676416->63680511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.532292:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c534c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63676416->63680511] (req 63676416->63680511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.532301:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c534c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63676416->63680511] (req 63676416->63680511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.532603:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554755968:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7608us (7968us total) trans 4294982855 rc 0/0 00000100:00100000:3.0:1789760778.532617:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15557 00000100:00100000:3.0:1789760778.532630:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721755177612 time 721770594739 state 0x2 ref 0 00000100:00100000:3.0:1789760778.537019:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760778.537118:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554756096 00000100:00100000:1.0:1789760778.537128:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554756096 00000100:00100000:1.0:1789760778.537184:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721755177612 time 721770594739 state 0x0 ref 1 00000100:00100000:1.0:1789760778.537200:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721770855155 time 721770594739 state 0x1 ref 1 00000100:00100000:1.0:1789760778.537232:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721770855155 time 721782851002 state 0x0 ref 1 00000100:00100000:1.0:1789760778.537246:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06490700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554756096:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760778.537291:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63680512, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e5f0 00010000:00010000:1.0:1789760778.537301:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5353 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.537316:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760778.537321:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.537328:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5353 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63680512->63684607] (req 63680512->63684607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.537342:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5353 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63680512->63684607] (req 63680512->63684607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.542252:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982856 is committed 00002000:00010000:1.0:1789760778.544909:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63680512 -> 63684608 00002000:00010000:1.0:1789760778.544922:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124400 -> 124408 00010000:00010000:1.0:1789760778.544939:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5353 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63680512->63684607] (req 63680512->63684607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.544956:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5353 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63680512->63684607] (req 63680512->63684607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.544973:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5353 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63680512->63684607] (req 63680512->63684607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.544982:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760778.544988:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5353 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63680512->63684607] (req 63680512->63684607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.544998:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5353 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63680512->63684607] (req 63680512->63684607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.545023:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5353 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63680512->63684607] (req 63680512->63684607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.545032:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5353 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63680512->63684607] (req 63680512->63684607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.545361:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06490700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554756096:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8100us (8334us total) trans 4294982856 rc 0/0 00000100:00100000:1.0:1789760778.545374:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15558 00000100:00100000:1.0:1789760778.545433:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721770855155 time 721782851002 state 0x2 ref 0 00000100:00100000:2.0:1789760778.555563:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760778.556769:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554756224 00000100:00100000:0.0:1789760778.556782:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554756224 00000100:00100000:0.0:1789760778.556842:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721770855155 time 721782851002 state 0x0 ref 1 00000100:00100000:0.0:1789760778.556857:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721783111418 time 721782851002 state 0x1 ref 1 00000100:00100000:0.0:1789760778.556945:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721783111418 time 721802562160 state 0x0 ref 1 00000100:00100000:0.0:1789760778.556958:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554756224:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760778.557006:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63684608, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e5f8 00010000:00010000:0.0:1789760778.557016:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c535a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.557032:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760778.557037:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.557044:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c535a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63684608->63688703] (req 63684608->63688703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.557058:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c535a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63684608->63688703] (req 63684608->63688703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.563963:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982857 is committed 00002000:00010000:0.0:1789760778.564187:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63684608 -> 63688704 00002000:00010000:0.0:1789760778.564198:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124408 -> 124416 00010000:00010000:0.0:1789760778.564212:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c535a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63684608->63688703] (req 63684608->63688703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.564225:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c535a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63684608->63688703] (req 63684608->63688703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.564234:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c535a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63684608->63688703] (req 63684608->63688703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.564241:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760778.564246:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c535a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63684608->63688703] (req 63684608->63688703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.564256:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c535a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63684608->63688703] (req 63684608->63688703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.564277:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c535a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63684608->63688703] (req 63684608->63688703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.564284:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c535a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63684608->63688703] (req 63684608->63688703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.565565:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554756224:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7476us (8882us total) trans 4294982857 rc 0/0 00000100:00100000:0.0:1789760778.565591:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15559 00000100:00100000:0.0:1789760778.565601:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721783111418 time 721802562160 state 0x2 ref 0 00000100:00100000:0.0:1789760778.569884:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760778.572577:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554756352 00000100:00100000:2.0:1789760778.572592:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554756352 00000100:00100000:2.0:1789760778.572656:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721783111418 time 721802562160 state 0x0 ref 1 00000100:00100000:2.0:1789760778.572671:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721802822576 time 721802562160 state 0x1 ref 1 00000100:00100000:2.0:1789760778.572729:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721802822576 time 721818346687 state 0x0 ref 1 00000100:00100000:2.0:1789760778.572742:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554756352:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760778.573070:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63688704, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e600 00010000:00010000:2.0:1789760778.573080:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5361 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.573100:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760778.573105:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.573112:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5361 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63688704->63692799] (req 63688704->63692799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.573126:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5361 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63688704->63692799] (req 63688704->63692799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.579959:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982858 is committed 00002000:00010000:2.0:1789760778.580873:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63688704 -> 63692800 00002000:00010000:2.0:1789760778.580882:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124416 -> 124424 00010000:00010000:2.0:1789760778.580898:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5361 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63688704->63692799] (req 63688704->63692799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.580914:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5361 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63688704->63692799] (req 63688704->63692799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.580925:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5361 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63688704->63692799] (req 63688704->63692799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.580939:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760778.580946:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5361 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63688704->63692799] (req 63688704->63692799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.580956:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5361 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63688704->63692799] (req 63688704->63692799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.580981:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5361 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63688704->63692799] (req 63688704->63692799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.580989:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5361 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63688704->63692799] (req 63688704->63692799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.581241:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554756352:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8493us (11358us total) trans 4294982858 rc 0/0 00000100:00100000:2.0:1789760778.581255:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15560 00000100:00100000:2.0:1789760778.581267:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721802822576 time 721818346687 state 0x2 ref 0 00000100:00100000:3.0:1789760778.586089:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760778.586313:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554756480 00000100:00100000:1.0:1789760778.586327:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554756480 00000100:00100000:1.0:1789760778.586588:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721802822576 time 721818346687 state 0x0 ref 1 00000100:00100000:1.0:1789760778.586608:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721818607103 time 721818346687 state 0x1 ref 1 00000100:00100000:1.0:1789760778.586687:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721818607103 time 721832304968 state 0x0 ref 1 00000100:00100000:1.0:1789760778.586719:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06493800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554756480:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760778.586852:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63692800, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e608 00010000:00010000:1.0:1789760778.586865:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5368 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.586885:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760778.586890:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.586899:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5368 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63692800->63696895] (req 63692800->63696895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.586919:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5368 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63692800->63696895] (req 63692800->63696895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.592427:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982859 is committed 00002000:00010000:1.0:1789760778.592610:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63692800 -> 63696896 00002000:00010000:1.0:1789760778.592616:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124424 -> 124432 00010000:00010000:1.0:1789760778.592635:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5368 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63692800->63696895] (req 63692800->63696895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.592651:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5368 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63692800->63696895] (req 63692800->63696895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.592661:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5368 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63692800->63696895] (req 63692800->63696895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.592676:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760778.592682:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5368 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63692800->63696895] (req 63692800->63696895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.592693:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5368 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63692800->63696895] (req 63692800->63696895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.592731:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5368 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63692800->63696895] (req 63692800->63696895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.592741:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5368 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63692800->63696895] (req 63692800->63696895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.592864:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06493800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554756480:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6150us (6785us total) trans 4294982859 rc 0/0 00000100:00100000:1.0:1789760778.592882:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15561 00000100:00100000:1.0:1789760778.592896:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721818607103 time 721832304968 state 0x2 ref 0 00000100:00100000:0.0:1789760778.595541:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760778.596024:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554756608 00000100:00100000:1.0:1789760778.596039:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554756608 00000100:00100000:1.0:1789760778.596348:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721818607103 time 721832304968 state 0x0 ref 1 00000100:00100000:1.0:1789760778.596370:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721832565384 time 721832304968 state 0x1 ref 1 00000100:00100000:1.0:1789760778.596505:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721832565384 time 721842121904 state 0x0 ref 1 00000100:00100000:1.0:1789760778.596524:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554756608:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760778.596589:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63696896, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e610 00010000:00010000:1.0:1789760778.596600:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c536f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.596620:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760778.596625:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.596633:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c536f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63696896->63700991] (req 63696896->63700991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.596649:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c536f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63696896->63700991] (req 63696896->63700991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.602303:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982860 is committed 00002000:00010000:1.0:1789760778.602621:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63696896 -> 63700992 00002000:00010000:1.0:1789760778.602627:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124432 -> 124440 00010000:00010000:1.0:1789760778.602644:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c536f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63696896->63700991] (req 63696896->63700991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.602660:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c536f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63696896->63700991] (req 63696896->63700991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.602670:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c536f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63696896->63700991] (req 63696896->63700991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.602677:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760778.602683:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c536f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63696896->63700991] (req 63696896->63700991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.602692:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c536f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63696896->63700991] (req 63696896->63700991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.602733:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c536f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63696896->63700991] (req 63696896->63700991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.602742:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c536f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63696896->63700991] (req 63696896->63700991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.602870:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554756608:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6343us (7337us total) trans 4294982860 rc 0/0 00000100:00100000:1.0:1789760778.602883:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15562 00000100:00100000:1.0:1789760778.602895:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721832565384 time 721842121904 state 0x2 ref 0 00000100:00100000:0.0:1789760778.607345:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760778.608893:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554756736 00000100:00100000:2.0:1789760778.608913:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554756736 00000100:00100000:2.0:1789760778.609019:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721832565384 time 721842121904 state 0x0 ref 1 00000100:00100000:2.0:1789760778.609038:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721842382320 time 721842121904 state 0x1 ref 1 00000100:00100000:2.0:1789760778.609076:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721842382320 time 721854693707 state 0x0 ref 1 00000100:00100000:2.0:1789760778.609095:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554756736:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760778.609159:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63700992, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e618 00010000:00010000:2.0:1789760778.609171:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5376 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.609193:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760778.609198:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.609206:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5376 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63700992->63705087] (req 63700992->63705087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.609222:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5376 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63700992->63705087] (req 63700992->63705087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.614283:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982861 is committed 00002000:00010000:2.0:1789760778.617516:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63700992 -> 63705088 00002000:00010000:2.0:1789760778.617526:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124440 -> 124448 00010000:00010000:2.0:1789760778.617540:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5376 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63700992->63705087] (req 63700992->63705087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.617553:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5376 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63700992->63705087] (req 63700992->63705087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.617562:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5376 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63700992->63705087] (req 63700992->63705087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.617568:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760778.617574:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5376 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63700992->63705087] (req 63700992->63705087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.617582:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5376 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63700992->63705087] (req 63700992->63705087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.617601:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5376 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63700992->63705087] (req 63700992->63705087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.617609:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5376 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63700992->63705087] (req 63700992->63705087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.617737:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554756736:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8639us (10401us total) trans 4294982861 rc 0/0 00000100:00100000:2.0:1789760778.617749:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15563 00000100:00100000:2.0:1789760778.617759:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721842382320 time 721854693707 state 0x2 ref 0 00000100:00100000:3.0:1789760778.622600:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760778.622855:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554756864 00000100:00100000:2.0:1789760778.622870:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554756864 00000100:00100000:2.0:1789760778.622942:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721842382320 time 721854693707 state 0x0 ref 1 00000100:00100000:2.0:1789760778.622962:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721854954123 time 721854693707 state 0x1 ref 1 00000100:00100000:2.0:1789760778.623167:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721854954123 time 721868782803 state 0x0 ref 1 00000100:00100000:2.0:1789760778.623184:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06490a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554756864:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760778.623247:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63705088, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e620 00010000:00010000:2.0:1789760778.623258:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c537d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.623278:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760778.623282:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.623289:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c537d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63705088->63709183] (req 63705088->63709183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.623319:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c537d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63705088->63709183] (req 63705088->63709183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.631789:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982862 is committed 00002000:00010000:2.0:1789760778.632057:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63705088 -> 63709184 00002000:00010000:2.0:1789760778.632063:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124448 -> 124456 00010000:00010000:2.0:1789760778.632089:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c537d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63705088->63709183] (req 63705088->63709183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.632105:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c537d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63705088->63709183] (req 63705088->63709183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.632115:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c537d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63705088->63709183] (req 63705088->63709183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.632123:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760778.632130:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c537d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63705088->63709183] (req 63705088->63709183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.632141:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c537d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63705088->63709183] (req 63705088->63709183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.632254:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c537d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63705088->63709183] (req 63705088->63709183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.632262:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c537d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63705088->63709183] (req 63705088->63709183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.632615:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06490a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554756864:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9427us (10022us total) trans 4294982862 rc 0/0 00000100:00100000:2.0:1789760778.632627:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15564 00000100:00100000:2.0:1789760778.632639:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721854954123 time 721868782803 state 0x2 ref 0 00000100:00100000:0.0:1789760778.639718:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760778.640121:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554756992 00000100:00100000:1.0:1789760778.640135:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554756992 00000100:00100000:1.0:1789760778.640278:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721854954123 time 721868782803 state 0x0 ref 1 00000100:00100000:1.0:1789760778.640298:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721869043219 time 721868782803 state 0x1 ref 1 00000100:00100000:1.0:1789760778.640500:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721869043219 time 721886116891 state 0x0 ref 1 00000100:00100000:1.0:1789760778.640517:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554756992:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760778.640585:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63709184, mtime: 0x6aad9509, atime: 0x0, ctime: 0x6aad9509, blocks: 0x1e628 00010000:00010000:1.0:1789760778.640597:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5384 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.640617:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760778.640623:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.640631:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5384 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63709184->63713279] (req 63709184->63713279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.640650:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5384 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63709184->63713279] (req 63709184->63713279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.646815:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982863 is committed 00002000:00010000:3.0:1789760778.647000:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63709184 -> 63713280 00002000:00010000:3.0:1789760778.647006:0:8441:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760777 -> 1789760778 00002000:00010000:3.0:1789760778.647009:0:8441:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760777 -> 1789760778 00002000:00010000:3.0:1789760778.647012:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124456 -> 124464 00010000:00010000:3.0:1789760778.647031:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5384 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63709184->63713279] (req 63709184->63713279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.647046:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5384 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63709184->63713279] (req 63709184->63713279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.647058:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5384 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63709184->63713279] (req 63709184->63713279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.647067:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760778.647146:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5384 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63709184->63713279] (req 63709184->63713279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.647159:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5384 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63709184->63713279] (req 63709184->63713279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.647196:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5384 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63709184->63713279] (req 63709184->63713279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.647207:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5384 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63709184->63713279] (req 63709184->63713279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.647373:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554756992:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6848us (7668us total) trans 4294982863 rc 0/0 00000100:00100000:3.0:1789760778.647436:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15565 00000100:00100000:3.0:1789760778.647451:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721869043219 time 721886116891 state 0x2 ref 0 00000100:00100000:0.0:1789760778.653918:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760778.653994:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554757120 00000100:00100000:1.0:1789760778.654008:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554757120 00000100:00100000:1.0:1789760778.654067:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721869043219 time 721886116891 state 0x0 ref 1 00000100:00100000:1.0:1789760778.654082:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721886377307 time 721886116891 state 0x1 ref 1 00000100:00100000:1.0:1789760778.654137:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721886377307 time 721899755141 state 0x0 ref 1 00000100:00100000:1.0:1789760778.654150:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554757120:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760778.654194:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63713280, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e630 00010000:00010000:1.0:1789760778.654203:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c538b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.654219:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760778.654224:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.654230:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c538b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63713280->63717375] (req 63713280->63717375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.654244:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c538b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63713280->63717375] (req 63713280->63717375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.660437:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982864 is committed 00002000:00010000:1.0:1789760778.661547:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63713280 -> 63717376 00002000:00010000:1.0:1789760778.661554:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124464 -> 124472 00010000:00010000:1.0:1789760778.661568:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c538b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63713280->63717375] (req 63713280->63717375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.661582:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c538b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63713280->63717375] (req 63713280->63717375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.661651:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c538b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63713280->63717375] (req 63713280->63717375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.661659:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760778.661665:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c538b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63713280->63717375] (req 63713280->63717375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.661674:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c538b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63713280->63717375] (req 63713280->63717375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.661697:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c538b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63713280->63717375] (req 63713280->63717375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.661718:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c538b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63713280->63717375] (req 63713280->63717375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.662006:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554757120:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7853us (8097us total) trans 4294982864 rc 0/0 00000100:00100000:1.0:1789760778.662016:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15566 00000100:00100000:1.0:1789760778.662026:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721886377307 time 721899755141 state 0x2 ref 0 00000100:00100000:0.0:1789760778.668843:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760778.668991:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554757248 00000100:00100000:2.0:1789760778.669002:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554757248 00000100:00100000:2.0:1789760778.669059:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721886377307 time 721899755141 state 0x0 ref 1 00000100:00100000:2.0:1789760778.669075:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721900015557 time 721899755141 state 0x1 ref 1 00000100:00100000:2.0:1789760778.669116:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721900015557 time 721914733203 state 0x0 ref 1 00000100:00100000:2.0:1789760778.669127:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554757248:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760778.669174:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63717376, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e638 00010000:00010000:2.0:1789760778.669184:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5392 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.669204:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760778.669208:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.669215:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5392 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63717376->63721471] (req 63717376->63721471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.669228:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5392 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63717376->63721471] (req 63717376->63721471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.674681:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982865 is committed 00002000:00010000:2.0:1789760778.679184:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63717376 -> 63721472 00002000:00010000:2.0:1789760778.679194:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124472 -> 124480 00010000:00010000:2.0:1789760778.679209:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5392 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63717376->63721471] (req 63717376->63721471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.679222:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5392 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63717376->63721471] (req 63717376->63721471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.679231:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5392 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63717376->63721471] (req 63717376->63721471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.679239:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760778.679244:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5392 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63717376->63721471] (req 63717376->63721471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.679254:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5392 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63717376->63721471] (req 63717376->63721471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.679280:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5392 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63717376->63721471] (req 63717376->63721471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.679287:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5392 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63717376->63721471] (req 63717376->63721471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.679792:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554757248:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10660us (10953us total) trans 4294982865 rc 0/0 00000100:00100000:2.0:1789760778.679804:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15567 00000100:00100000:2.0:1789760778.679814:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721900015557 time 721914733203 state 0x2 ref 0 00000100:00100000:0.0:1789760778.683313:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760778.683964:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554757376 00000100:00100000:2.0:1789760778.683977:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554757376 00000100:00100000:2.0:1789760778.684036:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721900015557 time 721914733203 state 0x0 ref 1 00000100:00100000:2.0:1789760778.684051:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721914993619 time 721914733203 state 0x1 ref 1 00000100:00100000:2.0:1789760778.684094:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721914993619 time 721929711931 state 0x0 ref 1 00000100:00100000:2.0:1789760778.684107:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554757376:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760778.684157:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63721472, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e640 00010000:00010000:2.0:1789760778.684166:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5399 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.684183:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760778.684190:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.684196:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5399 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63721472->63725567] (req 63721472->63725567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.684211:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5399 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63721472->63725567] (req 63721472->63725567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.692532:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982866 is committed 00002000:00010000:0.0:1789760778.692636:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63721472 -> 63725568 00002000:00010000:0.0:1789760778.692642:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124480 -> 124488 00010000:00010000:0.0:1789760778.692656:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5399 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63721472->63725567] (req 63721472->63725567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.692669:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5399 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63721472->63725567] (req 63721472->63725567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.692679:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5399 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63721472->63725567] (req 63721472->63725567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.692686:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760778.692692:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5399 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63721472->63725567] (req 63721472->63725567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.692702:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5399 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63721472->63725567] (req 63721472->63725567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.692732:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5399 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63721472->63725567] (req 63721472->63725567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.692740:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5399 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63721472->63725567] (req 63721472->63725567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.693697:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554757376:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9584us (10384us total) trans 4294982866 rc 0/0 00000100:00100000:0.0:1789760778.693720:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15568 00000100:00100000:0.0:1789760778.693730:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721914993619 time 721929711931 state 0x2 ref 0 00000100:00100000:0.0:1789760778.697887:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760778.698097:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554757504 00000100:00100000:3.0:1789760778.698108:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554757504 00000100:00100000:3.0:1789760778.698166:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721914993619 time 721929711931 state 0x0 ref 1 00000100:00100000:3.0:1789760778.698181:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721929972347 time 721929711931 state 0x1 ref 1 00000100:00100000:3.0:1789760778.698212:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721929972347 time 721943831386 state 0x0 ref 1 00000100:00100000:3.0:1789760778.698224:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554757504:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760778.698269:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63725568, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e648 00010000:00010000:3.0:1789760778.698278:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c53a0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.698293:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:3.0:1789760778.698298:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760778.698304:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c53a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63725568->63729663] (req 63725568->63729663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.698318:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c53a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63725568->63729663] (req 63725568->63729663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.705217:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982867 is committed 00002000:00010000:3.0:1789760778.705519:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63725568 -> 63729664 00002000:00010000:3.0:1789760778.705524:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124488 -> 124496 00010000:00010000:3.0:1789760778.705537:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c53a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63725568->63729663] (req 63725568->63729663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.705548:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c53a0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63725568->63729663] (req 63725568->63729663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.705558:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c53a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63725568->63729663] (req 63725568->63729663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.705570:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760778.705575:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c53a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63725568->63729663] (req 63725568->63729663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.705585:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c53a0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63725568->63729663] (req 63725568->63729663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.705604:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c53a0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63725568->63729663] (req 63725568->63729663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760778.705612:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c53a0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63725568->63729663] (req 63725568->63729663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760778.705722:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554757504:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7485us (7830us total) trans 4294982867 rc 0/0 00000100:00100000:3.0:1789760778.705733:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15569 00000100:00100000:3.0:1789760778.705743:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721929972347 time 721943831386 state 0x2 ref 0 00000100:00100000:1.0:1789760778.709153:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760778.710358:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554757632 00000100:00100000:0.0:1789760778.710370:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554757632 00000100:00100000:0.0:1789760778.710559:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721929972347 time 721943831386 state 0x0 ref 1 00000100:00100000:0.0:1789760778.710575:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721944091802 time 721943831386 state 0x1 ref 1 00000100:00100000:0.0:1789760778.710655:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721944091802 time 721956272027 state 0x0 ref 1 00000100:00100000:0.0:1789760778.710668:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe1c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554757632:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760778.710726:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63729664, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e650 00010000:00010000:0.0:1789760778.710736:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c53a7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.710753:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:0.0:1789760778.710756:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.710763:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c53a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63729664->63733759] (req 63729664->63733759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.710777:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c53a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63729664->63733759] (req 63729664->63733759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.716976:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982868 is committed 00002000:00010000:0.0:1789760778.718035:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63729664 -> 63733760 00002000:00010000:0.0:1789760778.718045:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124496 -> 124504 00010000:00010000:0.0:1789760778.718059:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c53a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63729664->63733759] (req 63729664->63733759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.718072:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c53a7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63729664->63733759] (req 63729664->63733759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.718081:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c53a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63729664->63733759] (req 63729664->63733759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.718089:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760778.718094:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c53a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63729664->63733759] (req 63729664->63733759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.718109:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c53a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63729664->63733759] (req 63729664->63733759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.718128:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c53a7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63729664->63733759] (req 63729664->63733759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.718188:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c53a7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63729664->63733759] (req 63729664->63733759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.718362:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe1c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554757632:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7690us (9211us total) trans 4294982868 rc 0/0 00000100:00100000:0.0:1789760778.718372:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15570 00000100:00100000:0.0:1789760778.718421:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721944091802 time 721956272027 state 0x2 ref 0 00000100:00100000:0.0:1789760778.724704:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760778.724806:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554757760 00000100:00100000:2.0:1789760778.724889:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554757760 00000100:00100000:2.0:1789760778.724949:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721944091802 time 721956272027 state 0x0 ref 1 00000100:00100000:2.0:1789760778.724964:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721956532443 time 721956272027 state 0x1 ref 1 00000100:00100000:2.0:1789760778.724995:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721956532443 time 721970613353 state 0x0 ref 1 00000100:00100000:2.0:1789760778.725007:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554757760:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760778.725053:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63733760, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e658 00010000:00010000:2.0:1789760778.725063:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c53ae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.725079:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760778.725084:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.725091:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c53ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63733760->63737855] (req 63733760->63737855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.725104:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c53ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63733760->63737855] (req 63733760->63737855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.734827:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982869 is committed 00002000:00010000:0.0:1789760778.734948:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63733760 -> 63737856 00002000:00010000:0.0:1789760778.734953:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124504 -> 124512 00010000:00010000:0.0:1789760778.734967:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c53ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63733760->63737855] (req 63733760->63737855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.734979:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c53ae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63733760->63737855] (req 63733760->63737855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.734988:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c53ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63733760->63737855] (req 63733760->63737855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.735001:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760778.735006:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c53ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63733760->63737855] (req 63733760->63737855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.735016:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c53ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63733760->63737855] (req 63733760->63737855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.735037:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c53ae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63733760->63737855] (req 63733760->63737855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.735045:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c53ae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63733760->63737855] (req 63733760->63737855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.735159:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554757760:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10149us (10461us total) trans 4294982869 rc 0/0 00000100:00100000:0.0:1789760778.735170:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15571 00000100:00100000:0.0:1789760778.735180:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721956532443 time 721970613353 state 0x2 ref 0 00000100:00100000:3.0:1789760778.738549:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760778.738690:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554757888 00000100:00100000:2.0:1789760778.738702:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554757888 00000100:00100000:2.0:1789760778.738768:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721956532443 time 721970613353 state 0x0 ref 1 00000100:00100000:2.0:1789760778.738788:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721970873769 time 721970613353 state 0x1 ref 1 00000100:00100000:2.0:1789760778.738816:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721970873769 time 721984434016 state 0x0 ref 1 00000100:00100000:2.0:1789760778.738828:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06491180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554757888:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760778.738880:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63737856, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e660 00010000:00010000:2.0:1789760778.738891:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c53b5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.738913:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:2.0:1789760778.738918:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.738925:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c53b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63737856->63741951] (req 63737856->63741951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.738940:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c53b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63737856->63741951] (req 63737856->63741951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.744608:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982870 is committed 00002000:00010000:2.0:1789760778.744804:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63737856 -> 63741952 00002000:00010000:2.0:1789760778.744819:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124512 -> 124520 00010000:00010000:2.0:1789760778.744935:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c53b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63737856->63741951] (req 63737856->63741951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.744951:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c53b5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63737856->63741951] (req 63737856->63741951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.744960:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c53b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63737856->63741951] (req 63737856->63741951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.744968:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760778.744972:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c53b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63737856->63741951] (req 63737856->63741951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.744982:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c53b5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63737856->63741951] (req 63737856->63741951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.745003:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c53b5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63737856->63741951] (req 63737856->63741951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.745012:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c53b5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63737856->63741951] (req 63737856->63741951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.745185:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06491180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554757888:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6353us (6641us total) trans 4294982870 rc 0/0 00000100:00100000:2.0:1789760778.745196:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15572 00000100:00100000:2.0:1789760778.745206:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721970873769 time 721984434016 state 0x2 ref 0 00000100:00100000:1.0:1789760778.750630:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760778.752202:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554758016 00000100:00100000:3.0:1789760778.752216:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554758016 00000100:00100000:1.0:1789760778.752533:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721970873769 time 721984434016 state 0x0 ref 1 00000100:00100000:1.0:1789760778.752550:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721984694432 time 721984434016 state 0x1 ref 1 00000100:00100000:1.0:1789760778.752579:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721984694432 time 721998197385 state 0x0 ref 1 00000100:00100000:1.0:1789760778.752591:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554758016:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760778.752638:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63741952, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e668 00010000:00010000:1.0:1789760778.752647:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c53bc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.752663:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=721s v=(0 0) 00010000:00010000:1.0:1789760778.752673:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.752679:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c53bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63741952->63746047] (req 63741952->63746047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.752778:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c53bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63741952->63746047] (req 63741952->63746047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.759641:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982871 is committed 00002000:00010000:1.0:1789760778.759924:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63741952 -> 63746048 00002000:00010000:1.0:1789760778.759930:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124520 -> 124528 00010000:00010000:1.0:1789760778.759945:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c53bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63741952->63746047] (req 63741952->63746047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.759957:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c53bc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63741952->63746047] (req 63741952->63746047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.759966:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c53bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63741952->63746047] (req 63741952->63746047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.759974:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760778.759979:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c53bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63741952->63746047] (req 63741952->63746047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.759989:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c53bc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63741952->63746047] (req 63741952->63746047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.760014:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c53bc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63741952->63746047] (req 63741952->63746047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.760022:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c53bc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63741952->63746047] (req 63741952->63746047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.760294:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554758016:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7700us (9669us total) trans 4294982871 rc 0/0 00000100:00100000:1.0:1789760778.760307:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15573 00000100:00100000:1.0:1789760778.760316:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721984694432 time 721998197385 state 0x2 ref 0 00000100:00100000:1.0:1789760778.767556:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760778.767889:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554758144 00000100:00100000:0.0:1789760778.767899:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554758144 00000100:00100000:0.0:1789760778.768012:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721984694432 time 721998197385 state 0x0 ref 1 00000100:00100000:0.0:1789760778.768028:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721998457801 time 721998197385 state 0x1 ref 1 00000100:00100000:0.0:1789760778.768065:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721998457801 time 722013683161 state 0x0 ref 1 00000100:00100000:0.0:1789760778.768078:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554758144:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760778.768127:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63746048, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e670 00010000:00010000:0.0:1789760778.768137:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c53c3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.768151:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760778.768155:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.768162:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c53c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63746048->63750143] (req 63746048->63750143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.768174:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c53c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63746048->63750143] (req 63746048->63750143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.774154:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982872 is committed 00002000:00010000:1.0:1789760778.774629:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63746048 -> 63750144 00002000:00010000:1.0:1789760778.774634:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124528 -> 124536 00010000:00010000:1.0:1789760778.774649:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c53c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63746048->63750143] (req 63746048->63750143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.774664:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c53c3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63746048->63750143] (req 63746048->63750143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.774744:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c53c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63746048->63750143] (req 63746048->63750143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.774754:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760778.774760:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c53c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63746048->63750143] (req 63746048->63750143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.774770:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c53c3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63746048->63750143] (req 63746048->63750143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.774791:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c53c3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63746048->63750143] (req 63746048->63750143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.774800:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c53c3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63746048->63750143] (req 63746048->63750143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.775059:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554758144:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6979us (7507us total) trans 4294982872 rc 0/0 00000100:00100000:1.0:1789760778.775068:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15574 00000100:00100000:1.0:1789760778.775077:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721998457801 time 722013683161 state 0x2 ref 0 00000100:00100000:1.0:1789760778.780296:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760778.780442:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554758272 00000100:00100000:0.0:1789760778.780452:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554758272 00000100:00100000:0.0:1789760778.780504:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 721998457801 time 722013683161 state 0x0 ref 1 00000100:00100000:0.0:1789760778.780519:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722013943577 time 722013683161 state 0x1 ref 1 00000100:00100000:0.0:1789760778.780549:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722013943577 time 722026168221 state 0x0 ref 1 00000100:00100000:0.0:1789760778.780562:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554758272:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760778.780609:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63750144, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e678 00010000:00010000:0.0:1789760778.780619:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c53ca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.780633:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760778.780638:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.780644:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c53ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63750144->63754239] (req 63750144->63754239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.781294:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c53ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63750144->63754239] (req 63750144->63754239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.787589:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982873 is committed 00002000:00010000:0.0:1789760778.787957:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63750144 -> 63754240 00002000:00010000:0.0:1789760778.787962:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124536 -> 124544 00010000:00010000:0.0:1789760778.787975:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c53ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63750144->63754239] (req 63750144->63754239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.787989:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c53ca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63750144->63754239] (req 63750144->63754239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.788000:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c53ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63750144->63754239] (req 63750144->63754239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.788008:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760778.788013:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c53ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63750144->63754239] (req 63750144->63754239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.788023:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c53ca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63750144->63754239] (req 63750144->63754239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.788043:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c53ca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63750144->63754239] (req 63750144->63754239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.788052:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c53ca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63750144->63754239] (req 63750144->63754239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.788215:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554758272:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7649us (7921us total) trans 4294982873 rc 0/0 00000100:00100000:0.0:1789760778.788225:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15575 00000100:00100000:0.0:1789760778.788234:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722013943577 time 722026168221 state 0x2 ref 0 00000100:00100000:2.0:1789760778.799512:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760778.799631:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554758400 00000100:00100000:0.0:1789760778.799641:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554758400 00000100:00100000:0.0:1789760778.799696:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722013943577 time 722026168221 state 0x0 ref 1 00000100:00100000:0.0:1789760778.799722:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722026428637 time 722026168221 state 0x1 ref 1 00000100:00100000:0.0:1789760778.799753:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722026428637 time 722045371547 state 0x0 ref 1 00000100:00100000:0.0:1789760778.799765:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554758400:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760778.799809:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63754240, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e680 00010000:00010000:0.0:1789760778.799819:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c53d1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.799836:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760778.799841:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.799847:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c53d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63754240->63758335] (req 63754240->63758335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.799862:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c53d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63754240->63758335] (req 63754240->63758335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760778.807450:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982874 is committed 00002000:00010000:2.0:1789760778.807599:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63754240 -> 63758336 00002000:00010000:2.0:1789760778.807608:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124544 -> 124552 00010000:00010000:2.0:1789760778.807620:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c53d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63754240->63758335] (req 63754240->63758335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.807633:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c53d1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63754240->63758335] (req 63754240->63758335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.807643:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c53d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63754240->63758335] (req 63754240->63758335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.807651:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760778.807656:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c53d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63754240->63758335] (req 63754240->63758335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.807665:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c53d1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63754240->63758335] (req 63754240->63758335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.807683:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c53d1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63754240->63758335] (req 63754240->63758335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.807690:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c53d1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63754240->63758335] (req 63754240->63758335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.807800:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554758400:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8032us (8292us total) trans 4294982874 rc 0/0 00000100:00100000:2.0:1789760778.807820:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15576 00000100:00100000:2.0:1789760778.807829:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722026428637 time 722045371547 state 0x2 ref 0 00000100:00100000:1.0:1789760778.812646:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760778.813305:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554758528 00000100:00100000:2.0:1789760778.813315:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554758528 00000100:00100000:2.0:1789760778.813368:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722026428637 time 722045371547 state 0x0 ref 1 00000100:00100000:2.0:1789760778.813941:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722045631963 time 722045371547 state 0x1 ref 1 00000100:00100000:2.0:1789760778.813968:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722045631963 time 722059586942 state 0x0 ref 1 00000100:00100000:2.0:1789760778.813982:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554758528:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760778.814032:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63758336, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e688 00010000:00010000:2.0:1789760778.814042:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c53d8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.814058:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760778.814063:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.814070:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c53d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63758336->63762431] (req 63758336->63762431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.814083:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c53d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63758336->63762431] (req 63758336->63762431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760778.820766:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982875 is committed 00002000:00010000:1.0:1789760778.822853:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63758336 -> 63762432 00002000:00010000:1.0:1789760778.822863:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124552 -> 124560 00010000:00010000:1.0:1789760778.822877:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c53d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63758336->63762431] (req 63758336->63762431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.822891:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c53d8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63758336->63762431] (req 63758336->63762431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.822899:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c53d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63758336->63762431] (req 63758336->63762431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.822906:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760778.822910:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c53d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63758336->63762431] (req 63758336->63762431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.822924:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c53d8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63758336->63762431] (req 63758336->63762431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.822945:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c53d8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63758336->63762431] (req 63758336->63762431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.822953:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c53d8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63758336->63762431] (req 63758336->63762431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.823589:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554758528:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9604us (10945us total) trans 4294982875 rc 0/0 00000100:00100000:1.0:1789760778.823601:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15577 00000100:00100000:1.0:1789760778.823611:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722045631963 time 722059586942 state 0x2 ref 0 00000100:00100000:1.0:1789760778.832256:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760778.832358:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554758656 00000100:00100000:2.0:1789760778.832369:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554758656 00000100:00100000:2.0:1789760778.832567:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722045631963 time 722059586942 state 0x0 ref 1 00000100:00100000:2.0:1789760778.832583:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722059847358 time 722059586942 state 0x1 ref 1 00000100:00100000:2.0:1789760778.832607:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722059847358 time 722078225722 state 0x0 ref 1 00000100:00100000:2.0:1789760778.832620:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554758656:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760778.832668:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63762432, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e690 00010000:00010000:2.0:1789760778.832677:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c53df lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.832698:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760778.832703:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.832718:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c53df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63762432->63766527] (req 63762432->63766527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.832732:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c53df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63762432->63766527] (req 63762432->63766527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.833702:0:6509:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760778.839717:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982876 is committed 00002000:00010000:2.0:1789760778.839841:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63762432 -> 63766528 00002000:00010000:2.0:1789760778.839846:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124560 -> 124568 00010000:00010000:2.0:1789760778.839858:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c53df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63762432->63766527] (req 63762432->63766527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.839870:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c53df lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63762432->63766527] (req 63762432->63766527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.839883:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c53df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63762432->63766527] (req 63762432->63766527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.839891:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760778.839895:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c53df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63762432->63766527] (req 63762432->63766527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.839905:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c53df lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63762432->63766527] (req 63762432->63766527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.839924:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c53df lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63762432->63766527] (req 63762432->63766527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.839932:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c53df lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63762432->63766527] (req 63762432->63766527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.840027:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554758656:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7403us (7774us total) trans 4294982876 rc 0/0 00000100:00100000:2.0:1789760778.840037:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15578 00000100:00100000:2.0:1789760778.840045:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722059847358 time 722078225722 state 0x2 ref 0 00000100:00100000:2.0:1789760778.844658:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760778.844959:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554758784 00000100:00100000:1.0:1789760778.844970:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554758784 00000100:00100000:1.0:1789760778.845043:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722059847358 time 722078225722 state 0x0 ref 1 00000100:00100000:1.0:1789760778.845059:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722078486138 time 722078225722 state 0x1 ref 1 00000100:00100000:1.0:1789760778.845096:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722078486138 time 722090707036 state 0x0 ref 1 00000100:00100000:1.0:1789760778.845108:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04562d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554758784:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760778.845156:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63766528, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e698 00010000:00010000:1.0:1789760778.845165:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c53e6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.845182:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760778.845186:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.845192:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c53e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63766528->63770623] (req 63766528->63770623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.845206:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c53e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63766528->63770623] (req 63766528->63770623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.854300:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982877 is committed 00002000:00010000:1.0:1789760778.854622:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63766528 -> 63770624 00002000:00010000:1.0:1789760778.854627:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124568 -> 124576 00010000:00010000:1.0:1789760778.854641:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c53e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63766528->63770623] (req 63766528->63770623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.854654:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c53e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63766528->63770623] (req 63766528->63770623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.854664:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c53e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63766528->63770623] (req 63766528->63770623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.854671:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760778.854676:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c53e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63766528->63770623] (req 63766528->63770623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.854690:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c53e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63766528->63770623] (req 63766528->63770623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.854717:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c53e6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63766528->63770623] (req 63766528->63770623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.854725:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c53e6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63766528->63770623] (req 63766528->63770623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.854924:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04562d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554758784:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9813us (10271us total) trans 4294982877 rc 0/0 00000100:00100000:1.0:1789760778.854935:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15579 00000100:00100000:1.0:1789760778.854944:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722078486138 time 722090707036 state 0x2 ref 0 00000100:00100000:1.0:1789760778.862048:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760778.862253:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554758912 00000100:00100000:0.0:1789760778.862263:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554758912 00000100:00100000:0.0:1789760778.862315:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722078486138 time 722090707036 state 0x0 ref 1 00000100:00100000:0.0:1789760778.862330:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722090967452 time 722090707036 state 0x1 ref 1 00000100:00100000:0.0:1789760778.862449:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722090967452 time 722108066562 state 0x0 ref 1 00000100:00100000:0.0:1789760778.862472:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554758912:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760778.862520:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63770624, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e6a0 00010000:00010000:0.0:1789760778.862530:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c53ed lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.862546:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760778.862550:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.862557:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c53ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63770624->63774719] (req 63770624->63774719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.862570:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c53ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63770624->63774719] (req 63770624->63774719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.868864:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982878 is committed 00002000:00010000:0.0:1789760778.872161:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63770624 -> 63774720 00002000:00010000:0.0:1789760778.872172:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124576 -> 124584 00010000:00010000:0.0:1789760778.872186:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c53ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63770624->63774719] (req 63770624->63774719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.872199:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c53ed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63770624->63774719] (req 63770624->63774719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.872209:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c53ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63770624->63774719] (req 63770624->63774719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.872216:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760778.872221:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c53ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63770624->63774719] (req 63770624->63774719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.872230:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c53ed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63770624->63774719] (req 63770624->63774719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.872250:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c53ed lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63770624->63774719] (req 63770624->63774719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.872258:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c53ed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63770624->63774719] (req 63770624->63774719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.872511:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554758912:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10045us (10466us total) trans 4294982878 rc 0/0 00000100:00100000:0.0:1789760778.872523:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15580 00000100:00100000:0.0:1789760778.872533:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722090967452 time 722108066562 state 0x2 ref 0 00000100:00100000:0.0:1789760778.879654:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760778.879756:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554759040 00000100:00100000:2.0:1789760778.879767:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554759040 00000100:00100000:2.0:1789760778.879827:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722090967452 time 722108066562 state 0x0 ref 1 00000100:00100000:2.0:1789760778.879847:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722108326978 time 722108066562 state 0x1 ref 1 00000100:00100000:2.0:1789760778.879881:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722108326978 time 722125499022 state 0x0 ref 1 00000100:00100000:2.0:1789760778.879895:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554759040:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760778.880138:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63774720, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e6a8 00010000:00010000:2.0:1789760778.880161:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c53f4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.880177:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760778.880182:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.880188:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c53f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63774720->63778815] (req 63774720->63778815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.880202:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c53f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63774720->63778815] (req 63774720->63778815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.887431:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982879 is committed 00002000:00010000:2.0:1789760778.887609:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63774720 -> 63778816 00002000:00010000:2.0:1789760778.887614:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124584 -> 124592 00010000:00010000:2.0:1789760778.887628:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c53f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63774720->63778815] (req 63774720->63778815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.887641:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c53f4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63774720->63778815] (req 63774720->63778815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.887649:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c53f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63774720->63778815] (req 63774720->63778815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.887657:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760778.887662:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c53f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63774720->63778815] (req 63774720->63778815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.887671:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c53f4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63774720->63778815] (req 63774720->63778815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.887690:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c53f4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63774720->63778815] (req 63774720->63778815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.887698:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c53f4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63774720->63778815] (req 63774720->63778815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.887865:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554759040:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7968us (8215us total) trans 4294982879 rc 0/0 00000100:00100000:2.0:1789760778.887877:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15581 00000100:00100000:2.0:1789760778.887887:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722108326978 time 722125499022 state 0x2 ref 0 00000100:00100000:2.0:1789760778.894010:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760778.894138:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554759168 00000100:00100000:1.0:1789760778.894149:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554759168 00000100:00100000:1.0:1789760778.894199:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722108326978 time 722125499022 state 0x0 ref 1 00000100:00100000:1.0:1789760778.894210:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722125759438 time 722125499022 state 0x1 ref 1 00000100:00100000:1.0:1789760778.894239:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722125759438 time 722139857303 state 0x0 ref 1 00000100:00100000:1.0:1789760778.894251:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04563800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554759168:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760778.894300:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63778816, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e6b0 00010000:00010000:1.0:1789760778.894308:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c53fb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.894320:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760778.894324:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.894328:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c53fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63778816->63782911] (req 63778816->63782911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.894338:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c53fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63778816->63782911] (req 63778816->63782911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.899775:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982880 is committed 00002000:00010000:2.0:1789760778.900323:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63778816 -> 63782912 00002000:00010000:2.0:1789760778.900329:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124592 -> 124600 00010000:00010000:2.0:1789760778.900343:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c53fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63778816->63782911] (req 63778816->63782911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.900356:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c53fb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63778816->63782911] (req 63778816->63782911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.900365:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c53fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63778816->63782911] (req 63778816->63782911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.900373:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655de00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760778.900524:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c53fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63778816->63782911] (req 63778816->63782911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.900537:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c53fb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63778816->63782911] (req 63778816->63782911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.900557:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c53fb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63778816->63782911] (req 63778816->63782911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.900565:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c53fb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63778816->63782911] (req 63778816->63782911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.900697:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04563800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554759168:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6442us (6694us total) trans 4294982880 rc 0/0 00000100:00100000:2.0:1789760778.900718:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15582 00000100:00100000:2.0:1789760778.900728:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722125759438 time 722139857303 state 0x2 ref 0 00000100:00100000:1.0:1789760778.907013:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760778.908113:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554759296 00000100:00100000:0.0:1789760778.908128:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554759296 00000100:00100000:0.0:1789760778.908192:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722125759438 time 722139857303 state 0x0 ref 1 00000100:00100000:0.0:1789760778.908209:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722140117719 time 722139857303 state 0x1 ref 1 00000100:00100000:0.0:1789760778.908247:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722140117719 time 722153865676 state 0x0 ref 1 00000100:00100000:0.0:1789760778.908260:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554759296:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760778.908311:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63782912, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e6b8 00010000:00010000:0.0:1789760778.908322:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5402 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.908340:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760778.908344:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.908592:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5402 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63782912->63787007] (req 63782912->63787007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.908608:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5402 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63782912->63787007] (req 63782912->63787007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.913717:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982881 is committed 00002000:00010000:0.0:1789760778.913849:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63782912 -> 63787008 00002000:00010000:0.0:1789760778.913855:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124600 -> 124608 00010000:00010000:0.0:1789760778.913867:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5402 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63782912->63787007] (req 63782912->63787007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.913879:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5402 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63782912->63787007] (req 63782912->63787007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.913888:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5402 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63782912->63787007] (req 63782912->63787007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.913895:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760778.913900:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5402 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63782912->63787007] (req 63782912->63787007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.913910:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5402 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63782912->63787007] (req 63782912->63787007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.914002:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5402 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63782912->63787007] (req 63782912->63787007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.914012:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5402 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63782912->63787007] (req 63782912->63787007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.914268:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554759296:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6006us (7262us total) trans 4294982881 rc 0/0 00000100:00100000:0.0:1789760778.914277:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15583 00000100:00100000:0.0:1789760778.914283:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722140117719 time 722153865676 state 0x2 ref 0 00000100:00100000:0.0:1789760778.919328:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760778.919548:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554759424 00000100:00100000:2.0:1789760778.919558:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554759424 00000100:00100000:2.0:1789760778.919611:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722140117719 time 722153865676 state 0x0 ref 1 00000100:00100000:2.0:1789760778.919626:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722154126092 time 722153865676 state 0x1 ref 1 00000100:00100000:2.0:1789760778.919649:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722154126092 time 722165267744 state 0x0 ref 1 00000100:00100000:2.0:1789760778.919661:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554759424:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760778.919713:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63787008, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e6c0 00010000:00010000:2.0:1789760778.919722:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5409 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.919737:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760778.919741:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.919748:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5409 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63787008->63791103] (req 63787008->63791103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.919762:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5409 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63787008->63791103] (req 63787008->63791103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.925736:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982882 is committed 00002000:00010000:2.0:1789760778.925856:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63787008 -> 63791104 00002000:00010000:2.0:1789760778.925861:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124608 -> 124616 00010000:00010000:2.0:1789760778.925874:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5409 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63787008->63791103] (req 63787008->63791103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.925888:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5409 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63787008->63791103] (req 63787008->63791103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.925897:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5409 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63787008->63791103] (req 63787008->63791103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.925909:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760778.925914:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5409 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63787008->63791103] (req 63787008->63791103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.925929:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5409 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63787008->63791103] (req 63787008->63791103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.925949:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5409 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63787008->63791103] (req 63787008->63791103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.925957:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5409 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63787008->63791103] (req 63787008->63791103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.926091:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554759424:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6426us (6767us total) trans 4294982882 rc 0/0 00000100:00100000:2.0:1789760778.926103:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15584 00000100:00100000:2.0:1789760778.926113:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722154126092 time 722165267744 state 0x2 ref 0 00000100:00100000:2.0:1789760778.933489:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760778.933590:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554759552 00000100:00100000:1.0:1789760778.933600:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554759552 00000100:00100000:1.0:1789760778.933653:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722154126092 time 722165267744 state 0x0 ref 1 00000100:00100000:1.0:1789760778.933668:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722165528160 time 722165267744 state 0x1 ref 1 00000100:00100000:1.0:1789760778.933690:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722165528160 time 722179308452 state 0x0 ref 1 00000100:00100000:1.0:1789760778.933701:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554759552:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760778.933758:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63791104, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e6c8 00010000:00010000:1.0:1789760778.933766:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5410 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.933788:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760778.933792:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.933798:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5410 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63791104->63795199] (req 63791104->63795199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.933811:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5410 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63791104->63795199] (req 63791104->63795199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.939841:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982883 is committed 00002000:00010000:2.0:1789760778.940294:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63791104 -> 63795200 00002000:00010000:2.0:1789760778.940299:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124616 -> 124624 00010000:00010000:2.0:1789760778.940314:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5410 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63791104->63795199] (req 63791104->63795199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.940327:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5410 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63791104->63795199] (req 63791104->63795199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.940337:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5410 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63791104->63795199] (req 63791104->63795199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.940344:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760778.940349:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5410 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63791104->63795199] (req 63791104->63795199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.940358:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5410 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63791104->63795199] (req 63791104->63795199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.940404:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5410 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63791104->63795199] (req 63791104->63795199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.940413:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5410 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63791104->63795199] (req 63791104->63795199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.940617:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554759552:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6912us (7132us total) trans 4294982883 rc 0/0 00000100:00100000:2.0:1789760778.940628:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15585 00000100:00100000:2.0:1789760778.940637:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722165528160 time 722179308452 state 0x2 ref 0 00000100:00100000:1.0:1789760778.948669:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760778.948850:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554759680 00000100:00100000:2.0:1789760778.948860:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554759680 00000100:00100000:2.0:1789760778.948912:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722165528160 time 722179308452 state 0x0 ref 1 00000100:00100000:2.0:1789760778.948995:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722179568868 time 722179308452 state 0x1 ref 1 00000100:00100000:2.0:1789760778.949067:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722179568868 time 722194684652 state 0x0 ref 1 00000100:00100000:2.0:1789760778.949080:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554759680:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760778.949150:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63795200, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e6d0 00010000:00010000:2.0:1789760778.949159:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5417 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.949175:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760778.949179:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.949185:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5417 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63795200->63799295] (req 63795200->63799295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.949198:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5417 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63795200->63799295] (req 63795200->63799295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760778.954353:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982884 is committed 00002000:00010000:2.0:1789760778.954671:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63795200 -> 63799296 00002000:00010000:2.0:1789760778.954677:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124624 -> 124632 00010000:00010000:2.0:1789760778.954754:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5417 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63795200->63799295] (req 63795200->63799295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.954768:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5417 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63795200->63799295] (req 63795200->63799295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.954777:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5417 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63795200->63799295] (req 63795200->63799295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.954789:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760778.954795:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5417 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63795200->63799295] (req 63795200->63799295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.954804:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5417 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63795200->63799295] (req 63795200->63799295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.954823:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5417 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63795200->63799295] (req 63795200->63799295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.954830:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5417 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63795200->63799295] (req 63795200->63799295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760778.954924:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554759680:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5840us (6259us total) trans 4294982884 rc 0/0 00000100:00100000:2.0:1789760778.954934:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15586 00000100:00100000:2.0:1789760778.954943:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722179568868 time 722194684652 state 0x2 ref 0 00000100:00100000:0.0:1789760778.959731:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760778.959856:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554759808 00000100:00100000:1.0:1789760778.959864:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554759808 00000100:00100000:1.0:1789760778.959909:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722179568868 time 722194684652 state 0x0 ref 1 00000100:00100000:1.0:1789760778.959921:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722194945068 time 722194684652 state 0x1 ref 1 00000100:00100000:1.0:1789760778.959945:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722194945068 time 722205564066 state 0x0 ref 1 00000100:00100000:1.0:1789760778.959957:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554759808:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760778.960009:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63799296, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e6d8 00010000:00010000:1.0:1789760778.960019:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c541e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.960035:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760778.960040:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.960046:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c541e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63799296->63803391] (req 63799296->63803391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.960058:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c541e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63799296->63803391] (req 63799296->63803391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.965669:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982885 is committed 00002000:00010000:1.0:1789760778.965826:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63799296 -> 63803392 00002000:00010000:1.0:1789760778.965831:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124632 -> 124640 00010000:00010000:1.0:1789760778.965844:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c541e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63799296->63803391] (req 63799296->63803391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.965861:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c541e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63799296->63803391] (req 63799296->63803391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.965871:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c541e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63799296->63803391] (req 63799296->63803391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.965879:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760778.965884:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c541e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63799296->63803391] (req 63799296->63803391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.965893:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c541e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63799296->63803391] (req 63799296->63803391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.965911:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c541e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63799296->63803391] (req 63799296->63803391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.965924:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c541e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63799296->63803391] (req 63799296->63803391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.966040:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554759808:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6080us (6314us total) trans 4294982885 rc 0/0 00000100:00100000:1.0:1789760778.966050:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15587 00000100:00100000:1.0:1789760778.966060:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722194945068 time 722205564066 state 0x2 ref 0 00000100:00100000:1.0:1789760778.971027:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760778.971105:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554759936 00000100:00100000:0.0:1789760778.971115:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554759936 00000100:00100000:0.0:1789760778.971163:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722194945068 time 722205564066 state 0x0 ref 1 00000100:00100000:0.0:1789760778.971177:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722205824482 time 722205564066 state 0x1 ref 1 00000100:00100000:0.0:1789760778.971206:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722205824482 time 722216824701 state 0x0 ref 1 00000100:00100000:0.0:1789760778.971216:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554759936:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760778.971256:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63803392, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e6e0 00010000:00010000:0.0:1789760778.971264:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5425 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.971278:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760778.971282:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760778.971288:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5425 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63803392->63807487] (req 63803392->63807487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.971301:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5425 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63803392->63807487] (req 63803392->63807487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.976989:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982886 is committed 00002000:00010000:0.0:1789760778.977110:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63803392 -> 63807488 00002000:00010000:0.0:1789760778.977115:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124640 -> 124648 00010000:00010000:0.0:1789760778.977128:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5425 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63803392->63807487] (req 63803392->63807487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.977142:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5425 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63803392->63807487] (req 63803392->63807487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.977152:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5425 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63803392->63807487] (req 63803392->63807487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.977160:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760778.977165:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5425 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63803392->63807487] (req 63803392->63807487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.977174:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5425 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63803392->63807487] (req 63803392->63807487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.977191:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5425 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63803392->63807487] (req 63803392->63807487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760778.977199:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5425 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63803392->63807487] (req 63803392->63807487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760778.977328:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554759936:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6108us (6305us total) trans 4294982886 rc 0/0 00000100:00100000:0.0:1789760778.977338:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15588 00000100:00100000:0.0:1789760778.977347:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722205824482 time 722216824701 state 0x2 ref 0 00000100:00100000:2.0:1789760778.982020:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760778.982596:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554760064 00000100:00100000:1.0:1789760778.982606:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554760064 00000100:00100000:1.0:1789760778.982665:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722205824482 time 722216824701 state 0x0 ref 1 00000100:00100000:1.0:1789760778.982681:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722217085117 time 722216824701 state 0x1 ref 1 00000100:00100000:1.0:1789760778.982774:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722217085117 time 722228391256 state 0x0 ref 1 00000100:00100000:1.0:1789760778.982788:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554760064:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760778.982836:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63807488, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e6e8 00010000:00010000:1.0:1789760778.982846:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c542c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.982861:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760778.982866:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760778.982872:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c542c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63807488->63811583] (req 63807488->63811583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.982887:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c542c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63807488->63811583] (req 63807488->63811583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760778.986987:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982887 is committed 00002000:00010000:1.0:1789760778.988983:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63807488 -> 63811584 00002000:00010000:1.0:1789760778.988992:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124648 -> 124656 00010000:00010000:1.0:1789760778.989006:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c542c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63807488->63811583] (req 63807488->63811583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.989019:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c542c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63807488->63811583] (req 63807488->63811583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.989029:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c542c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63807488->63811583] (req 63807488->63811583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.989036:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760778.989041:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c542c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63807488->63811583] (req 63807488->63811583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.989050:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c542c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63807488->63811583] (req 63807488->63811583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.989069:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c542c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63807488->63811583] (req 63807488->63811583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760778.989077:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c542c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63807488->63811583] (req 63807488->63811583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760778.989193:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554760064:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6402us (7177us total) trans 4294982887 rc 0/0 00000100:00100000:1.0:1789760778.989203:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15589 00000100:00100000:1.0:1789760778.989212:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722217085117 time 722228391256 state 0x2 ref 0 00000100:00100000:1.0:1789760778.993197:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760778.996407:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554760192 00000100:00100000:2.0:1789760778.996421:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554760192 00000100:00100000:2.0:1789760778.996481:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722217085117 time 722228391256 state 0x0 ref 1 00000100:00100000:2.0:1789760778.996496:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722228651672 time 722228391256 state 0x1 ref 1 00000100:00100000:2.0:1789760778.996519:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722228651672 time 722242138146 state 0x0 ref 1 00000100:00100000:2.0:1789760778.996534:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554760192:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760778.996580:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63811584, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e6f0 00010000:00010000:2.0:1789760778.996589:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5433 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.996605:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760778.996609:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760778.996615:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5433 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63811584->63815679] (req 63811584->63815679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760778.996629:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5433 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63811584->63815679] (req 63811584->63815679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.006899:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982888 is committed 00002000:00010000:2.0:1789760779.007053:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63811584 -> 63815680 00002000:00010000:2.0:1789760779.007058:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124656 -> 124664 00010000:00010000:2.0:1789760779.007072:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5433 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63811584->63815679] (req 63811584->63815679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.007084:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5433 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63811584->63815679] (req 63811584->63815679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.007093:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5433 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63811584->63815679] (req 63811584->63815679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.007100:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760779.007106:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5433 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63811584->63815679] (req 63811584->63815679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.007115:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5433 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63811584->63815679] (req 63811584->63815679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.007134:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5433 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63811584->63815679] (req 63811584->63815679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.007142:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5433 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63811584->63815679] (req 63811584->63815679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.007240:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554760192:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10705us (14047us total) trans 4294982888 rc 0/0 00000100:00100000:2.0:1789760779.007250:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15590 00000100:00100000:2.0:1789760779.007260:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722228651672 time 722242138146 state 0x2 ref 0 00000100:00100000:2.0:1789760779.010634:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760779.010742:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554760320 00000100:00100000:0.0:1789760779.010752:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554760320 00000100:00100000:0.0:1789760779.010834:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722228651672 time 722242138146 state 0x0 ref 1 00000100:00100000:0.0:1789760779.010850:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722242398562 time 722242138146 state 0x1 ref 1 00000100:00100000:0.0:1789760779.010882:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722242398562 time 722256500503 state 0x0 ref 1 00000100:00100000:0.0:1789760779.010892:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04562a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554760320:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760779.010931:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63815680, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e6f8 00010000:00010000:0.0:1789760779.010938:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c543a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.010951:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760779.010954:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.010959:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c543a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63815680->63819775] (req 63815680->63819775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.010969:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c543a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63815680->63819775] (req 63815680->63819775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760779.011304:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760779.016906:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982889 is committed 00002000:00010000:0.0:1789760779.017638:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63815680 -> 63819776 00002000:00010000:0.0:1789760779.017645:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124664 -> 124672 00010000:00010000:0.0:1789760779.017657:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c543a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63815680->63819775] (req 63815680->63819775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.017679:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c543a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63815680->63819775] (req 63815680->63819775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.017694:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c543a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63815680->63819775] (req 63815680->63819775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.017701:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760779.017714:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c543a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63815680->63819775] (req 63815680->63819775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.017724:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c543a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63815680->63819775] (req 63815680->63819775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.017752:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c543a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63815680->63819775] (req 63815680->63819775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.017759:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c543a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63815680->63819775] (req 63815680->63819775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.017848:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04562a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554760320:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6952us (7218us total) trans 4294982889 rc 0/0 00000100:00100000:0.0:1789760779.017858:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15591 00000100:00100000:0.0:1789760779.017867:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722242398562 time 722256500503 state 0x2 ref 0 00000100:00100000:1.0:1789760779.023166:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760779.024046:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554760448 00000100:00100000:0.0:1789760779.024057:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554760448 00000100:00100000:0.0:1789760779.024109:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722242398562 time 722256500503 state 0x0 ref 1 00000100:00100000:0.0:1789760779.024124:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722256760919 time 722256500503 state 0x1 ref 1 00000100:00100000:0.0:1789760779.024159:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722256760919 time 722269776951 state 0x0 ref 1 00000100:00100000:0.0:1789760779.024173:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554760448:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760779.024744:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63819776, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e700 00010000:00010000:0.0:1789760779.024755:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5441 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.024771:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760779.024776:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.024782:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5441 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63819776->63823871] (req 63819776->63823871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.024801:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5441 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63819776->63823871] (req 63819776->63823871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.033113:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982890 is committed 00002000:00010000:0.0:1789760779.033689:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63819776 -> 63823872 00002000:00010000:0.0:1789760779.033695:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124672 -> 124680 00010000:00010000:0.0:1789760779.033724:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5441 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63819776->63823871] (req 63819776->63823871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.034110:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5441 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63819776->63823871] (req 63819776->63823871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.034120:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5441 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63819776->63823871] (req 63819776->63823871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.034133:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760779.034139:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5441 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63819776->63823871] (req 63819776->63823871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.034147:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5441 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63819776->63823871] (req 63819776->63823871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.034167:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5441 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63819776->63823871] (req 63819776->63823871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.034176:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5441 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63819776->63823871] (req 63819776->63823871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.034343:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554760448:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10168us (11182us total) trans 4294982890 rc 0/0 00000100:00100000:0.0:1789760779.034354:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15592 00000100:00100000:0.0:1789760779.034363:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722256760919 time 722269776951 state 0x2 ref 0 00000100:00100000:0.0:1789760779.039335:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760779.039614:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554760576 00000100:00100000:2.0:1789760779.039623:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554760576 00000100:00100000:2.0:1789760779.039672:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722256760919 time 722269776951 state 0x0 ref 1 00000100:00100000:2.0:1789760779.039685:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722270037367 time 722269776951 state 0x1 ref 1 00000100:00100000:2.0:1789760779.039722:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722270037367 time 722285340610 state 0x0 ref 1 00000100:00100000:2.0:1789760779.039884:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554760576:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760779.039945:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63823872, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e708 00010000:00010000:2.0:1789760779.039953:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5448 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.039968:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760779.039973:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.039979:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5448 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63823872->63827967] (req 63823872->63827967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.039991:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5448 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63823872->63827967] (req 63823872->63827967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.047264:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982891 is committed 00002000:00010000:2.0:1789760779.048679:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63823872 -> 63827968 00002000:00010000:2.0:1789760779.048692:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124680 -> 124688 00010000:00010000:2.0:1789760779.048722:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5448 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63823872->63827967] (req 63823872->63827967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.048735:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5448 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63823872->63827967] (req 63823872->63827967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.048743:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5448 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63823872->63827967] (req 63823872->63827967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.048751:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760779.048756:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5448 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63823872->63827967] (req 63823872->63827967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.048771:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5448 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63823872->63827967] (req 63823872->63827967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.048793:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5448 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63823872->63827967] (req 63823872->63827967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.048801:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5448 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63823872->63827967] (req 63823872->63827967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.049013:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554760576:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9127us (9683us total) trans 4294982891 rc 0/0 00000100:00100000:2.0:1789760779.049025:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15593 00000100:00100000:2.0:1789760779.049033:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722270037367 time 722285340610 state 0x2 ref 0 00000100:00100000:2.0:1789760779.055421:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.055537:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554760704 00000100:00100000:1.0:1789760779.055546:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554760704 00000100:00100000:1.0:1789760779.055603:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722270037367 time 722285340610 state 0x0 ref 1 00000100:00100000:1.0:1789760779.055617:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722285601026 time 722285340610 state 0x1 ref 1 00000100:00100000:1.0:1789760779.055648:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722285601026 time 722301266304 state 0x0 ref 1 00000100:00100000:1.0:1789760779.055660:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04562300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554760704:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760779.055719:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63827968, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e710 00010000:00010000:1.0:1789760779.055736:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c544f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.055751:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760779.055756:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.055762:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c544f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63827968->63832063] (req 63827968->63832063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.055775:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c544f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63827968->63832063] (req 63827968->63832063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.064114:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982892 is committed 00002000:00010000:1.0:1789760779.064399:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63827968 -> 63832064 00002000:00010000:1.0:1789760779.064405:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124688 -> 124696 00010000:00010000:1.0:1789760779.064418:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c544f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63827968->63832063] (req 63827968->63832063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.064431:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c544f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63827968->63832063] (req 63827968->63832063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.064440:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c544f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63827968->63832063] (req 63827968->63832063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.064448:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760779.064453:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c544f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63827968->63832063] (req 63827968->63832063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.064476:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c544f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63827968->63832063] (req 63827968->63832063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.064494:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c544f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63827968->63832063] (req 63827968->63832063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.064506:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c544f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63827968->63832063] (req 63827968->63832063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.065075:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04562300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554760704:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9411us (9690us total) trans 4294982892 rc 0/0 00000100:00100000:1.0:1789760779.065093:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15594 00000100:00100000:1.0:1789760779.065103:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722285601026 time 722301266304 state 0x2 ref 0 00000100:00100000:1.0:1789760779.074259:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760779.074557:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554760832 00000100:00100000:0.0:1789760779.074567:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554760832 00000100:00100000:0.0:1789760779.074687:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722285601026 time 722301266304 state 0x0 ref 1 00000100:00100000:0.0:1789760779.074712:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722301526720 time 722301266304 state 0x1 ref 1 00000100:00100000:0.0:1789760779.074853:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722301526720 time 722320468774 state 0x0 ref 1 00000100:00100000:0.0:1789760779.074868:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554760832:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760779.074924:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63832064, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e718 00010000:00010000:0.0:1789760779.074934:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5456 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.074950:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760779.074954:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.074960:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5456 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63832064->63836159] (req 63832064->63836159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.074974:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5456 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63832064->63836159] (req 63832064->63836159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.080642:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982893 is committed 00002000:00010000:1.0:1789760779.080938:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63832064 -> 63836160 00002000:00010000:1.0:1789760779.080945:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124696 -> 124704 00010000:00010000:1.0:1789760779.080959:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5456 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63832064->63836159] (req 63832064->63836159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.080972:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5456 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63832064->63836159] (req 63832064->63836159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.080980:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5456 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63832064->63836159] (req 63832064->63836159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.080987:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760779.080992:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5456 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63832064->63836159] (req 63832064->63836159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.081000:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5456 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63832064->63836159] (req 63832064->63836159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.081021:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5456 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63832064->63836159] (req 63832064->63836159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.081028:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5456 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63832064->63836159] (req 63832064->63836159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.081130:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554760832:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6260us (6876us total) trans 4294982893 rc 0/0 00000100:00100000:1.0:1789760779.081140:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15595 00000100:00100000:1.0:1789760779.081149:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722301526720 time 722320468774 state 0x2 ref 0 00000100:00100000:0.0:1789760779.086083:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760779.086185:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554760960 00000100:00100000:2.0:1789760779.086195:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554760960 00000100:00100000:2.0:1789760779.086246:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722301526720 time 722320468774 state 0x0 ref 1 00000100:00100000:2.0:1789760779.086261:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722320729190 time 722320468774 state 0x1 ref 1 00000100:00100000:2.0:1789760779.086294:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722320729190 time 722331911621 state 0x0 ref 1 00000100:00100000:2.0:1789760779.086307:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554760960:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760779.086353:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63836160, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e720 00010000:00010000:2.0:1789760779.086363:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c545d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.087023:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760779.087032:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.087040:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c545d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63836160->63840255] (req 63836160->63840255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.087055:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c545d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63836160->63840255] (req 63836160->63840255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.097649:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982894 is committed 00002000:00010000:2.0:1789760779.097973:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63836160 -> 63840256 00002000:00010000:2.0:1789760779.097978:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124704 -> 124712 00010000:00010000:2.0:1789760779.097993:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c545d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63836160->63840255] (req 63836160->63840255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.098008:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c545d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63836160->63840255] (req 63836160->63840255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.098017:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c545d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63836160->63840255] (req 63836160->63840255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.098026:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760779.098031:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c545d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63836160->63840255] (req 63836160->63840255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.098041:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c545d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63836160->63840255] (req 63836160->63840255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.098063:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c545d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63836160->63840255] (req 63836160->63840255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.098071:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c545d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63836160->63840255] (req 63836160->63840255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.098212:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554760960:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11904us (12136us total) trans 4294982894 rc 0/0 00000100:00100000:2.0:1789760779.098224:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15596 00000100:00100000:2.0:1789760779.098234:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722320729190 time 722331911621 state 0x2 ref 0 00000100:00100000:2.0:1789760779.106179:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.106351:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554761088 00000100:00100000:1.0:1789760779.106362:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554761088 00000100:00100000:1.0:1789760779.106460:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722320729190 time 722331911621 state 0x0 ref 1 00000100:00100000:1.0:1789760779.106474:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722332172037 time 722331911621 state 0x1 ref 1 00000100:00100000:1.0:1789760779.106495:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722332172037 time 722352114292 state 0x0 ref 1 00000100:00100000:1.0:1789760779.106507:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04560380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554761088:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760779.106559:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63840256, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e728 00010000:00010000:1.0:1789760779.106568:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5464 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.106584:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760779.106597:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.106605:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5464 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63840256->63844351] (req 63840256->63844351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.106618:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5464 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63840256->63844351] (req 63840256->63844351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.114198:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982895 is committed 00002000:00010000:1.0:1789760779.114316:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63840256 -> 63844352 00002000:00010000:1.0:1789760779.114321:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124712 -> 124720 00010000:00010000:1.0:1789760779.114334:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5464 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63840256->63844351] (req 63840256->63844351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.114346:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5464 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63840256->63844351] (req 63840256->63844351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.114356:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5464 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63840256->63844351] (req 63840256->63844351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.114363:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760779.114368:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5464 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63840256->63844351] (req 63840256->63844351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.114824:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5464 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63840256->63844351] (req 63840256->63844351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.114849:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5464 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63840256->63844351] (req 63840256->63844351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.114863:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5464 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63840256->63844351] (req 63840256->63844351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.115002:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04560380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554761088:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8492us (8829us total) trans 4294982895 rc 0/0 00000100:00100000:1.0:1789760779.115013:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15597 00000100:00100000:1.0:1789760779.115023:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722332172037 time 722352114292 state 0x2 ref 0 00000100:00100000:1.0:1789760779.124014:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760779.125607:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554761216 00000100:00100000:2.0:1789760779.125621:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554761216 00000100:00100000:2.0:1789760779.125679:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722332172037 time 722352114292 state 0x0 ref 1 00000100:00100000:2.0:1789760779.125694:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722352374708 time 722352114292 state 0x1 ref 1 00000100:00100000:2.0:1789760779.125786:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722352374708 time 722371403403 state 0x0 ref 1 00000100:00100000:2.0:1789760779.125800:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554761216:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760779.125849:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63844352, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e730 00010000:00010000:2.0:1789760779.125859:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c546b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.125891:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760779.125896:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.125904:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c546b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63844352->63848447] (req 63844352->63848447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.125919:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c546b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63844352->63848447] (req 63844352->63848447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.130983:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982896 is committed 00002000:00010000:2.0:1789760779.131175:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63844352 -> 63848448 00002000:00010000:2.0:1789760779.131180:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124720 -> 124728 00010000:00010000:2.0:1789760779.131192:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c546b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63844352->63848447] (req 63844352->63848447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.131205:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c546b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63844352->63848447] (req 63844352->63848447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.131214:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c546b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63844352->63848447] (req 63844352->63848447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.131222:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760779.131227:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c546b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63844352->63848447] (req 63844352->63848447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.131236:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c546b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63844352->63848447] (req 63844352->63848447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.131259:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c546b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63844352->63848447] (req 63844352->63848447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.131267:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c546b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63844352->63848447] (req 63844352->63848447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.131434:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554761216:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5631us (7425us total) trans 4294982896 rc 0/0 00000100:00100000:2.0:1789760779.131444:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15598 00000100:00100000:2.0:1789760779.131453:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722352374708 time 722371403403 state 0x2 ref 0 00000100:00100000:0.0:1789760779.137135:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.137447:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554761344 00000100:00100000:1.0:1789760779.137458:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554761344 00000100:00100000:1.0:1789760779.137515:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722352374708 time 722371403403 state 0x0 ref 1 00000100:00100000:1.0:1789760779.137531:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722371663819 time 722371403403 state 0x1 ref 1 00000100:00100000:1.0:1789760779.137563:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722371663819 time 722383181150 state 0x0 ref 1 00000100:00100000:1.0:1789760779.137574:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554761344:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760779.137633:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63848448, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e738 00010000:00010000:1.0:1789760779.137653:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5472 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.137668:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760779.137674:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.137679:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5472 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63848448->63852543] (req 63848448->63852543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.137693:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5472 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63848448->63852543] (req 63848448->63852543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.143175:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982897 is committed 00002000:00010000:1.0:1789760779.143450:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63848448 -> 63852544 00002000:00010000:1.0:1789760779.143456:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124728 -> 124736 00010000:00010000:1.0:1789760779.143470:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5472 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63848448->63852543] (req 63848448->63852543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.143482:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5472 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63848448->63852543] (req 63848448->63852543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.143491:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5472 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63848448->63852543] (req 63848448->63852543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.143499:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760779.143504:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5472 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63848448->63852543] (req 63848448->63852543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.143514:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5472 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63848448->63852543] (req 63848448->63852543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.143532:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5472 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63848448->63852543] (req 63848448->63852543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.143537:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5472 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63848448->63852543] (req 63848448->63852543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.143755:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554761344:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6168us (6617us total) trans 4294982897 rc 0/0 00000100:00100000:1.0:1789760779.143874:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15599 00000100:00100000:1.0:1789760779.143885:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722371663819 time 722383181150 state 0x2 ref 0 00000100:00100000:1.0:1789760779.146810:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760779.147191:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554761472 00000100:00100000:2.0:1789760779.147200:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554761472 00000100:00100000:2.0:1789760779.147253:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722371663819 time 722383181150 state 0x0 ref 1 00000100:00100000:2.0:1789760779.147267:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722383441566 time 722383181150 state 0x1 ref 1 00000100:00100000:2.0:1789760779.147340:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722383441566 time 722392957966 state 0x0 ref 1 00000100:00100000:2.0:1789760779.147351:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554761472:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760779.147426:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63852544, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e740 00010000:00010000:2.0:1789760779.147435:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5479 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.147448:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760779.147452:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.147457:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5479 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63852544->63856639] (req 63852544->63856639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.147467:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5479 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63852544->63856639] (req 63852544->63856639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.151785:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982898 is committed 00002000:00010000:2.0:1789760779.151910:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63852544 -> 63856640 00002000:00010000:2.0:1789760779.151915:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124736 -> 124744 00010000:00010000:2.0:1789760779.151928:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5479 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63852544->63856639] (req 63852544->63856639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.151940:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5479 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63852544->63856639] (req 63852544->63856639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.151949:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5479 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63852544->63856639] (req 63852544->63856639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.151957:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760779.151961:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5479 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63852544->63856639] (req 63852544->63856639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.151972:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5479 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63852544->63856639] (req 63852544->63856639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.152072:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5479 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63852544->63856639] (req 63852544->63856639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.152142:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5479 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63852544->63856639] (req 63852544->63856639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.152234:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554761472:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4879us (5427us total) trans 4294982898 rc 0/0 00000100:00100000:2.0:1789760779.152244:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15600 00000100:00100000:2.0:1789760779.152254:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722383441566 time 722392957966 state 0x2 ref 0 00000100:00100000:2.0:1789760779.158485:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.158607:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554761600 00000100:00100000:1.0:1789760779.158617:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554761600 00000100:00100000:1.0:1789760779.158670:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722383441566 time 722392957966 state 0x0 ref 1 00000100:00100000:1.0:1789760779.158685:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722393218382 time 722392957966 state 0x1 ref 1 00000100:00100000:1.0:1789760779.158836:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722393218382 time 722404453058 state 0x0 ref 1 00000100:00100000:1.0:1789760779.158850:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04563b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554761600:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760779.158894:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63856640, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e748 00010000:00010000:1.0:1789760779.158904:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5480 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.158919:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760779.158923:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.158929:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5480 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63856640->63860735] (req 63856640->63860735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.158943:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5480 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63856640->63860735] (req 63856640->63860735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.163615:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982899 is committed 00002000:00010000:2.0:1789760779.163724:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63856640 -> 63860736 00002000:00010000:2.0:1789760779.163737:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124744 -> 124752 00010000:00010000:2.0:1789760779.163758:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5480 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63856640->63860735] (req 63856640->63860735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.163771:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5480 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63856640->63860735] (req 63856640->63860735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.163780:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5480 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63856640->63860735] (req 63856640->63860735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.163788:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760779.163792:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5480 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63856640->63860735] (req 63856640->63860735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.163801:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5480 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63856640->63860735] (req 63856640->63860735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.163821:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5480 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63856640->63860735] (req 63856640->63860735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.163839:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5480 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63856640->63860735] (req 63856640->63860735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.163944:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04563b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554761600:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5093us (5464us total) trans 4294982899 rc 0/0 00000100:00100000:2.0:1789760779.163954:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15601 00000100:00100000:2.0:1789760779.163963:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722393218382 time 722404453058 state 0x2 ref 0 00000100:00100000:0.0:1789760779.169773:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.169963:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554761728 00000100:00100000:1.0:1789760779.169972:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554761728 00000100:00100000:1.0:1789760779.170275:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722393218382 time 722404453058 state 0x0 ref 1 00000100:00100000:1.0:1789760779.170291:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722404713474 time 722404453058 state 0x1 ref 1 00000100:00100000:1.0:1789760779.170364:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722404713474 time 722415981816 state 0x0 ref 1 00000100:00100000:1.0:1789760779.170412:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554761728:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760779.170461:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63860736, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e750 00010000:00010000:1.0:1789760779.170471:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5487 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.170486:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760779.170491:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.170496:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5487 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63860736->63864831] (req 63860736->63864831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.170509:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5487 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63860736->63864831] (req 63860736->63864831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.175039:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982900 is committed 00002000:00010000:1.0:1789760779.175671:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63860736 -> 63864832 00002000:00010000:1.0:1789760779.175677:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124752 -> 124760 00010000:00010000:1.0:1789760779.175690:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5487 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63860736->63864831] (req 63860736->63864831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.175717:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5487 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63860736->63864831] (req 63860736->63864831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.175725:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5487 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63860736->63864831] (req 63860736->63864831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.175732:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760779.175737:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5487 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63860736->63864831] (req 63860736->63864831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.175746:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5487 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63860736->63864831] (req 63860736->63864831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.175765:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5487 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63860736->63864831] (req 63860736->63864831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.175773:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5487 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63860736->63864831] (req 63860736->63864831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.175869:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554761728:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5489us (6100us total) trans 4294982900 rc 0/0 00000100:00100000:1.0:1789760779.175879:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15602 00000100:00100000:1.0:1789760779.175889:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722404713474 time 722415981816 state 0x2 ref 0 00000100:00100000:1.0:1789760779.182519:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760779.182579:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554761856 00000100:00100000:0.0:1789760779.182589:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554761856 00000100:00100000:0.0:1789760779.182643:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722404713474 time 722415981816 state 0x0 ref 1 00000100:00100000:0.0:1789760779.182660:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722416242232 time 722415981816 state 0x1 ref 1 00000100:00100000:0.0:1789760779.182929:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722416242232 time 722428546343 state 0x0 ref 1 00000100:00100000:0.0:1789760779.182943:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554761856:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760779.182988:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63864832, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e758 00010000:00010000:0.0:1789760779.182997:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c548e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.183023:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760779.183027:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.183033:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c548e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63864832->63868927] (req 63864832->63868927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.183046:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c548e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63864832->63868927] (req 63864832->63868927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.191431:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982901 is committed 00002000:00010000:1.0:1789760779.191542:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63864832 -> 63868928 00002000:00010000:1.0:1789760779.191547:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124760 -> 124768 00010000:00010000:1.0:1789760779.191567:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c548e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63864832->63868927] (req 63864832->63868927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.191580:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c548e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63864832->63868927] (req 63864832->63868927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.191588:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c548e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63864832->63868927] (req 63864832->63868927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.191595:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760779.191599:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c548e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63864832->63868927] (req 63864832->63868927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.191607:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c548e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63864832->63868927] (req 63864832->63868927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.191626:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c548e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63864832->63868927] (req 63864832->63868927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.191634:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c548e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63864832->63868927] (req 63864832->63868927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.191738:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554761856:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8793us (9221us total) trans 4294982901 rc 0/0 00000100:00100000:1.0:1789760779.191748:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15603 00000100:00100000:1.0:1789760779.191758:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722416242232 time 722428546343 state 0x2 ref 0 00000100:00100000:2.0:1789760779.195527:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760779.195576:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554761984 00000100:00100000:0.0:1789760779.195586:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554761984 00000100:00100000:0.0:1789760779.195640:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722416242232 time 722428546343 state 0x0 ref 1 00000100:00100000:0.0:1789760779.195655:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722428806759 time 722428546343 state 0x1 ref 1 00000100:00100000:0.0:1789760779.195688:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722428806759 time 722441306299 state 0x0 ref 1 00000100:00100000:0.0:1789760779.195721:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554761984:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760779.195777:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63868928, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e760 00010000:00010000:0.0:1789760779.195785:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5495 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.195801:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760779.195805:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.195812:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5495 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63868928->63873023] (req 63868928->63873023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.195825:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5495 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63868928->63873023] (req 63868928->63873023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.201714:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982902 is committed 00002000:00010000:0.0:1789760779.201960:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63868928 -> 63873024 00002000:00010000:0.0:1789760779.201966:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124768 -> 124776 00010000:00010000:0.0:1789760779.201979:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5495 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63868928->63873023] (req 63868928->63873023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.201993:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5495 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63868928->63873023] (req 63868928->63873023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.202003:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5495 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63868928->63873023] (req 63868928->63873023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.202086:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760779.202093:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5495 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63868928->63873023] (req 63868928->63873023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.202103:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5495 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63868928->63873023] (req 63868928->63873023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.202124:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5495 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63868928->63873023] (req 63868928->63873023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.202132:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5495 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63868928->63873023] (req 63868928->63873023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.202227:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554761984:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6504us (6701us total) trans 4294982902 rc 0/0 00000100:00100000:0.0:1789760779.202238:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15604 00000100:00100000:0.0:1789760779.202247:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722428806759 time 722441306299 state 0x2 ref 0 00000100:00100000:0.0:1789760779.209917:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760779.210819:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554762112 00000100:00100000:2.0:1789760779.210829:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554762112 00000100:00100000:2.0:1789760779.210944:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722428806759 time 722441306299 state 0x0 ref 1 00000100:00100000:2.0:1789760779.210960:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722441566715 time 722441306299 state 0x1 ref 1 00000100:00100000:2.0:1789760779.210992:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722441566715 time 722456610520 state 0x0 ref 1 00000100:00100000:2.0:1789760779.211003:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554762112:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760779.211221:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63873024, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e768 00010000:00010000:2.0:1789760779.211231:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c549c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.211252:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760779.211257:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.211263:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c549c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63873024->63877119] (req 63873024->63877119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.211275:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c549c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63873024->63877119] (req 63873024->63877119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.219220:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982903 is committed 00002000:00010000:2.0:1789760779.219358:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63873024 -> 63877120 00002000:00010000:2.0:1789760779.219363:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124776 -> 124784 00010000:00010000:2.0:1789760779.219544:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c549c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63873024->63877119] (req 63873024->63877119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.219558:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c549c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63873024->63877119] (req 63873024->63877119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.219567:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c549c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63873024->63877119] (req 63873024->63877119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.219574:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760779.219579:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c549c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63873024->63877119] (req 63873024->63877119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.219588:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c549c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63873024->63877119] (req 63873024->63877119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.219612:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c549c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63873024->63877119] (req 63873024->63877119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.219619:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c549c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63873024->63877119] (req 63873024->63877119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.219990:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554762112:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8982us (10076us total) trans 4294982903 rc 0/0 00000100:00100000:2.0:1789760779.220001:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15605 00000100:00100000:2.0:1789760779.220011:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722441566715 time 722456610520 state 0x2 ref 0 00000100:00100000:1.0:1789760779.229233:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760779.229424:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554762240 00000100:00100000:2.0:1789760779.229434:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554762240 00000100:00100000:2.0:1789760779.229497:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722441566715 time 722456610520 state 0x0 ref 1 00000100:00100000:2.0:1789760779.229513:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722456870936 time 722456610520 state 0x1 ref 1 00000100:00100000:2.0:1789760779.229535:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722456870936 time 722475153555 state 0x0 ref 1 00000100:00100000:2.0:1789760779.229552:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554762240:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760779.229596:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63877120, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e770 00010000:00010000:2.0:1789760779.229605:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54a3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.229620:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760779.229624:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.229630:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63877120->63881215] (req 63877120->63881215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.229644:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63877120->63881215] (req 63877120->63881215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.237012:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982904 is committed 00002000:00010000:2.0:1789760779.237171:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63877120 -> 63881216 00002000:00010000:2.0:1789760779.237176:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124784 -> 124792 00010000:00010000:2.0:1789760779.237190:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63877120->63881215] (req 63877120->63881215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.237203:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63877120->63881215] (req 63877120->63881215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.237213:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63877120->63881215] (req 63877120->63881215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.237221:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760779.237225:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63877120->63881215] (req 63877120->63881215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.237234:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54a3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63877120->63881215] (req 63877120->63881215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.237255:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54a3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63877120->63881215] (req 63877120->63881215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.237267:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54a3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63877120->63881215] (req 63877120->63881215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.237422:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554762240:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7871us (8193us total) trans 4294982904 rc 0/0 00000100:00100000:2.0:1789760779.237433:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15606 00000100:00100000:2.0:1789760779.237442:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722456870936 time 722475153555 state 0x2 ref 0 00000100:00100000:1.0:1789760779.248341:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760779.248449:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554762368 00000100:00100000:0.0:1789760779.248459:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554762368 00000100:00100000:0.0:1789760779.248573:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722456870936 time 722475153555 state 0x0 ref 1 00000100:00100000:0.0:1789760779.248588:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722475413971 time 722475153555 state 0x1 ref 1 00000100:00100000:0.0:1789760779.248668:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722475413971 time 722494285688 state 0x0 ref 1 00000100:00100000:0.0:1789760779.248681:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554762368:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760779.248739:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63881216, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e778 00010000:00010000:0.0:1789760779.248749:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54aa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.248764:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760779.248769:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.248775:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63881216->63885311] (req 63881216->63885311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.248789:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63881216->63885311] (req 63881216->63885311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.255714:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982905 is committed 00002000:00010000:0.0:1789760779.256011:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63881216 -> 63885312 00002000:00010000:0.0:1789760779.256016:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124792 -> 124800 00010000:00010000:0.0:1789760779.256029:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63881216->63885311] (req 63881216->63885311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.256042:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54aa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63881216->63885311] (req 63881216->63885311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.256052:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63881216->63885311] (req 63881216->63885311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.256061:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760779.256066:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63881216->63885311] (req 63881216->63885311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.256137:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54aa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63881216->63885311] (req 63881216->63885311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.256158:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54aa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63881216->63885311] (req 63881216->63885311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.256167:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54aa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63881216->63885311] (req 63881216->63885311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.256342:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554762368:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7656us (8004us total) trans 4294982905 rc 0/0 00000100:00100000:0.0:1789760779.256353:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15607 00000100:00100000:0.0:1789760779.256363:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722475413971 time 722494285688 state 0x2 ref 0 00000100:00100000:0.0:1789760779.261087:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760779.261161:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554762496 00000100:00100000:2.0:1789760779.261171:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554762496 00000100:00100000:2.0:1789760779.261234:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722475413971 time 722494285688 state 0x0 ref 1 00000100:00100000:2.0:1789760779.261249:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722494546104 time 722494285688 state 0x1 ref 1 00000100:00100000:2.0:1789760779.261282:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722494546104 time 722506900227 state 0x0 ref 1 00000100:00100000:2.0:1789760779.261295:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554762496:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760779.261342:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63885312, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e780 00010000:00010000:2.0:1789760779.261351:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54b1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.261367:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760779.261371:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.261403:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63885312->63889407] (req 63885312->63889407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.261422:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63885312->63889407] (req 63885312->63889407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.266593:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982906 is committed 00002000:00010000:2.0:1789760779.268836:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63885312 -> 63889408 00002000:00010000:2.0:1789760779.268846:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124800 -> 124808 00010000:00010000:2.0:1789760779.268860:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63885312->63889407] (req 63885312->63889407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.268872:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54b1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63885312->63889407] (req 63885312->63889407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.268882:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63885312->63889407] (req 63885312->63889407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.268889:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760779.268893:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63885312->63889407] (req 63885312->63889407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.268901:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54b1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63885312->63889407] (req 63885312->63889407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.268920:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54b1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63885312->63889407] (req 63885312->63889407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.268927:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54b1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63885312->63889407] (req 63885312->63889407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.269066:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554762496:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7768us (7994us total) trans 4294982906 rc 0/0 00000100:00100000:2.0:1789760779.269087:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15608 00000100:00100000:2.0:1789760779.269098:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722494546104 time 722506900227 state 0x2 ref 0 00000100:00100000:1.0:1789760779.275183:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760779.275300:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554762624 00000100:00100000:2.0:1789760779.275309:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554762624 00000100:00100000:2.0:1789760779.275361:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722494546104 time 722506900227 state 0x0 ref 1 00000100:00100000:2.0:1789760779.275405:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722507160643 time 722506900227 state 0x1 ref 1 00000100:00100000:2.0:1789760779.275428:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722507160643 time 722521046212 state 0x0 ref 1 00000100:00100000:2.0:1789760779.275439:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554762624:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760779.275491:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63889408, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e788 00010000:00010000:2.0:1789760779.275500:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54b8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.275515:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760779.275519:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.275526:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63889408->63893503] (req 63889408->63893503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.275539:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63889408->63893503] (req 63889408->63893503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.281156:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982907 is committed 00002000:00010000:2.0:1789760779.281622:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63889408 -> 63893504 00002000:00010000:2.0:1789760779.281628:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124808 -> 124816 00010000:00010000:2.0:1789760779.281641:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63889408->63893503] (req 63889408->63893503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.281655:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54b8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63889408->63893503] (req 63889408->63893503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.281665:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63889408->63893503] (req 63889408->63893503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.281673:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760779.281678:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63889408->63893503] (req 63889408->63893503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.281688:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63889408->63893503] (req 63889408->63893503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.281721:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54b8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63889408->63893503] (req 63889408->63893503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.281800:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c54b8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63889408->63893503] (req 63889408->63893503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.281944:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554762624:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6501us (6766us total) trans 4294982907 rc 0/0 00000100:00100000:2.0:1789760779.281955:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15609 00000100:00100000:2.0:1789760779.281964:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722507160643 time 722521046212 state 0x2 ref 0 00000100:00100000:1.0:1789760779.287030:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760779.287181:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554762752 00000100:00100000:2.0:1789760779.287191:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554762752 00000100:00100000:2.0:1789760779.287327:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722507160643 time 722521046212 state 0x0 ref 1 00000100:00100000:2.0:1789760779.287344:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722521306628 time 722521046212 state 0x1 ref 1 00000100:00100000:2.0:1789760779.287457:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722521306628 time 722533074742 state 0x0 ref 1 00000100:00100000:2.0:1789760779.287480:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554762752:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760779.287529:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63893504, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e790 00010000:00010000:2.0:1789760779.287539:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c54bf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.287554:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760779.287559:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.287565:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c54bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63893504->63897599] (req 63893504->63897599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.287580:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c54bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63893504->63897599] (req 63893504->63897599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.296312:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982908 is committed 00002000:00010000:2.0:1789760779.296432:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63893504 -> 63897600 00002000:00010000:2.0:1789760779.296437:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124816 -> 124824 00010000:00010000:2.0:1789760779.296450:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c54bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63893504->63897599] (req 63893504->63897599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.296474:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c54bf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63893504->63897599] (req 63893504->63897599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.296483:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c54bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63893504->63897599] (req 63893504->63897599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.296490:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760779.296499:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c54bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63893504->63897599] (req 63893504->63897599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.296508:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c54bf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63893504->63897599] (req 63893504->63897599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.296528:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c54bf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63893504->63897599] (req 63893504->63897599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.296536:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c54bf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63893504->63897599] (req 63893504->63897599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.296634:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554762752:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9152us (9610us total) trans 4294982908 rc 0/0 00000100:00100000:2.0:1789760779.296645:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15610 00000100:00100000:2.0:1789760779.296655:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722521306628 time 722533074742 state 0x2 ref 0 00000100:00100000:0.0:1789760779.301797:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.301955:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554762880 00000100:00100000:1.0:1789760779.301964:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554762880 00000100:00100000:1.0:1789760779.302018:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722521306628 time 722533074742 state 0x0 ref 1 00000100:00100000:1.0:1789760779.302033:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722533335158 time 722533074742 state 0x1 ref 1 00000100:00100000:1.0:1789760779.302059:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722533335158 time 722547677613 state 0x0 ref 1 00000100:00100000:1.0:1789760779.302071:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554762880:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760779.302126:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63897600, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e798 00010000:00010000:1.0:1789760779.302134:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c54c6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.302149:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760779.302153:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.302159:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c54c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63897600->63901695] (req 63897600->63901695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.302175:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c54c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63897600->63901695] (req 63897600->63901695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.308875:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982909 is committed 00002000:00010000:1.0:1789760779.308992:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63897600 -> 63901696 00002000:00010000:1.0:1789760779.308997:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124824 -> 124832 00010000:00010000:1.0:1789760779.309010:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c54c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63897600->63901695] (req 63897600->63901695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.309022:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c54c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63897600->63901695] (req 63897600->63901695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.309031:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c54c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63897600->63901695] (req 63897600->63901695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.309038:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655de00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760779.309043:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c54c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63897600->63901695] (req 63897600->63901695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.309052:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c54c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63897600->63901695] (req 63897600->63901695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.309070:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c54c6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63897600->63901695] (req 63897600->63901695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.309089:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c54c6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63897600->63901695] (req 63897600->63901695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.309189:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554762880:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7115us (7396us total) trans 4294982909 rc 0/0 00000100:00100000:1.0:1789760779.309200:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15611 00000100:00100000:1.0:1789760779.309210:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722533335158 time 722547677613 state 0x2 ref 0 00000100:00100000:1.0:1789760779.315826:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760779.315919:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554763008 00000100:00100000:0.0:1789760779.315927:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554763008 00000100:00100000:0.0:1789760779.315981:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722533335158 time 722547677613 state 0x0 ref 1 00000100:00100000:0.0:1789760779.315996:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722547938029 time 722547677613 state 0x1 ref 1 00000100:00100000:0.0:1789760779.316026:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722547938029 time 722561644670 state 0x0 ref 1 00000100:00100000:0.0:1789760779.316038:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554763008:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760779.316157:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63901696, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e7a0 00010000:00010000:0.0:1789760779.316167:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54cd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.316183:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760779.316188:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.316195:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63901696->63905791] (req 63901696->63905791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.316210:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63901696->63905791] (req 63901696->63905791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.322605:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982910 is committed 00002000:00010000:0.0:1789760779.322858:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63901696 -> 63905792 00002000:00010000:0.0:1789760779.322866:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124832 -> 124840 00010000:00010000:0.0:1789760779.322883:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63901696->63905791] (req 63901696->63905791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.322895:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63901696->63905791] (req 63901696->63905791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.322907:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63901696->63905791] (req 63901696->63905791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.322914:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760779.322922:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63901696->63905791] (req 63901696->63905791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.322942:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54cd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63901696->63905791] (req 63901696->63905791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.322966:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54cd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63901696->63905791] (req 63901696->63905791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.322974:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54cd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63901696->63905791] (req 63901696->63905791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.323563:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554763008:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7088us (7308us total) trans 4294982910 rc 0/0 00000100:00100000:0.0:1789760779.323577:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15612 00000100:00100000:0.0:1789760779.323587:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722547938029 time 722561644670 state 0x2 ref 0 00000100:00100000:2.0:1789760779.328680:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.328853:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554763136 00000100:00100000:1.0:1789760779.328862:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554763136 00000100:00100000:1.0:1789760779.328917:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722547938029 time 722561644670 state 0x0 ref 1 00000100:00100000:1.0:1789760779.328932:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722561905086 time 722561644670 state 0x1 ref 1 00000100:00100000:1.0:1789760779.329004:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722561905086 time 722574621474 state 0x0 ref 1 00000100:00100000:1.0:1789760779.329017:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554763136:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760779.329062:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63905792, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e7a8 00010000:00010000:1.0:1789760779.329071:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c54d4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.329097:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760779.329101:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.329107:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c54d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63905792->63909887] (req 63905792->63909887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.329121:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c54d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63905792->63909887] (req 63905792->63909887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.334887:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982911 is committed 00002000:00010000:1.0:1789760779.335189:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63905792 -> 63909888 00002000:00010000:1.0:1789760779.335195:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124840 -> 124848 00010000:00010000:1.0:1789760779.335209:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c54d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63905792->63909887] (req 63905792->63909887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.335222:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c54d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63905792->63909887] (req 63905792->63909887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.335231:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c54d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63905792->63909887] (req 63905792->63909887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.335237:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760779.335241:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c54d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63905792->63909887] (req 63905792->63909887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.335251:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c54d4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63905792->63909887] (req 63905792->63909887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.335270:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c54d4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63905792->63909887] (req 63905792->63909887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.335278:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c54d4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63905792->63909887] (req 63905792->63909887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.335477:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554763136:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6457us (6800us total) trans 4294982911 rc 0/0 00000100:00100000:1.0:1789760779.335489:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15613 00000100:00100000:1.0:1789760779.335498:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722561905086 time 722574621474 state 0x2 ref 0 00000100:00100000:1.0:1789760779.342244:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760779.342352:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554763264 00000100:00100000:0.0:1789760779.342362:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554763264 00000100:00100000:0.0:1789760779.342544:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722561905086 time 722574621474 state 0x0 ref 1 00000100:00100000:0.0:1789760779.342559:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722574881890 time 722574621474 state 0x1 ref 1 00000100:00100000:0.0:1789760779.342630:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722574881890 time 722588247795 state 0x0 ref 1 00000100:00100000:0.0:1789760779.342644:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554763264:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760779.342691:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63909888, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e7b0 00010000:00010000:0.0:1789760779.342712:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54db lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.342838:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760779.342844:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.342850:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63909888->63913983] (req 63909888->63913983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.342865:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63909888->63913983] (req 63909888->63913983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.352275:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982912 is committed 00002000:00010000:0.0:1789760779.352594:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63909888 -> 63913984 00002000:00010000:0.0:1789760779.352600:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124848 -> 124856 00010000:00010000:0.0:1789760779.352918:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63909888->63913983] (req 63909888->63913983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.352937:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63909888->63913983] (req 63909888->63913983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.352946:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63909888->63913983] (req 63909888->63913983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.352954:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760779.352959:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63909888->63913983] (req 63909888->63913983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.352969:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54db lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63909888->63913983] (req 63909888->63913983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.352994:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54db lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63909888->63913983] (req 63909888->63913983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.353002:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54db lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63909888->63913983] (req 63909888->63913983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.353161:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554763264:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10514us (10921us total) trans 4294982912 rc 0/0 00000100:00100000:0.0:1789760779.353172:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15614 00000100:00100000:0.0:1789760779.353181:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722574881890 time 722588247795 state 0x2 ref 0 00000100:00100000:2.0:1789760779.362573:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760779.364584:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554763392 00000100:00100000:0.0:1789760779.364598:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554763392 00000100:00100000:0.0:1789760779.364667:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722574881890 time 722588247795 state 0x0 ref 1 00000100:00100000:0.0:1789760779.364684:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722588508211 time 722588247795 state 0x1 ref 1 00000100:00100000:0.0:1789760779.364720:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722588508211 time 722610337897 state 0x0 ref 1 00000100:00100000:0.0:1789760779.364734:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554763392:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760779.364781:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63913984, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e7b8 00010000:00010000:0.0:1789760779.364790:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54e2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.364806:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760779.364811:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.364818:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63913984->63918079] (req 63913984->63918079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.364833:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63913984->63918079] (req 63913984->63918079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.372085:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982913 is committed 00002000:00010000:0.0:1789760779.372211:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63913984 -> 63918080 00002000:00010000:0.0:1789760779.372217:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124856 -> 124864 00010000:00010000:0.0:1789760779.372230:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63913984->63918079] (req 63913984->63918079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.372244:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63913984->63918079] (req 63913984->63918079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.372255:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63913984->63918079] (req 63913984->63918079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.372263:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760779.372268:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63913984->63918079] (req 63913984->63918079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.372279:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63913984->63918079] (req 63913984->63918079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.372298:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54e2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63913984->63918079] (req 63913984->63918079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.372313:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54e2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63913984->63918079] (req 63913984->63918079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.372633:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554763392:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7897us (10068us total) trans 4294982913 rc 0/0 00000100:00100000:0.0:1789760779.372644:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15615 00000100:00100000:0.0:1789760779.372653:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722588508211 time 722610337897 state 0x2 ref 0 00000100:00100000:2.0:1789760779.381013:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.381106:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554763520 00000100:00100000:1.0:1789760779.381116:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554763520 00000100:00100000:1.0:1789760779.381169:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722588508211 time 722610337897 state 0x0 ref 1 00000100:00100000:1.0:1789760779.381185:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722610598313 time 722610337897 state 0x1 ref 1 00000100:00100000:1.0:1789760779.381218:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722610598313 time 722626836473 state 0x0 ref 1 00000100:00100000:1.0:1789760779.381229:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554763520:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760779.381280:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63918080, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e7c0 00010000:00010000:1.0:1789760779.381289:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c54e9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.381304:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760779.381308:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.381314:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c54e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63918080->63922175] (req 63918080->63922175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.381327:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c54e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63918080->63922175] (req 63918080->63922175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.389245:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982914 is committed 00002000:00010000:1.0:1789760779.389797:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63918080 -> 63922176 00002000:00010000:1.0:1789760779.389802:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124864 -> 124872 00010000:00010000:1.0:1789760779.389814:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c54e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63918080->63922175] (req 63918080->63922175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.389826:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c54e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63918080->63922175] (req 63918080->63922175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.389848:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c54e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63918080->63922175] (req 63918080->63922175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.389856:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760779.389861:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c54e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63918080->63922175] (req 63918080->63922175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.389870:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c54e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63918080->63922175] (req 63918080->63922175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.389890:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c54e9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63918080->63922175] (req 63918080->63922175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.389898:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c54e9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63918080->63922175] (req 63918080->63922175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.389993:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554763520:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8761us (8984us total) trans 4294982914 rc 0/0 00000100:00100000:1.0:1789760779.390003:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15616 00000100:00100000:1.0:1789760779.390013:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722610598313 time 722626836473 state 0x2 ref 0 00000100:00100000:1.0:1789760779.395943:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760779.399961:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554763648 00000100:00100000:2.0:1789760779.399974:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554763648 00000100:00100000:2.0:1789760779.400037:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722610598313 time 722626836473 state 0x0 ref 1 00000100:00100000:2.0:1789760779.400051:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722627096889 time 722626836473 state 0x1 ref 1 00000100:00100000:2.0:1789760779.400074:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722627096889 time 722645692537 state 0x0 ref 1 00000100:00100000:2.0:1789760779.400087:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554763648:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760779.400144:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63922176, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e7c8 00010000:00010000:2.0:1789760779.400153:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c54f0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.400169:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760779.400174:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.400180:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c54f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63922176->63926271] (req 63922176->63926271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.400194:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c54f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63922176->63926271] (req 63922176->63926271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.409315:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982915 is committed 00002000:00010000:2.0:1789760779.410262:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63922176 -> 63926272 00002000:00010000:2.0:1789760779.410268:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124872 -> 124880 00010000:00010000:2.0:1789760779.410283:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c54f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63922176->63926271] (req 63922176->63926271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.410295:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c54f0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63922176->63926271] (req 63922176->63926271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.410303:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c54f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63922176->63926271] (req 63922176->63926271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.410311:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760779.410316:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c54f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63922176->63926271] (req 63922176->63926271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.410325:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c54f0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63922176->63926271] (req 63922176->63926271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.410343:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c54f0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63922176->63926271] (req 63922176->63926271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.410350:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c54f0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63922176->63926271] (req 63922176->63926271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.410576:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554763648:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10486us (14637us total) trans 4294982915 rc 0/0 00000100:00100000:2.0:1789760779.410814:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15617 00000100:00100000:2.0:1789760779.410825:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722627096889 time 722645692537 state 0x2 ref 0 00000100:00100000:2.0:1789760779.419618:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760779.419715:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554763776 00000100:00100000:0.0:1789760779.419732:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554763776 00000100:00100000:0.0:1789760779.419790:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722627096889 time 722645692537 state 0x0 ref 1 00000100:00100000:0.0:1789760779.419806:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722645952953 time 722645692537 state 0x1 ref 1 00000100:00100000:0.0:1789760779.419848:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722645952953 time 722665466085 state 0x0 ref 1 00000100:00100000:0.0:1789760779.419861:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554763776:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760779.419922:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63926272, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e7d0 00010000:00010000:0.0:1789760779.419933:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54f7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.419950:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760779.419954:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.419961:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63926272->63930367] (req 63926272->63930367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.419982:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63926272->63930367] (req 63926272->63930367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.428321:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982916 is committed 00002000:00010000:0.0:1789760779.428629:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63926272 -> 63930368 00002000:00010000:0.0:1789760779.428635:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124880 -> 124888 00010000:00010000:0.0:1789760779.428648:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63926272->63930367] (req 63926272->63930367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.428661:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54f7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63926272->63930367] (req 63926272->63930367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.428670:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63926272->63930367] (req 63926272->63930367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.428678:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760779.428682:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63926272->63930367] (req 63926272->63930367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.428691:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54f7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63926272->63930367] (req 63926272->63930367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.428720:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54f7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63926272->63930367] (req 63926272->63930367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.428728:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54f7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63926272->63930367] (req 63926272->63930367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.428964:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554763776:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9100us (9350us total) trans 4294982916 rc 0/0 00000100:00100000:0.0:1789760779.428976:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15618 00000100:00100000:0.0:1789760779.428986:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722645952953 time 722665466085 state 0x2 ref 0 00000100:00100000:1.0:1789760779.436213:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760779.436335:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554763904 00000100:00100000:0.0:1789760779.436347:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554763904 00000100:00100000:0.0:1789760779.436439:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722645952953 time 722665466085 state 0x0 ref 1 00000100:00100000:0.0:1789760779.436455:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722665726501 time 722665466085 state 0x1 ref 1 00000100:00100000:0.0:1789760779.436494:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722665726501 time 722682111691 state 0x0 ref 1 00000100:00100000:0.0:1789760779.436506:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554763904:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760779.436551:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63930368, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e7d8 00010000:00010000:0.0:1789760779.436560:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54fe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.436576:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760779.436580:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.436586:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63930368->63934463] (req 63930368->63934463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.436600:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63930368->63934463] (req 63930368->63934463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.442821:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982917 is committed 00002000:00010000:1.0:1789760779.442946:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63930368 -> 63934464 00002000:00010000:1.0:1789760779.442951:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124888 -> 124896 00010000:00010000:1.0:1789760779.442964:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63930368->63934463] (req 63930368->63934463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.442975:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54fe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63930368->63934463] (req 63930368->63934463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.442984:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63930368->63934463] (req 63930368->63934463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.442991:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760779.442996:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63930368->63934463] (req 63930368->63934463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.443005:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54fe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63930368->63934463] (req 63930368->63934463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.443025:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54fe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63930368->63934463] (req 63930368->63934463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.443034:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c54fe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63930368->63934463] (req 63930368->63934463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.443236:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554763904:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6726us (7028us total) trans 4294982917 rc 0/0 00000100:00100000:1.0:1789760779.443247:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15619 00000100:00100000:1.0:1789760779.443257:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722665726501 time 722682111691 state 0x2 ref 0 00000100:00100000:0.0:1789760779.460564:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.460689:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554764032 00000100:00100000:1.0:1789760779.460777:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554764032 00000100:00100000:1.0:1789760779.461149:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722665726501 time 722682111691 state 0x0 ref 1 00000100:00100000:1.0:1789760779.461167:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722682372107 time 722682111691 state 0x1 ref 1 00000100:00100000:1.0:1789760779.461201:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722682372107 time 722706819677 state 0x0 ref 1 00000100:00100000:1.0:1789760779.461216:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554764032:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760779.461264:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63934464, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e7e0 00010000:00010000:1.0:1789760779.461273:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5505 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.461288:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760779.461293:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.461299:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5505 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63934464->63938559] (req 63934464->63938559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.461311:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5505 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63934464->63938559] (req 63934464->63938559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.470827:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982918 is committed 00002000:00010000:1.0:1789760779.470932:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63934464 -> 63938560 00002000:00010000:1.0:1789760779.470937:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124896 -> 124904 00010000:00010000:1.0:1789760779.470950:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5505 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63934464->63938559] (req 63934464->63938559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.470962:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5505 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63934464->63938559] (req 63934464->63938559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.470972:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5505 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63934464->63938559] (req 63934464->63938559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.470980:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760779.470984:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5505 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63934464->63938559] (req 63934464->63938559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.470995:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5505 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63934464->63938559] (req 63934464->63938559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.471015:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5505 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63934464->63938559] (req 63934464->63938559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.471023:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5505 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63934464->63938559] (req 63934464->63938559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.471123:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554764032:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9907us (10562us total) trans 4294982918 rc 0/0 00000100:00100000:1.0:1789760779.471135:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15620 00000100:00100000:1.0:1789760779.471144:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722682372107 time 722706819677 state 0x2 ref 0 00000100:00100000:2.0:1789760779.479289:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.479410:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554764160 00000100:00100000:1.0:1789760779.479419:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554764160 00000100:00100000:1.0:1789760779.479483:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722682372107 time 722706819677 state 0x0 ref 1 00000100:00100000:1.0:1789760779.479498:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722707080093 time 722706819677 state 0x1 ref 1 00000100:00100000:1.0:1789760779.479519:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722707080093 time 722725137799 state 0x0 ref 1 00000100:00100000:1.0:1789760779.479531:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554764160:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760779.479575:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63938560, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e7e8 00010000:00010000:1.0:1789760779.479583:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c550c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.479599:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760779.479603:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.479609:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c550c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63938560->63942655] (req 63938560->63942655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.479633:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c550c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63938560->63942655] (req 63938560->63942655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.484488:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982919 is committed 00002000:00010000:1.0:1789760779.484663:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63938560 -> 63942656 00002000:00010000:1.0:1789760779.484668:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124904 -> 124912 00010000:00010000:1.0:1789760779.484682:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c550c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63938560->63942655] (req 63938560->63942655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.484695:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c550c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63938560->63942655] (req 63938560->63942655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.484716:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c550c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63938560->63942655] (req 63938560->63942655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.484723:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760779.484728:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c550c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63938560->63942655] (req 63938560->63942655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.484736:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c550c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63938560->63942655] (req 63938560->63942655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.484757:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c550c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63938560->63942655] (req 63938560->63942655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.484765:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c550c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63938560->63942655] (req 63938560->63942655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.484908:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554764160:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5373us (5623us total) trans 4294982919 rc 0/0 00000100:00100000:1.0:1789760779.484917:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15621 00000100:00100000:1.0:1789760779.484926:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722707080093 time 722725137799 state 0x2 ref 0 00000100:00100000:2.0:1789760779.495863:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760779.498291:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554764288 00000100:00100000:0.0:1789760779.498306:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554764288 00000100:00100000:0.0:1789760779.498368:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722707080093 time 722725137799 state 0x0 ref 1 00000100:00100000:0.0:1789760779.498419:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722725398215 time 722725137799 state 0x1 ref 1 00000100:00100000:0.0:1789760779.498442:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722725398215 time 722744060991 state 0x0 ref 1 00000100:00100000:0.0:1789760779.498455:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554764288:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760779.498503:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63942656, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e7f0 00010000:00010000:0.0:1789760779.498512:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5513 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.498528:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760779.498531:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.498538:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5513 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63942656->63946751] (req 63942656->63946751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.498552:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5513 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63942656->63946751] (req 63942656->63946751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.504277:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982920 is committed 00002000:00010000:0.0:1789760779.504420:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63942656 -> 63946752 00002000:00010000:0.0:1789760779.504425:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124912 -> 124920 00010000:00010000:0.0:1789760779.504439:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5513 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63942656->63946751] (req 63942656->63946751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.504453:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5513 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63942656->63946751] (req 63942656->63946751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.504461:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5513 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63942656->63946751] (req 63942656->63946751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.504468:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760779.504473:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5513 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63942656->63946751] (req 63942656->63946751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.504482:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5513 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63942656->63946751] (req 63942656->63946751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.504502:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5513 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63942656->63946751] (req 63942656->63946751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.504508:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5513 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63942656->63946751] (req 63942656->63946751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.504590:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554764288:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6133us (8733us total) trans 4294982920 rc 0/0 00000100:00100000:0.0:1789760779.504602:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15622 00000100:00100000:0.0:1789760779.504612:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722725398215 time 722744060991 state 0x2 ref 0 00000100:00100000:0.0:1789760779.512750:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.513786:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554764416 00000100:00100000:1.0:1789760779.513798:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554764416 00000100:00100000:2.0:1789760779.514231:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722725398215 time 722744060991 state 0x0 ref 1 00000100:00100000:2.0:1789760779.514247:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722744321407 time 722744060991 state 0x1 ref 1 00000100:00100000:2.0:1789760779.514319:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722744321407 time 722759937719 state 0x0 ref 1 00000100:00100000:2.0:1789760779.514332:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554764416:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760779.514403:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63946752, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e7f8 00010000:00010000:2.0:1789760779.514413:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c551a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.514430:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760779.514435:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.514442:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c551a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63946752->63950847] (req 63946752->63950847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.514525:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c551a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63946752->63950847] (req 63946752->63950847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.520658:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982921 is committed 00002000:00010000:2.0:1789760779.521008:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63946752 -> 63950848 00002000:00010000:2.0:1789760779.521013:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124920 -> 124928 00010000:00010000:2.0:1789760779.521027:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c551a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63946752->63950847] (req 63946752->63950847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.521041:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c551a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63946752->63950847] (req 63946752->63950847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.521051:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c551a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63946752->63950847] (req 63946752->63950847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.521058:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760779.521064:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c551a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63946752->63950847] (req 63946752->63950847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.521072:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c551a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63946752->63950847] (req 63946752->63950847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.521151:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c551a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63946752->63950847] (req 63946752->63950847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.521160:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c551a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63946752->63950847] (req 63946752->63950847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.521305:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554764416:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6970us (8560us total) trans 4294982921 rc 0/0 00000100:00100000:2.0:1789760779.521316:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15623 00000100:00100000:2.0:1789760779.521325:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722744321407 time 722759937719 state 0x2 ref 0 00000100:00100000:2.0:1789760779.531275:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.531400:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554764544 00000100:00100000:1.0:1789760779.531410:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554764544 00000100:00100000:1.0:1789760779.531463:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722744321407 time 722759937719 state 0x0 ref 1 00000100:00100000:1.0:1789760779.531477:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722760198135 time 722759937719 state 0x1 ref 1 00000100:00100000:1.0:1789760779.531508:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722760198135 time 722777127186 state 0x0 ref 1 00000100:00100000:1.0:1789760779.531520:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554764544:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760779.531566:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63950848, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e800 00010000:00010000:1.0:1789760779.531574:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5521 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.531589:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760779.531594:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.531600:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5521 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63950848->63954943] (req 63950848->63954943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.531614:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5521 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63950848->63954943] (req 63950848->63954943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.537783:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982922 is committed 00002000:00010000:1.0:1789760779.537906:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63950848 -> 63954944 00002000:00010000:1.0:1789760779.537911:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124928 -> 124936 00010000:00010000:1.0:1789760779.537923:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5521 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63950848->63954943] (req 63950848->63954943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.537937:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5521 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63950848->63954943] (req 63950848->63954943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.537946:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5521 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63950848->63954943] (req 63950848->63954943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.537952:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760779.537957:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5521 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63950848->63954943] (req 63950848->63954943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.537965:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5521 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63950848->63954943] (req 63950848->63954943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.537983:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5521 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63950848->63954943] (req 63950848->63954943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.537990:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5521 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63950848->63954943] (req 63950848->63954943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.538105:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554764544:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6581us (6838us total) trans 4294982922 rc 0/0 00000100:00100000:1.0:1789760779.538115:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15624 00000100:00100000:1.0:1789760779.538124:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722760198135 time 722777127186 state 0x2 ref 0 00000100:00100000:2.0:1789760779.542054:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.542738:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554764672 00000100:00100000:1.0:1789760779.542750:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554764672 00000100:00100000:1.0:1789760779.542801:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722760198135 time 722777127186 state 0x0 ref 1 00000100:00100000:1.0:1789760779.542816:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722777387602 time 722777127186 state 0x1 ref 1 00000100:00100000:1.0:1789760779.545550:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722777387602 time 722791038760 state 0x0 ref 1 00000100:00100000:1.0:1789760779.545581:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554764672:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760779.545634:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63954944, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e808 00010000:00010000:1.0:1789760779.545643:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5528 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.545659:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760779.545663:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.545669:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5528 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63954944->63959039] (req 63954944->63959039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.545684:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5528 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63954944->63959039] (req 63954944->63959039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.550869:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982923 is committed 00002000:00010000:1.0:1789760779.551032:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63954944 -> 63959040 00002000:00010000:1.0:1789760779.551038:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124936 -> 124944 00010000:00010000:1.0:1789760779.551051:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5528 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63954944->63959039] (req 63954944->63959039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.551064:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5528 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63954944->63959039] (req 63954944->63959039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.551079:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5528 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63954944->63959039] (req 63954944->63959039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.551086:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760779.551091:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5528 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63954944->63959039] (req 63954944->63959039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.551099:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5528 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63954944->63959039] (req 63954944->63959039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.551118:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5528 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63954944->63959039] (req 63954944->63959039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.551127:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5528 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63954944->63959039] (req 63954944->63959039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.551218:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554764672:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5644us (9169us total) trans 4294982923 rc 0/0 00000100:00100000:1.0:1789760779.551229:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15625 00000100:00100000:1.0:1789760779.551238:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722777387602 time 722791038760 state 0x2 ref 0 00000100:00100000:0.0:1789760779.553993:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.554736:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554764800 00000100:00100000:1.0:1789760779.554746:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554764800 00000100:00100000:1.0:1789760779.554800:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722777387602 time 722791038760 state 0x0 ref 1 00000100:00100000:1.0:1789760779.554813:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722791299176 time 722791038760 state 0x1 ref 1 00000100:00100000:1.0:1789760779.554848:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722791299176 time 722800467202 state 0x0 ref 1 00000100:00100000:1.0:1789760779.554862:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554764800:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760779.554909:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63959040, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e810 00010000:00010000:1.0:1789760779.554917:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c552f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.554932:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760779.554936:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.554941:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c552f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63959040->63963135] (req 63959040->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.554954:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c552f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63959040->63963135] (req 63959040->63963135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.559194:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982924 is committed 00002000:00010000:1.0:1789760779.559556:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63959040 -> 63963136 00002000:00010000:1.0:1789760779.559561:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124944 -> 124952 00010000:00010000:1.0:1789760779.559574:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c552f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63959040->63963135] (req 63959040->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.559586:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c552f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63959040->63963135] (req 63959040->63963135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.559594:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c552f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63959040->63963135] (req 63959040->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.559606:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760779.559611:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c552f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63959040->63963135] (req 63959040->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.559618:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c552f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63959040->63963135] (req 63959040->63963135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.559634:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c552f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63959040->63963135] (req 63959040->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.559641:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c552f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63959040->63963135] (req 63959040->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.559755:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554764800:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4883us (5769us total) trans 4294982924 rc 0/0 00000100:00100000:1.0:1789760779.559766:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15626 00000100:00100000:1.0:1789760779.559775:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722791299176 time 722800467202 state 0x2 ref 0 00000100:00100000:2.0:1789760779.565841:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760779.565936:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554764928 00000100:00100000:0.0:1789760779.565947:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554764928 00000100:00100000:0.0:1789760779.566002:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722791299176 time 722800467202 state 0x0 ref 1 00000100:00100000:0.0:1789760779.566017:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722800727618 time 722800467202 state 0x1 ref 1 00000100:00100000:0.0:1789760779.566052:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722800727618 time 722811670413 state 0x0 ref 1 00000100:00100000:0.0:1789760779.566064:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe18850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554764928:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760779.566111:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63963136, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e818 00010000:00010000:0.0:1789760779.566119:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5536 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.566135:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760779.566139:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.566145:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5536 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63963136->63967231] (req 63963136->63967231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.566164:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5536 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63963136->63967231] (req 63963136->63967231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.574458:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982925 is committed 00002000:00010000:0.0:1789760779.574634:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63963136 -> 63967232 00002000:00010000:0.0:1789760779.574640:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124952 -> 124960 00010000:00010000:0.0:1789760779.574653:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5536 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63963136->63967231] (req 63963136->63967231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.574665:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5536 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63963136->63967231] (req 63963136->63967231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.574673:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5536 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63963136->63967231] (req 63963136->63967231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.574679:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760779.574684:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5536 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63963136->63967231] (req 63963136->63967231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.574692:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5536 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63963136->63967231] (req 63963136->63967231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.574770:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5536 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63963136->63967231] (req 63963136->63967231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.574779:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5536 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63963136->63967231] (req 63963136->63967231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.574918:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe18850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554764928:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8851us (9094us total) trans 4294982925 rc 0/0 00000100:00100000:0.0:1789760779.574929:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15627 00000100:00100000:0.0:1789760779.574938:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722800727618 time 722811670413 state 0x2 ref 0 00000100:00100000:2.0:1789760779.579728:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760779.579838:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554765056 00000100:00100000:0.0:1789760779.579848:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554765056 00000100:00100000:0.0:1789760779.579904:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722800727618 time 722811670413 state 0x0 ref 1 00000100:00100000:0.0:1789760779.579920:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722811930829 time 722811670413 state 0x1 ref 1 00000100:00100000:0.0:1789760779.579941:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722811930829 time 722825560006 state 0x0 ref 1 00000100:00100000:0.0:1789760779.579953:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554765056:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760779.580003:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63967232, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e820 00010000:00010000:0.0:1789760779.580013:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c553d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.580029:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760779.580033:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.580039:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c553d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63967232->63971327] (req 63967232->63971327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.580052:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c553d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63967232->63971327] (req 63967232->63971327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.587724:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982926 is committed 00002000:00010000:0.0:1789760779.588335:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63967232 -> 63971328 00002000:00010000:0.0:1789760779.588342:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124960 -> 124968 00010000:00010000:0.0:1789760779.588355:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c553d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63967232->63971327] (req 63967232->63971327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.588369:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c553d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63967232->63971327] (req 63967232->63971327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.588797:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c553d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63967232->63971327] (req 63967232->63971327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.588807:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760779.588811:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c553d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63967232->63971327] (req 63967232->63971327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.588820:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c553d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63967232->63971327] (req 63967232->63971327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.588839:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c553d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63967232->63971327] (req 63967232->63971327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.588847:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c553d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63967232->63971327] (req 63967232->63971327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.588986:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554765056:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9029us (9260us total) trans 4294982926 rc 0/0 00000100:00100000:0.0:1789760779.588998:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15628 00000100:00100000:0.0:1789760779.589007:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722811930829 time 722825560006 state 0x2 ref 0 00000100:00100000:2.0:1789760779.595249:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.595372:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554765184 00000100:00100000:1.0:1789760779.595593:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554765184 00000100:00100000:1.0:1789760779.595649:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722811930829 time 722825560006 state 0x0 ref 1 00000100:00100000:1.0:1789760779.595665:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722825820422 time 722825560006 state 0x1 ref 1 00000100:00100000:1.0:1789760779.595690:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722825820422 time 722841309215 state 0x0 ref 1 00000100:00100000:1.0:1789760779.595718:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554765184:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760779.595768:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63971328, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e828 00010000:00010000:1.0:1789760779.595777:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5544 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.595791:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760779.595795:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.595801:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5544 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63971328->63975423] (req 63971328->63975423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.595813:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5544 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63971328->63975423] (req 63971328->63975423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.601196:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982927 is committed 00002000:00010000:3.0:1789760779.601353:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63971328 -> 63975424 00002000:00010000:3.0:1789760779.601359:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124968 -> 124976 00010000:00010000:3.0:1789760779.601372:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5544 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63971328->63975423] (req 63971328->63975423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.601515:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5544 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63971328->63975423] (req 63971328->63975423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.601525:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5544 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63971328->63975423] (req 63971328->63975423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.601532:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760779.601537:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5544 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63971328->63975423] (req 63971328->63975423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.601547:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5544 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63971328->63975423] (req 63971328->63975423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.601565:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5544 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63971328->63975423] (req 63971328->63975423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.601572:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5544 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63971328->63975423] (req 63971328->63975423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.601676:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554765184:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5947us (6425us total) trans 4294982927 rc 0/0 00000100:00100000:3.0:1789760779.601686:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15629 00000100:00100000:3.0:1789760779.601694:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722825820422 time 722841309215 state 0x2 ref 0 00000100:00100000:2.0:1789760779.609752:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.611290:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554765312 00000100:00100000:1.0:1789760779.611304:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554765312 00000100:00100000:1.0:1789760779.611371:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722825820422 time 722841309215 state 0x0 ref 1 00000100:00100000:1.0:1789760779.611404:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722841569631 time 722841309215 state 0x1 ref 1 00000100:00100000:1.0:1789760779.611428:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722841569631 time 722857046822 state 0x0 ref 1 00000100:00100000:1.0:1789760779.611442:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554765312:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760779.611693:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63975424, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e830 00010000:00010000:1.0:1789760779.611711:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c554b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.611733:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760779.611737:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.611743:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c554b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63975424->63979519] (req 63975424->63979519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.611757:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c554b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63975424->63979519] (req 63975424->63979519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.616557:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982928 is committed 00002000:00010000:1.0:1789760779.616871:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63975424 -> 63979520 00002000:00010000:1.0:1789760779.616876:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124976 -> 124984 00010000:00010000:1.0:1789760779.616890:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c554b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63975424->63979519] (req 63975424->63979519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.616903:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c554b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63975424->63979519] (req 63975424->63979519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.616913:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c554b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63975424->63979519] (req 63975424->63979519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.616920:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760779.616925:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c554b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63975424->63979519] (req 63975424->63979519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.616934:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c554b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63975424->63979519] (req 63975424->63979519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.616960:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c554b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63975424->63979519] (req 63975424->63979519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.616968:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c554b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63975424->63979519] (req 63975424->63979519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.617952:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554765312:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6504us (8200us total) trans 4294982928 rc 0/0 00000100:00100000:1.0:1789760779.617964:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15630 00000100:00100000:1.0:1789760779.617974:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722841569631 time 722857046822 state 0x2 ref 0 00000100:00100000:2.0:1789760779.621988:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760779.622236:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554765440 00000100:00100000:0.0:1789760779.622247:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554765440 00000100:00100000:0.0:1789760779.622300:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722841569631 time 722857046822 state 0x0 ref 1 00000100:00100000:0.0:1789760779.622315:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722857307238 time 722857046822 state 0x1 ref 1 00000100:00100000:0.0:1789760779.622412:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722857307238 time 722868029191 state 0x0 ref 1 00000100:00100000:0.0:1789760779.622425:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554765440:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760779.622596:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63979520, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e838 00010000:00010000:0.0:1789760779.622607:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5552 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.622624:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760779.622629:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.622636:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5552 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63979520->63983615] (req 63979520->63983615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.622650:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5552 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63979520->63983615] (req 63979520->63983615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.626928:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982929 is committed 00002000:00010000:0.0:1789760779.627085:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63979520 -> 63983616 00002000:00010000:0.0:1789760779.627091:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124984 -> 124992 00010000:00010000:0.0:1789760779.627106:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5552 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63979520->63983615] (req 63979520->63983615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.627119:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5552 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63979520->63983615] (req 63979520->63983615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.627129:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5552 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63979520->63983615] (req 63979520->63983615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.627136:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760779.627141:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5552 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63979520->63983615] (req 63979520->63983615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.627150:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5552 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63979520->63983615] (req 63979520->63983615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.627169:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5552 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63979520->63983615] (req 63979520->63983615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.627177:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5552 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63979520->63983615] (req 63979520->63983615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.627271:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554765440:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4843us (5288us total) trans 4294982929 rc 0/0 00000100:00100000:0.0:1789760779.627281:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15631 00000100:00100000:0.0:1789760779.627290:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722857307238 time 722868029191 state 0x2 ref 0 00000100:00100000:2.0:1789760779.630917:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760779.631035:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554765568 00000100:00100000:0.0:1789760779.631046:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554765568 00000100:00100000:0.0:1789760779.631160:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722857307238 time 722868029191 state 0x0 ref 1 00000100:00100000:0.0:1789760779.631176:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722868289607 time 722868029191 state 0x1 ref 1 00000100:00100000:0.0:1789760779.631262:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722868289607 time 722876880030 state 0x0 ref 1 00000100:00100000:0.0:1789760779.631276:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554765568:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760779.631322:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63983616, mtime: 0x6aad950a, atime: 0x0, ctime: 0x6aad950a, blocks: 0x1e840 00010000:00010000:0.0:1789760779.631331:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5559 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.631347:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760779.631351:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.631357:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5559 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63983616->63987711] (req 63983616->63987711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.631370:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5559 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63983616->63987711] (req 63983616->63987711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.635491:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982930 is committed 00002000:00010000:0.0:1789760779.636596:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63983616 -> 63987712 00002000:00010000:0.0:1789760779.636603:0:15101:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760778 -> 1789760779 00002000:00010000:0.0:1789760779.636605:0:15101:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760778 -> 1789760779 00002000:00010000:0.0:1789760779.636608:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 124992 -> 125000 00010000:00010000:0.0:1789760779.636620:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5559 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63983616->63987711] (req 63983616->63987711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.636632:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5559 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63983616->63987711] (req 63983616->63987711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.636640:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5559 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63983616->63987711] (req 63983616->63987711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.636646:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760779.636650:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5559 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63983616->63987711] (req 63983616->63987711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.636659:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5559 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63983616->63987711] (req 63983616->63987711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.636678:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5559 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63983616->63987711] (req 63983616->63987711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.636685:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5559 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63983616->63987711] (req 63983616->63987711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.636837:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554765568:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5560us (5926us total) trans 4294982930 rc 0/0 00000100:00100000:0.0:1789760779.636849:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15632 00000100:00100000:0.0:1789760779.636858:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722868289607 time 722876880030 state 0x2 ref 0 00000100:00100000:2.0:1789760779.641934:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.642029:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554765696 00000100:00100000:1.0:1789760779.642039:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554765696 00000100:00100000:1.0:1789760779.642102:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722868289607 time 722876880030 state 0x0 ref 1 00000100:00100000:1.0:1789760779.642119:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722877140446 time 722876880030 state 0x1 ref 1 00000100:00100000:1.0:1789760779.642156:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722877140446 time 722887775085 state 0x0 ref 1 00000100:00100000:1.0:1789760779.642170:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554765696:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760779.642218:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63987712, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e848 00010000:00010000:1.0:1789760779.642229:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5560 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.642244:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760779.642250:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.642256:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5560 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63987712->63991807] (req 63987712->63991807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.642271:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5560 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63987712->63991807] (req 63987712->63991807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.648016:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982931 is committed 00002000:00010000:1.0:1789760779.648175:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63987712 -> 63991808 00002000:00010000:1.0:1789760779.648181:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125000 -> 125008 00010000:00010000:1.0:1789760779.648194:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5560 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63987712->63991807] (req 63987712->63991807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.648207:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5560 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63987712->63991807] (req 63987712->63991807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.648215:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5560 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63987712->63991807] (req 63987712->63991807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.648221:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760779.648225:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5560 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63987712->63991807] (req 63987712->63991807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.648233:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5560 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63987712->63991807] (req 63987712->63991807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.648252:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5560 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63987712->63991807] (req 63987712->63991807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.648266:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5560 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63987712->63991807] (req 63987712->63991807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.648370:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554765696:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6200us (6442us total) trans 4294982931 rc 0/0 00000100:00100000:1.0:1789760779.648419:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15633 00000100:00100000:1.0:1789760779.648429:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722877140446 time 722887775085 state 0x2 ref 0 00000100:00100000:2.0:1789760779.652349:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.652649:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554765824 00000100:00100000:1.0:1789760779.652668:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554765824 00000100:00100000:1.0:1789760779.652911:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722877140446 time 722887775085 state 0x0 ref 1 00000100:00100000:1.0:1789760779.652929:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722888035501 time 722887775085 state 0x1 ref 1 00000100:00100000:1.0:1789760779.652958:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722888035501 time 722898577089 state 0x0 ref 1 00000100:00100000:1.0:1789760779.652970:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554765824:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760779.653020:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63991808, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e850 00010000:00010000:1.0:1789760779.653029:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5567 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.653046:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760779.653051:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.653057:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5567 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63991808->63995903] (req 63991808->63995903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.653076:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5567 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63991808->63995903] (req 63991808->63995903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.657665:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982932 is committed 00002000:00010000:0.0:1789760779.657875:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63991808 -> 63995904 00002000:00010000:0.0:1789760779.657881:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125008 -> 125016 00010000:00010000:0.0:1789760779.657894:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5567 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63991808->63995903] (req 63991808->63995903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.657907:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5567 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63991808->63995903] (req 63991808->63995903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.657916:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5567 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63991808->63995903] (req 63991808->63995903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.657923:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760779.657928:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5567 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63991808->63995903] (req 63991808->63995903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.657936:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5567 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63991808->63995903] (req 63991808->63995903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.657955:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5567 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63991808->63995903] (req 63991808->63995903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.657963:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5567 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63991808->63995903] (req 63991808->63995903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.658061:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554765824:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5089us (5717us total) trans 4294982932 rc 0/0 00000100:00100000:0.0:1789760779.658073:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15634 00000100:00100000:0.0:1789760779.658139:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722888035501 time 722898577089 state 0x2 ref 0 00000100:00100000:2.0:1789760779.664008:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760779.664124:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554765952 00000100:00100000:0.0:1789760779.664134:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554765952 00000100:00100000:0.0:1789760779.664192:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722888035501 time 722898577089 state 0x0 ref 1 00000100:00100000:0.0:1789760779.664208:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722898837505 time 722898577089 state 0x1 ref 1 00000100:00100000:0.0:1789760779.664242:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722898837505 time 722909860661 state 0x0 ref 1 00000100:00100000:0.0:1789760779.664254:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554765952:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760779.664681:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 63995904, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e858 00010000:00010000:0.0:1789760779.664691:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c556e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.664716:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760779.664720:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.664727:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c556e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63995904->63999999] (req 63995904->63999999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.664741:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c556e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63995904->63999999] (req 63995904->63999999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.665949:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cc1d94df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573916288:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789760779.666010:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760779.666060:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573916288 00000100:00100000:2.0:1789760779.666067:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573916288 00000100:00100000:2.0:1789760779.666132:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 360 00000100:00100000:2.0:1789760779.666140:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06490e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3646:x1876699573916288:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1789760779.666156:0:8440:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484945600 pending 28672 free 3860066304 avail 3624730624 00000100:00080000:3.0:1789760779.666192:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789760779.666202:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06490e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3646:x1876699573916288:12345-0@lo:13:osp-pre-0-1.0 Request processed in 63us (191us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760779.666210:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 360 00000100:00100000:2.0:1789760779.666266:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cc1d94df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573916288:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1789760779.669862:0:3646:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760779.669887:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb04f85500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573916416:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760779.669956:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760779.670038:0:8439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573916416 00000100:00100000:0.0:1789760779.670046:0:8439:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573916416 00000100:00100000:0.0:1789760779.670104:0:8439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 361 00000100:00100000:0.0:1789760779.670112:0:8439:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573916416:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789760779.670135:0:8439:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484945600 pending 28672 free 3860066304 avail 3624730624 00000100:00100000:0.0:1789760779.670178:0:8439:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573916416:12345-0@lo:13:osp-pre-0-0.0 Request processed in 66us (222us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760779.670185:0:8439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 361 00000100:00080000:0.0:1789760779.670205:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760779.670242:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb04f85500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573916416:0@lo:13:osp-pre-0-0.0 00000001:00080000:0.0:1789760779.672465:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982933 is committed 00002000:00010000:2.0:1789760779.672571:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 63995904 -> 64000000 00002000:00010000:2.0:1789760779.672575:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125016 -> 125024 00010000:00010000:2.0:1789760779.672588:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c556e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63995904->63999999] (req 63995904->63999999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.672607:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c556e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63995904->63999999] (req 63995904->63999999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.672616:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c556e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63995904->63999999] (req 63995904->63999999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.672623:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760779.672628:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c556e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63995904->63999999] (req 63995904->63999999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.672638:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c556e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63995904->63999999] (req 63995904->63999999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.672657:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c556e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63995904->63999999] (req 63995904->63999999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.672665:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c556e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [63995904->63999999] (req 63995904->63999999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.672767:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554765952:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8511us (8764us total) trans 4294982933 rc 0/0 00000100:00100000:2.0:1789760779.672778:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15635 00000100:00100000:2.0:1789760779.672787:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722898837505 time 722909860661 state 0x2 ref 0 00000100:00100000:2.0:1789760779.677103:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760779.677229:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554766080 00000100:00100000:0.0:1789760779.677238:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554766080 00000100:00100000:0.0:1789760779.677300:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722898837505 time 722909860661 state 0x0 ref 1 00000100:00100000:0.0:1789760779.677314:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722910121077 time 722909860661 state 0x1 ref 1 00000100:00100000:0.0:1789760779.677347:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722910121077 time 722922965642 state 0x0 ref 1 00000100:00100000:0.0:1789760779.677358:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554766080:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760779.677436:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64000000, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e860 00010000:00010000:0.0:1789760779.677445:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5575 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.677473:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760779.677477:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.677484:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5575 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64000000->64004095] (req 64000000->64004095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.677504:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5575 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64000000->64004095] (req 64000000->64004095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.682274:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982934 is committed 00002000:00010000:2.0:1789760779.682894:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64000000 -> 64004096 00002000:00010000:2.0:1789760779.682900:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125024 -> 125032 00010000:00010000:2.0:1789760779.682914:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5575 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64000000->64004095] (req 64000000->64004095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.682925:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5575 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64000000->64004095] (req 64000000->64004095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.682934:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5575 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64000000->64004095] (req 64000000->64004095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.682940:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760779.682945:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5575 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64000000->64004095] (req 64000000->64004095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.682955:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5575 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64000000->64004095] (req 64000000->64004095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.682977:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5575 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64000000->64004095] (req 64000000->64004095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.682984:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5575 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64000000->64004095] (req 64000000->64004095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.683129:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554766080:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5766us (6029us total) trans 4294982934 rc 0/0 00000100:00100000:2.0:1789760779.683141:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15636 00000100:00100000:2.0:1789760779.683152:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722910121077 time 722922965642 state 0x2 ref 0 00000100:00100000:3.0:1789760779.687479:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.688602:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554766208 00000100:00100000:1.0:1789760779.688613:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554766208 00000100:00100000:1.0:1789760779.688669:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722910121077 time 722922965642 state 0x0 ref 1 00000100:00100000:1.0:1789760779.688684:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722923226058 time 722922965642 state 0x1 ref 1 00000100:00100000:1.0:1789760779.688877:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722923226058 time 722934494079 state 0x0 ref 1 00000100:00100000:1.0:1789760779.688891:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06491c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554766208:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760779.688941:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64004096, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e868 00010000:00010000:1.0:1789760779.688950:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c557c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.688967:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760779.688970:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.688976:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c557c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64004096->64008191] (req 64004096->64008191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.688989:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c557c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64004096->64008191] (req 64004096->64008191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.693941:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982935 is committed 00002000:00010000:1.0:1789760779.694792:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64004096 -> 64008192 00002000:00010000:1.0:1789760779.694799:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125032 -> 125040 00010000:00010000:1.0:1789760779.694812:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c557c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64004096->64008191] (req 64004096->64008191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.694824:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c557c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64004096->64008191] (req 64004096->64008191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.694833:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c557c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64004096->64008191] (req 64004096->64008191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.694840:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760779.694845:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c557c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64004096->64008191] (req 64004096->64008191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.694853:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c557c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64004096->64008191] (req 64004096->64008191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.694871:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c557c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64004096->64008191] (req 64004096->64008191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.694880:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c557c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64004096->64008191] (req 64004096->64008191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.694972:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06491c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554766208:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6079us (7497us total) trans 4294982935 rc 0/0 00000100:00100000:1.0:1789760779.694984:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15637 00000100:00100000:1.0:1789760779.694993:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722923226058 time 722934494079 state 0x2 ref 0 00000100:00100000:2.0:1789760779.701653:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.701737:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554766336 00000100:00100000:1.0:1789760779.701746:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554766336 00000100:00100000:1.0:1789760779.701801:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722923226058 time 722934494079 state 0x0 ref 1 00000100:00100000:1.0:1789760779.701816:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722934754495 time 722934494079 state 0x1 ref 1 00000100:00100000:1.0:1789760779.701844:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722934754495 time 722947462864 state 0x0 ref 1 00000100:00100000:1.0:1789760779.701856:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a272a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554766336:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760779.701910:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64008192, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e870 00010000:00010000:1.0:1789760779.701919:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5583 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.701935:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789760779.701939:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.701946:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5583 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64008192->64012287] (req 64008192->64012287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.701960:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5583 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64008192->64012287] (req 64008192->64012287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.705858:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982936 is committed 00002000:00010000:1.0:1789760779.705955:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64008192 -> 64012288 00002000:00010000:1.0:1789760779.705960:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125040 -> 125048 00010000:00010000:1.0:1789760779.705977:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5583 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64008192->64012287] (req 64008192->64012287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.705994:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5583 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64008192->64012287] (req 64008192->64012287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.706003:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5583 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64008192->64012287] (req 64008192->64012287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.706009:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760779.706015:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5583 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64008192->64012287] (req 64008192->64012287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.706023:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5583 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64008192->64012287] (req 64008192->64012287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.706041:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5583 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64008192->64012287] (req 64008192->64012287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.706048:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5583 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64008192->64012287] (req 64008192->64012287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.706139:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a272a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554766336:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4281us (4491us total) trans 4294982936 rc 0/0 00000100:00100000:1.0:1789760779.706148:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15638 00000100:00100000:1.0:1789760779.706156:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722934754495 time 722947462864 state 0x2 ref 0 00000100:00100000:2.0:1789760779.713027:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760779.713127:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554766464 00000100:00100000:0.0:1789760779.713138:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554766464 00000100:00100000:0.0:1789760779.713191:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722934754495 time 722947462864 state 0x0 ref 1 00000100:00100000:0.0:1789760779.713206:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722947723280 time 722947462864 state 0x1 ref 1 00000100:00100000:0.0:1789760779.713235:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722947723280 time 722958854309 state 0x0 ref 1 00000100:00100000:0.0:1789760779.713246:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a270380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554766464:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760779.713289:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64012288, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e878 00010000:00010000:0.0:1789760779.713298:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c558a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.713314:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760779.713318:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.713324:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c558a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64012288->64016383] (req 64012288->64016383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.713339:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c558a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64012288->64016383] (req 64012288->64016383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.717486:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982937 is committed 00002000:00010000:0.0:1789760779.718066:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64012288 -> 64016384 00002000:00010000:0.0:1789760779.718072:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125048 -> 125056 00010000:00010000:0.0:1789760779.718144:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c558a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64012288->64016383] (req 64012288->64016383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.718158:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c558a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64012288->64016383] (req 64012288->64016383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.718167:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c558a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64012288->64016383] (req 64012288->64016383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.718173:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760779.718178:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c558a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64012288->64016383] (req 64012288->64016383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.718186:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c558a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64012288->64016383] (req 64012288->64016383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.718206:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c558a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64012288->64016383] (req 64012288->64016383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.718212:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c558a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64012288->64016383] (req 64012288->64016383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.718310:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a270380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554766464:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5061us (5289us total) trans 4294982937 rc 0/0 00000100:00100000:0.0:1789760779.718321:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15639 00000100:00100000:0.0:1789760779.718330:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722947723280 time 722958854309 state 0x2 ref 0 00000100:00100000:3.0:1789760779.721957:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760779.723282:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554766592 00000100:00100000:2.0:1789760779.723294:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554766592 00000100:00100000:2.0:1789760779.723352:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722947723280 time 722958854309 state 0x0 ref 1 00000100:00100000:2.0:1789760779.723367:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722959114725 time 722958854309 state 0x1 ref 1 00000100:00100000:2.0:1789760779.723696:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722959114725 time 722969313689 state 0x0 ref 1 00000100:00100000:2.0:1789760779.723720:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06490000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554766592:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760779.723763:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64016384, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e880 00010000:00010000:2.0:1789760779.723771:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5591 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.723786:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789760779.723790:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.723796:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5591 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64016384->64020479] (req 64016384->64020479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.723809:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5591 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64016384->64020479] (req 64016384->64020479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.730788:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982938 is committed 00002000:00010000:2.0:1789760779.730907:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64016384 -> 64020480 00002000:00010000:2.0:1789760779.730913:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125056 -> 125064 00010000:00010000:2.0:1789760779.730926:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5591 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64016384->64020479] (req 64016384->64020479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.730938:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5591 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64016384->64020479] (req 64016384->64020479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.730947:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5591 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64016384->64020479] (req 64016384->64020479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.730954:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760779.730959:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5591 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64016384->64020479] (req 64016384->64020479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.730968:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5591 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64016384->64020479] (req 64016384->64020479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.730988:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5591 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64016384->64020479] (req 64016384->64020479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.730997:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5591 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64016384->64020479] (req 64016384->64020479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.731206:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06490000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554766592:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7483us (9252us total) trans 4294982938 rc 0/0 00000100:00100000:2.0:1789760779.731217:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15640 00000100:00100000:2.0:1789760779.731228:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722959114725 time 722969313689 state 0x2 ref 0 00000100:00100000:2.0:1789760779.738834:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.738981:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554766720 00000100:00100000:1.0:1789760779.738991:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554766720 00000100:00100000:1.0:1789760779.739042:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722959114725 time 722969313689 state 0x0 ref 1 00000100:00100000:1.0:1789760779.739056:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722969574105 time 722969313689 state 0x1 ref 1 00000100:00100000:0.0:1789760779.739121:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722969574105 time 722984736422 state 0x0 ref 1 00000100:00100000:0.0:1789760779.739136:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a270700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554766720:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760779.739184:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64020480, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e888 00010000:00010000:0.0:1789760779.739193:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5598 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.739208:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760779.739212:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.739219:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5598 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64020480->64024575] (req 64020480->64024575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.739237:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5598 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64020480->64024575] (req 64020480->64024575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.747268:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982939 is committed 00002000:00010000:0.0:1789760779.747469:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64020480 -> 64024576 00002000:00010000:0.0:1789760779.747474:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125064 -> 125072 00010000:00010000:0.0:1789760779.747489:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5598 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64020480->64024575] (req 64020480->64024575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.747500:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5598 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64020480->64024575] (req 64020480->64024575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.747508:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5598 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64020480->64024575] (req 64020480->64024575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.747520:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760779.747526:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5598 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64020480->64024575] (req 64020480->64024575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.747535:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5598 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64020480->64024575] (req 64020480->64024575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.747554:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5598 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64020480->64024575] (req 64020480->64024575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.747562:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5598 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64020480->64024575] (req 64020480->64024575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.747676:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a270700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554766720:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8538us (8846us total) trans 4294982939 rc 0/0 00000100:00100000:0.0:1789760779.747687:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15641 00000100:00100000:0.0:1789760779.747697:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722969574105 time 722984736422 state 0x2 ref 0 00000100:00100000:1.0:1789760779.751610:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760779.751796:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554766848 00000100:00100000:0.0:1789760779.751806:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554766848 00000100:00100000:0.0:1789760779.751858:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722969574105 time 722984736422 state 0x0 ref 1 00000100:00100000:0.0:1789760779.751872:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722984996838 time 722984736422 state 0x1 ref 1 00000100:00100000:0.0:1789760779.751910:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722984996838 time 722997527996 state 0x0 ref 1 00000100:00100000:0.0:1789760779.751924:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554766848:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760779.751969:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64024576, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e890 00010000:00010000:0.0:1789760779.751978:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c559f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.751995:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=722s v=(0 0) 00010000:00010000:0.0:1789760779.752000:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.752006:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c559f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64024576->64028671] (req 64024576->64028671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.752021:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c559f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64024576->64028671] (req 64024576->64028671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.757177:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982940 is committed 00002000:00010000:0.0:1789760779.757354:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64024576 -> 64028672 00002000:00010000:0.0:1789760779.757359:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125072 -> 125080 00010000:00010000:0.0:1789760779.757372:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c559f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64024576->64028671] (req 64024576->64028671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.757608:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c559f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64024576->64028671] (req 64024576->64028671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.757630:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c559f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64024576->64028671] (req 64024576->64028671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.757729:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760779.757734:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c559f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64024576->64028671] (req 64024576->64028671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.757745:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c559f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64024576->64028671] (req 64024576->64028671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.757766:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c559f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64024576->64028671] (req 64024576->64028671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.757774:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c559f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64024576->64028671] (req 64024576->64028671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.757867:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554766848:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5939us (6260us total) trans 4294982940 rc 0/0 00000100:00100000:0.0:1789760779.757877:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15642 00000100:00100000:0.0:1789760779.757887:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722984996838 time 722997527996 state 0x2 ref 0 00000100:00100000:3.0:1789760779.761628:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.761709:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554766976 00000100:00100000:1.0:1789760779.761721:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554766976 00000100:00100000:1.0:1789760779.761771:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722984996838 time 722997527996 state 0x0 ref 1 00000100:00100000:1.0:1789760779.761787:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722997788412 time 722997527996 state 0x1 ref 1 00000100:00100000:1.0:1789760779.761818:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722997788412 time 723007436544 state 0x0 ref 1 00000100:00100000:1.0:1789760779.761830:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554766976:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760779.761875:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64028672, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e898 00010000:00010000:1.0:1789760779.761884:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55a6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.761900:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760779.761904:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.761910:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64028672->64032767] (req 64028672->64032767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.761923:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64028672->64032767] (req 64028672->64032767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.767139:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982941 is committed 00002000:00010000:1.0:1789760779.767328:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64028672 -> 64032768 00002000:00010000:1.0:1789760779.767334:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125080 -> 125088 00010000:00010000:1.0:1789760779.767509:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64028672->64032767] (req 64028672->64032767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.767523:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55a6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64028672->64032767] (req 64028672->64032767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.767533:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64028672->64032767] (req 64028672->64032767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.767541:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760779.767546:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64028672->64032767] (req 64028672->64032767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.767556:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55a6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64028672->64032767] (req 64028672->64032767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.767577:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55a6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64028672->64032767] (req 64028672->64032767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.767585:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55a6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64028672->64032767] (req 64028672->64032767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.767679:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554766976:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5846us (6052us total) trans 4294982941 rc 0/0 00000100:00100000:1.0:1789760779.767691:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15643 00000100:00100000:1.0:1789760779.767711:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722997788412 time 723007436544 state 0x2 ref 0 00000100:00100000:2.0:1789760779.771977:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.772054:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554767104 00000100:00100000:1.0:1789760779.772064:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554767104 00000100:00100000:1.0:1789760779.772118:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 722997788412 time 723007436544 state 0x0 ref 1 00000100:00100000:1.0:1789760779.772134:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723007696960 time 723007436544 state 0x1 ref 1 00000100:00100000:1.0:1789760779.772164:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723007696960 time 723017782770 state 0x0 ref 1 00000100:00100000:1.0:1789760779.772176:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554767104:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760779.772223:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64032768, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e8a0 00010000:00010000:1.0:1789760779.772232:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55ad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.772247:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760779.772252:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.772258:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64032768->64036863] (req 64032768->64036863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.772273:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64032768->64036863] (req 64032768->64036863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.776237:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982942 is committed 00002000:00010000:1.0:1789760779.776335:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64032768 -> 64036864 00002000:00010000:1.0:1789760779.776447:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125088 -> 125096 00010000:00010000:1.0:1789760779.776518:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64032768->64036863] (req 64032768->64036863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.776534:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55ad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64032768->64036863] (req 64032768->64036863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.776542:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64032768->64036863] (req 64032768->64036863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.776548:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760779.776552:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64032768->64036863] (req 64032768->64036863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.776559:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64032768->64036863] (req 64032768->64036863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.776578:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55ad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64032768->64036863] (req 64032768->64036863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.776585:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55ad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64032768->64036863] (req 64032768->64036863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.776707:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554767104:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4520us (4730us total) trans 4294982942 rc 0/0 00000100:00100000:1.0:1789760779.776718:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15644 00000100:00100000:1.0:1789760779.776727:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723007696960 time 723017782770 state 0x2 ref 0 00000100:00100000:2.0:1789760779.783009:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760779.783293:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554767232 00000100:00100000:0.0:1789760779.783302:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554767232 00000100:00100000:0.0:1789760779.783356:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723007696960 time 723017782770 state 0x0 ref 1 00000100:00100000:0.0:1789760779.783371:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723018043186 time 723017782770 state 0x1 ref 1 00000100:00100000:0.0:1789760779.783502:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723018043186 time 723029119744 state 0x0 ref 1 00000100:00100000:0.0:1789760779.783516:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554767232:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760779.783564:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64036864, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e8a8 00010000:00010000:0.0:1789760779.783573:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c55b4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.783586:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760779.783590:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.783596:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c55b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64036864->64040959] (req 64036864->64040959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.783609:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c55b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64036864->64040959] (req 64036864->64040959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.787934:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982943 is committed 00002000:00010000:2.0:1789760779.788049:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64036864 -> 64040960 00002000:00010000:2.0:1789760779.788058:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125096 -> 125104 00010000:00010000:2.0:1789760779.788081:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c55b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64036864->64040959] (req 64036864->64040959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.788094:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c55b4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64036864->64040959] (req 64036864->64040959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.788103:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c55b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64036864->64040959] (req 64036864->64040959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.788110:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760779.788114:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c55b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64036864->64040959] (req 64036864->64040959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.788123:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c55b4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64036864->64040959] (req 64036864->64040959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.788143:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c55b4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64036864->64040959] (req 64036864->64040959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.788151:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c55b4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64036864->64040959] (req 64036864->64040959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.788250:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554767232:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4732us (5246us total) trans 4294982943 rc 0/0 00000100:00100000:2.0:1789760779.788261:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15645 00000100:00100000:2.0:1789760779.788271:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723018043186 time 723029119744 state 0x2 ref 0 00000100:00100000:3.0:1789760779.794622:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760779.795022:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554767360 00000100:00100000:0.0:1789760779.795032:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554767360 00000100:00100000:0.0:1789760779.795093:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723018043186 time 723029119744 state 0x0 ref 1 00000100:00100000:0.0:1789760779.795108:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723029380160 time 723029119744 state 0x1 ref 1 00000100:00100000:0.0:1789760779.795139:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723029380160 time 723040757218 state 0x0 ref 1 00000100:00100000:0.0:1789760779.795151:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06492680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554767360:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760779.795199:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64040960, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e8b0 00010000:00010000:0.0:1789760779.795208:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c55bb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.795222:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760779.795226:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.795232:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c55bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64040960->64045055] (req 64040960->64045055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.795244:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c55bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64040960->64045055] (req 64040960->64045055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.801150:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982944 is committed 00002000:00010000:1.0:1789760779.805071:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64040960 -> 64045056 00002000:00010000:1.0:1789760779.805082:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125104 -> 125112 00010000:00010000:1.0:1789760779.805095:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c55bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64040960->64045055] (req 64040960->64045055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.805108:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c55bb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64040960->64045055] (req 64040960->64045055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.805116:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c55bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64040960->64045055] (req 64040960->64045055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.805123:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760779.805128:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c55bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64040960->64045055] (req 64040960->64045055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.805136:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c55bb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64040960->64045055] (req 64040960->64045055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.805156:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c55bb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64040960->64045055] (req 64040960->64045055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.805163:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c55bb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64040960->64045055] (req 64040960->64045055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.805314:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06492680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554767360:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10160us (10695us total) trans 4294982944 rc 0/0 00000100:00100000:1.0:1789760779.805325:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15646 00000100:00100000:1.0:1789760779.805334:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723029380160 time 723040757218 state 0x2 ref 0 00000100:00100000:0.0:1789760779.812579:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.812669:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554767488 00000100:00100000:1.0:1789760779.812679:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554767488 00000100:00100000:1.0:1789760779.812751:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723029380160 time 723040757218 state 0x0 ref 1 00000100:00100000:1.0:1789760779.812766:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723041017634 time 723040757218 state 0x1 ref 1 00000100:00100000:1.0:1789760779.812798:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723041017634 time 723058416503 state 0x0 ref 1 00000100:00100000:1.0:1789760779.812810:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554767488:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760779.812854:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64045056, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e8b8 00010000:00010000:1.0:1789760779.812863:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55c2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.812878:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760779.812882:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.812888:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64045056->64049151] (req 64045056->64049151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.812902:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64045056->64049151] (req 64045056->64049151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.818368:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982945 is committed 00002000:00010000:1.0:1789760779.820303:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64045056 -> 64049152 00002000:00010000:1.0:1789760779.820312:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125112 -> 125120 00010000:00010000:1.0:1789760779.820327:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64045056->64049151] (req 64045056->64049151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.820357:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55c2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64045056->64049151] (req 64045056->64049151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.820365:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64045056->64049151] (req 64045056->64049151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.820372:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760779.820401:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64045056->64049151] (req 64045056->64049151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.820411:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55c2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64045056->64049151] (req 64045056->64049151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.820430:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55c2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64045056->64049151] (req 64045056->64049151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.820438:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55c2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64045056->64049151] (req 64045056->64049151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.820555:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554767488:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7741us (7982us total) trans 4294982945 rc 0/0 00000100:00100000:1.0:1789760779.820567:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15647 00000100:00100000:1.0:1789760779.820576:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723041017634 time 723058416503 state 0x2 ref 0 00000100:00100000:2.0:1789760779.827765:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.827947:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554767616 00000100:00100000:1.0:1789760779.827960:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554767616 00000100:00100000:1.0:1789760779.828017:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723041017634 time 723058416503 state 0x0 ref 1 00000100:00100000:1.0:1789760779.828033:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723058676919 time 723058416503 state 0x1 ref 1 00000100:00100000:1.0:1789760779.828069:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723058676919 time 723073687452 state 0x0 ref 1 00000100:00100000:1.0:1789760779.828082:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554767616:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760779.828131:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64049152, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e8c0 00010000:00010000:1.0:1789760779.828141:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55c9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.828159:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760779.828164:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.828171:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64049152->64053247] (req 64049152->64053247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.828185:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64049152->64053247] (req 64049152->64053247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.835824:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982946 is committed 00002000:00010000:1.0:1789760779.835953:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64049152 -> 64053248 00002000:00010000:1.0:1789760779.835959:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125120 -> 125128 00010000:00010000:1.0:1789760779.835973:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64049152->64053247] (req 64049152->64053247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.835985:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55c9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64049152->64053247] (req 64049152->64053247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.835999:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64049152->64053247] (req 64049152->64053247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.836007:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760779.836011:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64049152->64053247] (req 64049152->64053247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.836020:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55c9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64049152->64053247] (req 64049152->64053247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.836037:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55c9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64049152->64053247] (req 64049152->64053247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.836045:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55c9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64049152->64053247] (req 64049152->64053247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.836323:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554767616:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8238us (8562us total) trans 4294982946 rc 0/0 00000100:00100000:1.0:1789760779.836334:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15648 00000100:00100000:1.0:1789760779.836344:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723058676919 time 723073687452 state 0x2 ref 0 00000100:00100000:3.0:1789760779.844955:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760779.845036:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554767744 00000100:00100000:2.0:1789760779.845048:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554767744 00000100:00100000:2.0:1789760779.845104:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723058676919 time 723073687452 state 0x0 ref 1 00000100:00100000:2.0:1789760779.845118:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723073947868 time 723073687452 state 0x1 ref 1 00000100:00100000:2.0:1789760779.845153:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723073947868 time 723090771792 state 0x0 ref 1 00000100:00100000:2.0:1789760779.845168:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554767744:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760779.845218:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64053248, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e8c8 00010000:00010000:2.0:1789760779.845229:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c55d0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.845246:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760779.845251:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.845258:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c55d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64053248->64057343] (req 64053248->64057343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.845273:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c55d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64053248->64057343] (req 64053248->64057343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.852575:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982947 is committed 00002000:00010000:0.0:1789760779.855545:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64053248 -> 64057344 00002000:00010000:0.0:1789760779.855554:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125128 -> 125136 00010000:00010000:0.0:1789760779.855569:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c55d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64053248->64057343] (req 64053248->64057343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.855583:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c55d0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64053248->64057343] (req 64053248->64057343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.855593:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c55d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64053248->64057343] (req 64053248->64057343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.855600:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760779.855604:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c55d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64053248->64057343] (req 64053248->64057343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.855614:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c55d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64053248->64057343] (req 64053248->64057343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.855633:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c55d0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64053248->64057343] (req 64053248->64057343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.855642:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c55d0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64053248->64057343] (req 64053248->64057343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.855760:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554767744:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10591us (10810us total) trans 4294982947 rc 0/0 00000100:00100000:0.0:1789760779.855772:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15649 00000100:00100000:0.0:1789760779.855781:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723073947868 time 723090771792 state 0x2 ref 0 00010000:00010000:2.0:1789760779.857545:0:6508:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760779.860140:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.860279:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554767872 00000100:00100000:1.0:1789760779.860289:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554767872 00000100:00100000:1.0:1789760779.860340:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723073947868 time 723090771792 state 0x0 ref 1 00000100:00100000:1.0:1789760779.860356:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723091032208 time 723090771792 state 0x1 ref 1 00000100:00100000:1.0:1789760779.860516:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723091032208 time 723106133516 state 0x0 ref 1 00000100:00100000:1.0:1789760779.860529:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554767872:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760779.860578:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64057344, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e8d0 00010000:00010000:1.0:1789760779.860588:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55d7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.860603:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760779.860607:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.860614:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64057344->64061439] (req 64057344->64061439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.860627:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64057344->64061439] (req 64057344->64061439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760779.872726:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982948 is committed 00002000:00010000:1.0:1789760779.872957:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64057344 -> 64061440 00002000:00010000:1.0:1789760779.872964:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125136 -> 125144 00010000:00010000:1.0:1789760779.872976:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64057344->64061439] (req 64057344->64061439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.872988:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55d7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64057344->64061439] (req 64057344->64061439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.872997:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64057344->64061439] (req 64057344->64061439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.873009:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760779.873014:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64057344->64061439] (req 64057344->64061439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.873023:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55d7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64057344->64061439] (req 64057344->64061439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.873041:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55d7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64057344->64061439] (req 64057344->64061439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.873049:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55d7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64057344->64061439] (req 64057344->64061439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.873212:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554767872:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12680us (13077us total) trans 4294982948 rc 0/0 00000100:00100000:1.0:1789760779.873223:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15650 00000100:00100000:1.0:1789760779.873233:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723091032208 time 723106133516 state 0x2 ref 0 00000100:00100000:2.0:1789760779.882287:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.885494:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554768000 00000100:00100000:1.0:1789760779.885511:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554768000 00000100:00100000:1.0:1789760779.885696:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723091032208 time 723106133516 state 0x0 ref 1 00000100:00100000:1.0:1789760779.885722:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723106393932 time 723106133516 state 0x1 ref 1 00000100:00100000:1.0:1789760779.885753:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723106393932 time 723131371849 state 0x0 ref 1 00000100:00100000:1.0:1789760779.885770:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554768000:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760779.886270:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64061440, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e8d8 00010000:00010000:1.0:1789760779.886281:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55de lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.886298:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760779.886303:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.886309:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64061440->64065535] (req 64061440->64065535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.886324:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64061440->64065535] (req 64061440->64065535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.894985:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982949 is committed 00002000:00010000:1.0:1789760779.895223:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64061440 -> 64065536 00002000:00010000:1.0:1789760779.895228:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125144 -> 125152 00010000:00010000:1.0:1789760779.895242:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64061440->64065535] (req 64061440->64065535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.895256:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55de lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64061440->64065535] (req 64061440->64065535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.895265:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64061440->64065535] (req 64061440->64065535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.895273:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760779.895278:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64061440->64065535] (req 64061440->64065535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.895286:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55de lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64061440->64065535] (req 64061440->64065535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.895306:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55de lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64061440->64065535] (req 64061440->64065535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.895313:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c55de lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64061440->64065535] (req 64061440->64065535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.895643:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554768000:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9868us (13359us total) trans 4294982949 rc 0/0 00000100:00100000:1.0:1789760779.895656:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15651 00000100:00100000:1.0:1789760779.895667:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723106393932 time 723131371849 state 0x2 ref 0 00000100:00100000:3.0:1789760779.907259:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760779.909021:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554768128 00000100:00100000:2.0:1789760779.909034:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554768128 00000100:00100000:2.0:1789760779.909091:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723106393932 time 723131371849 state 0x0 ref 1 00000100:00100000:2.0:1789760779.909105:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723131632265 time 723131371849 state 0x1 ref 1 00000100:00100000:2.0:1789760779.909193:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723131632265 time 723154810983 state 0x0 ref 1 00000100:00100000:2.0:1789760779.909207:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06491180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554768128:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760779.909249:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64065536, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e8e0 00010000:00010000:2.0:1789760779.909259:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c55e5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.909285:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760779.909289:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760779.909296:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c55e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64065536->64069631] (req 64065536->64069631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.909311:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c55e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64065536->64069631] (req 64065536->64069631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.916003:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982950 is committed 00002000:00010000:2.0:1789760779.916270:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64065536 -> 64069632 00002000:00010000:2.0:1789760779.916275:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125152 -> 125160 00010000:00010000:2.0:1789760779.916288:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c55e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64065536->64069631] (req 64065536->64069631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.916299:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c55e5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64065536->64069631] (req 64065536->64069631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.916314:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c55e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64065536->64069631] (req 64065536->64069631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.916322:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760779.916327:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c55e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64065536->64069631] (req 64065536->64069631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.916336:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c55e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64065536->64069631] (req 64065536->64069631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.916356:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c55e5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64065536->64069631] (req 64065536->64069631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.916365:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c55e5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64065536->64069631] (req 64065536->64069631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.916489:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06491180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554768128:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7279us (9235us total) trans 4294982950 rc 0/0 00000100:00100000:2.0:1789760779.916499:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15652 00000100:00100000:2.0:1789760779.916510:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723131632265 time 723154810983 state 0x2 ref 0 00000100:00100000:0.0:1789760779.922317:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.923316:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554768256 00000100:00100000:1.0:1789760779.923327:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554768256 00000100:00100000:1.0:1789760779.923390:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723131632265 time 723154810983 state 0x0 ref 1 00000100:00100000:1.0:1789760779.923405:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723155071399 time 723154810983 state 0x1 ref 1 00000100:00100000:1.0:1789760779.923593:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723155071399 time 723169209587 state 0x0 ref 1 00000100:00100000:1.0:1789760779.923607:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554768256:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760779.923654:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64069632, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e8e8 00010000:00010000:1.0:1789760779.923663:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55ec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.923679:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760779.923684:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.923689:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64069632->64073727] (req 64069632->64073727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.923714:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64069632->64073727] (req 64069632->64073727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.930624:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982951 is committed 00002000:00010000:1.0:1789760779.930748:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64069632 -> 64073728 00002000:00010000:1.0:1789760779.930754:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125160 -> 125168 00010000:00010000:1.0:1789760779.930767:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64069632->64073727] (req 64069632->64073727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.930779:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55ec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64069632->64073727] (req 64069632->64073727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.930789:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64069632->64073727] (req 64069632->64073727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.930796:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760779.930800:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64069632->64073727] (req 64069632->64073727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.930815:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64069632->64073727] (req 64069632->64073727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.930835:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55ec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64069632->64073727] (req 64069632->64073727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.930842:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55ec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64069632->64073727] (req 64069632->64073727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760779.930942:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554768256:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7332us (8629us total) trans 4294982951 rc 0/0 00000100:00100000:1.0:1789760779.930953:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15653 00000100:00100000:1.0:1789760779.930963:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723155071399 time 723169209587 state 0x2 ref 0 00000100:00100000:1.0:1789760779.934135:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760779.934225:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554768384 00000100:00100000:0.0:1789760779.934234:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554768384 00000100:00100000:0.0:1789760779.934287:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723155071399 time 723169209587 state 0x0 ref 1 00000100:00100000:0.0:1789760779.934302:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723169470003 time 723169209587 state 0x1 ref 1 00000100:00100000:0.0:1789760779.934336:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723169470003 time 723179954146 state 0x0 ref 1 00000100:00100000:0.0:1789760779.934348:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554768384:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760779.934433:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64073728, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e8f0 00010000:00010000:0.0:1789760779.934444:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c55f3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.934468:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760779.934474:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.934480:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c55f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64073728->64077823] (req 64073728->64077823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.934494:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c55f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64073728->64077823] (req 64073728->64077823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760779.939855:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982952 is committed 00002000:00010000:0.0:1789760779.939964:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64073728 -> 64077824 00002000:00010000:0.0:1789760779.939969:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125168 -> 125176 00010000:00010000:0.0:1789760779.939982:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c55f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64073728->64077823] (req 64073728->64077823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.939996:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c55f3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64073728->64077823] (req 64073728->64077823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.940005:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c55f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64073728->64077823] (req 64073728->64077823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.940014:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760779.940019:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c55f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64073728->64077823] (req 64073728->64077823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.940029:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c55f3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64073728->64077823] (req 64073728->64077823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.940049:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c55f3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64073728->64077823] (req 64073728->64077823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.940063:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c55f3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64073728->64077823] (req 64073728->64077823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.940168:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554768384:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5817us (6038us total) trans 4294982952 rc 0/0 00000100:00100000:0.0:1789760779.940178:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15654 00000100:00100000:0.0:1789760779.940188:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723169470003 time 723179954146 state 0x2 ref 0 00000100:00100000:3.0:1789760779.943370:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760779.943572:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554768512 00000100:00100000:1.0:1789760779.943582:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554768512 00000100:00100000:1.0:1789760779.943631:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723169470003 time 723179954146 state 0x0 ref 1 00000100:00100000:1.0:1789760779.943645:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723180214562 time 723179954146 state 0x1 ref 1 00000100:00100000:1.0:1789760779.943674:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723180214562 time 723189292806 state 0x0 ref 1 00000100:00100000:1.0:1789760779.943685:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554768512:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760779.943739:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64077824, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e8f8 00010000:00010000:1.0:1789760779.943749:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55fa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.943763:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760779.943767:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760779.943773:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64077824->64081919] (req 64077824->64081919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760779.943786:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64077824->64081919] (req 64077824->64081919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760779.949992:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982953 is committed 00002000:00010000:3.0:1789760779.950158:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64077824 -> 64081920 00002000:00010000:3.0:1789760779.950163:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125176 -> 125184 00010000:00010000:3.0:1789760779.950174:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64077824->64081919] (req 64077824->64081919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.950183:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55fa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64077824->64081919] (req 64077824->64081919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.950189:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64077824->64081919] (req 64077824->64081919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.950194:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760779.950197:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64077824->64081919] (req 64077824->64081919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.950206:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55fa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64077824->64081919] (req 64077824->64081919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.950221:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55fa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64077824->64081919] (req 64077824->64081919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.950225:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c55fa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64077824->64081919] (req 64077824->64081919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760779.950297:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554768512:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6609us (6932us total) trans 4294982953 rc 0/0 00000100:00100000:3.0:1789760779.950308:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15655 00000100:00100000:3.0:1789760779.950318:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723180214562 time 723189292806 state 0x2 ref 0 00000100:00100000:2.0:1789760779.954841:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760779.954930:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554768640 00000100:00100000:3.0:1789760779.954940:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554768640 00000100:00100000:3.0:1789760779.954986:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723180214562 time 723189292806 state 0x0 ref 1 00000100:00100000:3.0:1789760779.954998:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723189553222 time 723189292806 state 0x1 ref 1 00000100:00100000:3.0:1789760779.955022:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723189553222 time 723200642337 state 0x0 ref 1 00000100:00100000:3.0:1789760779.955031:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554768640:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760779.955068:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64081920, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e900 00010000:00010000:3.0:1789760779.955076:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5601 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.955093:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760779.955098:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.955104:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5601 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64081920->64086015] (req 64081920->64086015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.955119:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5601 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64081920->64086015] (req 64081920->64086015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.961085:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982954 is committed 00002000:00010000:2.0:1789760779.961302:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64081920 -> 64086016 00002000:00010000:2.0:1789760779.961305:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125184 -> 125192 00010000:00010000:2.0:1789760779.961317:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5601 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64081920->64086015] (req 64081920->64086015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.961327:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5601 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64081920->64086015] (req 64081920->64086015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.961337:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5601 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64081920->64086015] (req 64081920->64086015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.961343:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760779.961346:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5601 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64081920->64086015] (req 64081920->64086015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.961352:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5601 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64081920->64086015] (req 64081920->64086015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.961366:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5601 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64081920->64086015] (req 64081920->64086015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760779.961374:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5601 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64081920->64086015] (req 64081920->64086015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760779.961653:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554768640:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6616us (6815us total) trans 4294982954 rc 0/0 00000100:00100000:2.0:1789760779.961664:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15656 00000100:00100000:2.0:1789760779.961672:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723189553222 time 723200642337 state 0x2 ref 0 00000100:00100000:1.0:1789760779.965592:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760779.965738:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554768768 00000100:00100000:0.0:1789760779.965747:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554768768 00000100:00100000:0.0:1789760779.965799:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723189553222 time 723200642337 state 0x0 ref 1 00000100:00100000:0.0:1789760779.965814:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723200902753 time 723200642337 state 0x1 ref 1 00000100:00100000:0.0:1789760779.965844:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723200902753 time 723211461767 state 0x0 ref 1 00000100:00100000:0.0:1789760779.965856:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554768768:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760779.965900:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64086016, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e908 00010000:00010000:0.0:1789760779.965908:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5608 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.965925:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760779.965930:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.965937:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5608 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64086016->64090111] (req 64086016->64090111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.965958:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5608 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64086016->64090111] (req 64086016->64090111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.971450:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982955 is committed 00002000:00010000:0.0:1789760779.971558:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64086016 -> 64090112 00002000:00010000:0.0:1789760779.971562:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125192 -> 125200 00010000:00010000:0.0:1789760779.971575:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5608 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64086016->64090111] (req 64086016->64090111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.971587:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5608 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64086016->64090111] (req 64086016->64090111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.971597:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5608 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64086016->64090111] (req 64086016->64090111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.971605:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760779.971610:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5608 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64086016->64090111] (req 64086016->64090111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.971620:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5608 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64086016->64090111] (req 64086016->64090111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.971711:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5608 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64086016->64090111] (req 64086016->64090111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.971721:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5608 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64086016->64090111] (req 64086016->64090111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.971819:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554768768:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5960us (6233us total) trans 4294982955 rc 0/0 00000100:00100000:0.0:1789760779.971829:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15657 00000100:00100000:0.0:1789760779.971839:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723200902753 time 723211461767 state 0x2 ref 0 00000100:00100000:3.0:1789760779.979254:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760779.979516:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554768896 00000100:00100000:0.0:1789760779.979526:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554768896 00000100:00100000:0.0:1789760779.979576:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723200902753 time 723211461767 state 0x0 ref 1 00000100:00100000:0.0:1789760779.979590:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723211722183 time 723211461767 state 0x1 ref 1 00000100:00100000:0.0:1789760779.979622:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723211722183 time 723225240608 state 0x0 ref 1 00000100:00100000:0.0:1789760779.979634:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06490700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554768896:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760779.979680:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64090112, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e910 00010000:00010000:0.0:1789760779.979689:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c560f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.979715:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760779.979720:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760779.979726:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c560f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64090112->64094207] (req 64090112->64094207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.979739:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c560f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64090112->64094207] (req 64090112->64094207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.984589:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982956 is committed 00002000:00010000:0.0:1789760779.984708:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64090112 -> 64094208 00002000:00010000:0.0:1789760779.984713:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125200 -> 125208 00010000:00010000:0.0:1789760779.984727:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c560f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64090112->64094207] (req 64090112->64094207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.984740:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c560f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64090112->64094207] (req 64090112->64094207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.984750:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c560f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64090112->64094207] (req 64090112->64094207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.984758:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760779.984763:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c560f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64090112->64094207] (req 64090112->64094207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.984772:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c560f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64090112->64094207] (req 64090112->64094207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.984790:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c560f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64090112->64094207] (req 64090112->64094207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760779.984799:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c560f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64090112->64094207] (req 64090112->64094207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760779.984999:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06490700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554768896:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5360us (5748us total) trans 4294982956 rc 0/0 00000100:00100000:0.0:1789760779.985010:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15658 00000100:00100000:0.0:1789760779.985020:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723211722183 time 723225240608 state 0x2 ref 0 00000100:00100000:2.0:1789760779.991654:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760779.991795:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554769024 00000100:00100000:3.0:1789760779.991819:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554769024 00000100:00100000:3.0:1789760779.991872:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723211722183 time 723225240608 state 0x0 ref 1 00000100:00100000:3.0:1789760779.991893:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723225501024 time 723225240608 state 0x1 ref 1 00000100:00100000:3.0:1789760779.991922:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723225501024 time 723237540650 state 0x0 ref 1 00000100:00100000:3.0:1789760779.991934:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554769024:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760779.991979:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64094208, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e918 00010000:00010000:3.0:1789760779.991988:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5616 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.992004:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760779.992010:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760779.992016:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5616 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64094208->64098303] (req 64094208->64098303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760779.992030:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5616 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64094208->64098303] (req 64094208->64098303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760779.996651:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982957 is committed 00002000:00010000:1.0:1789760780.001978:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64094208 -> 64098304 00002000:00010000:1.0:1789760780.001988:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125208 -> 125216 00010000:00010000:1.0:1789760780.002001:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5616 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64094208->64098303] (req 64094208->64098303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.002015:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5616 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64094208->64098303] (req 64094208->64098303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.002025:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5616 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64094208->64098303] (req 64094208->64098303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.002033:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760780.002038:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5616 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64094208->64098303] (req 64094208->64098303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.002054:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5616 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64094208->64098303] (req 64094208->64098303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.002149:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5616 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64094208->64098303] (req 64094208->64098303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.002159:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5616 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64094208->64098303] (req 64094208->64098303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.002316:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554769024:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10378us (10667us total) trans 4294982957 rc 0/0 00000100:00100000:1.0:1789760780.002327:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15659 00000100:00100000:1.0:1789760780.002336:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723225501024 time 723237540650 state 0x2 ref 0 00000100:00100000:1.0:1789760780.008549:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760780.008685:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554769152 00000100:00100000:2.0:1789760780.008704:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554769152 00000100:00100000:2.0:1789760780.008758:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723225501024 time 723237540650 state 0x0 ref 1 00000100:00100000:2.0:1789760780.008773:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723237801066 time 723237540650 state 0x1 ref 1 00000100:00100000:2.0:1789760780.008856:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723237801066 time 723254474021 state 0x0 ref 1 00000100:00100000:2.0:1789760780.008870:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554769152:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760780.008927:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64098304, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e920 00010000:00010000:2.0:1789760780.008936:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c561d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.008954:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760780.008958:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.008965:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c561d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64098304->64102399] (req 64098304->64102399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.008981:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c561d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64098304->64102399] (req 64098304->64102399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.015337:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982958 is committed 00002000:00010000:2.0:1789760780.015879:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64098304 -> 64102400 00002000:00010000:2.0:1789760780.015884:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125216 -> 125224 00010000:00010000:2.0:1789760780.015955:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c561d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64098304->64102399] (req 64098304->64102399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.015969:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c561d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64098304->64102399] (req 64098304->64102399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.015979:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c561d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64098304->64102399] (req 64098304->64102399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.015986:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760780.015991:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c561d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64098304->64102399] (req 64098304->64102399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.016000:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c561d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64098304->64102399] (req 64098304->64102399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.016125:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c561d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64098304->64102399] (req 64098304->64102399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.016135:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c561d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64098304->64102399] (req 64098304->64102399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.016267:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554769152:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7393us (7722us total) trans 4294982958 rc 0/0 00000100:00100000:2.0:1789760780.016278:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15660 00000100:00100000:2.0:1789760780.016287:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723237801066 time 723254474021 state 0x2 ref 0 00000100:00100000:3.0:1789760780.023695:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760780.025389:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554769280 00000100:00100000:0.0:1789760780.025401:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554769280 00000100:00100000:0.0:1789760780.025519:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723237801066 time 723254474021 state 0x0 ref 1 00000100:00100000:0.0:1789760780.025536:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723254734437 time 723254474021 state 0x1 ref 1 00000100:00100000:0.0:1789760780.025605:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723254734437 time 723271222544 state 0x0 ref 1 00000100:00100000:0.0:1789760780.025619:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06493800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554769280:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760780.025663:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64102400, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e928 00010000:00010000:0.0:1789760780.025672:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5624 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.025688:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760780.025692:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.025707:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5624 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64102400->64106495] (req 64102400->64106495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.025722:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5624 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64102400->64106495] (req 64102400->64106495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.030180:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982959 is committed 00002000:00010000:0.0:1789760780.030332:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64102400 -> 64106496 00002000:00010000:0.0:1789760780.030337:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125224 -> 125232 00010000:00010000:0.0:1789760780.030351:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5624 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64102400->64106495] (req 64102400->64106495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.030364:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5624 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64102400->64106495] (req 64102400->64106495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.030373:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5624 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64102400->64106495] (req 64102400->64106495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.030404:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760780.030409:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5624 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64102400->64106495] (req 64102400->64106495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.030419:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5624 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64102400->64106495] (req 64102400->64106495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.030443:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5624 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64102400->64106495] (req 64102400->64106495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.030451:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5624 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64102400->64106495] (req 64102400->64106495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.030819:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06493800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554769280:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5198us (7127us total) trans 4294982959 rc 0/0 00000100:00100000:0.0:1789760780.030829:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15661 00000100:00100000:0.0:1789760780.030840:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723254734437 time 723271222544 state 0x2 ref 0 00000100:00100000:0.0:1789760780.039781:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760780.040082:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554769408 00000100:00100000:1.0:1789760780.040092:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554769408 00000100:00100000:1.0:1789760780.040144:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723254734437 time 723271222544 state 0x0 ref 1 00000100:00100000:1.0:1789760780.040160:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723271482960 time 723271222544 state 0x1 ref 1 00000100:00100000:1.0:1789760780.040230:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723271482960 time 723285848308 state 0x0 ref 1 00000100:00100000:1.0:1789760780.040244:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554769408:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760780.040286:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64106496, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e930 00010000:00010000:1.0:1789760780.040296:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c562b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.040311:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760780.040316:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.040322:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c562b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64106496->64110591] (req 64106496->64110591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.040336:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c562b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64106496->64110591] (req 64106496->64110591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.048923:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982960 is committed 00002000:00010000:1.0:1789760780.049089:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64106496 -> 64110592 00002000:00010000:1.0:1789760780.049094:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125232 -> 125240 00010000:00010000:1.0:1789760780.049107:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c562b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64106496->64110591] (req 64106496->64110591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.049120:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c562b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64106496->64110591] (req 64106496->64110591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.049129:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c562b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64106496->64110591] (req 64106496->64110591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.049137:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760780.049142:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c562b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64106496->64110591] (req 64106496->64110591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.049153:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c562b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64106496->64110591] (req 64106496->64110591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.049174:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c562b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64106496->64110591] (req 64106496->64110591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.049183:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c562b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64106496->64110591] (req 64106496->64110591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.049270:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554769408:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9024us (9494us total) trans 4294982960 rc 0/0 00000100:00100000:1.0:1789760780.049281:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15662 00000100:00100000:1.0:1789760780.049290:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723271482960 time 723285848308 state 0x2 ref 0 00000100:00100000:1.0:1789760780.054245:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760780.054319:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554769536 00000100:00100000:3.0:1789760780.054329:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554769536 00000100:00100000:3.0:1789760780.054419:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723271482960 time 723285848308 state 0x0 ref 1 00000100:00100000:3.0:1789760780.054435:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723286108724 time 723285848308 state 0x1 ref 1 00000100:00100000:3.0:1789760780.054456:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723286108724 time 723300074974 state 0x0 ref 1 00000100:00100000:3.0:1789760780.054467:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554769536:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760780.054512:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64110592, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e938 00010000:00010000:3.0:1789760780.054523:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5632 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.054539:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760780.054544:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.054551:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5632 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64110592->64114687] (req 64110592->64114687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.054565:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5632 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64110592->64114687] (req 64110592->64114687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.063083:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982961 is committed 00002000:00010000:3.0:1789760780.063252:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64110592 -> 64114688 00002000:00010000:3.0:1789760780.063257:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125240 -> 125248 00010000:00010000:3.0:1789760780.063279:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5632 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64110592->64114687] (req 64110592->64114687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.063293:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5632 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64110592->64114687] (req 64110592->64114687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.063303:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5632 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64110592->64114687] (req 64110592->64114687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.063310:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760780.063314:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5632 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64110592->64114687] (req 64110592->64114687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.063323:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5632 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64110592->64114687] (req 64110592->64114687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.063343:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5632 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64110592->64114687] (req 64110592->64114687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.063356:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5632 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64110592->64114687] (req 64110592->64114687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.063481:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554769536:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9009us (9239us total) trans 4294982961 rc 0/0 00000100:00100000:3.0:1789760780.063490:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15663 00000100:00100000:3.0:1789760780.063500:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723286108724 time 723300074974 state 0x2 ref 0 00000100:00100000:3.0:1789760780.068577:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760780.068636:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554769664 00000100:00100000:2.0:1789760780.068647:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554769664 00000100:00100000:2.0:1789760780.068709:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723286108724 time 723300074974 state 0x0 ref 1 00000100:00100000:2.0:1789760780.068729:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723300335390 time 723300074974 state 0x1 ref 1 00000100:00100000:2.0:1789760780.068763:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723300335390 time 723314381868 state 0x0 ref 1 00000100:00100000:2.0:1789760780.068775:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06490a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554769664:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760780.068823:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64114688, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e940 00010000:00010000:2.0:1789760780.068832:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5639 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.068850:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760780.068855:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.068861:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5639 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64114688->64118783] (req 64114688->64118783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.068875:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5639 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64114688->64118783] (req 64114688->64118783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.074459:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982962 is committed 00002000:00010000:2.0:1789760780.074875:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64114688 -> 64118784 00002000:00010000:2.0:1789760780.074881:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125248 -> 125256 00010000:00010000:2.0:1789760780.074893:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5639 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64114688->64118783] (req 64114688->64118783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.074907:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5639 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64114688->64118783] (req 64114688->64118783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.074916:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5639 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64114688->64118783] (req 64114688->64118783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.074923:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760780.074928:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5639 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64114688->64118783] (req 64114688->64118783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.074937:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5639 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64114688->64118783] (req 64114688->64118783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.074956:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5639 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64114688->64118783] (req 64114688->64118783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.074965:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5639 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64114688->64118783] (req 64114688->64118783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.075244:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06490a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554769664:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6464us (6669us total) trans 4294982962 rc 0/0 00000100:00100000:2.0:1789760780.075256:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15664 00000100:00100000:2.0:1789760780.075266:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723300335390 time 723314381868 state 0x2 ref 0 00000100:00100000:0.0:1789760780.081638:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760780.082042:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554769792 00000100:00100000:1.0:1789760780.082052:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554769792 00000100:00100000:1.0:1789760780.082102:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723300335390 time 723314381868 state 0x0 ref 1 00000100:00100000:1.0:1789760780.082117:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723314642284 time 723314381868 state 0x1 ref 1 00000100:00100000:1.0:1789760780.082145:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723314642284 time 723327763788 state 0x0 ref 1 00000100:00100000:1.0:1789760780.082156:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554769792:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760780.082203:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64118784, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e948 00010000:00010000:1.0:1789760780.082212:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5640 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.082229:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760780.082233:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.082239:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5640 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64118784->64122879] (req 64118784->64122879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.082252:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5640 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64118784->64122879] (req 64118784->64122879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.091025:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982963 is committed 00002000:00010000:1.0:1789760780.091193:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64118784 -> 64122880 00002000:00010000:1.0:1789760780.091198:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125256 -> 125264 00010000:00010000:1.0:1789760780.091211:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5640 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64118784->64122879] (req 64118784->64122879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.091224:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5640 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64118784->64122879] (req 64118784->64122879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.091234:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5640 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64118784->64122879] (req 64118784->64122879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.091243:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760780.091248:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5640 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64118784->64122879] (req 64118784->64122879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.091257:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5640 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64118784->64122879] (req 64118784->64122879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.091283:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5640 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64118784->64122879] (req 64118784->64122879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.091292:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5640 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64118784->64122879] (req 64118784->64122879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.091517:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554769792:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9358us (9885us total) trans 4294982963 rc 0/0 00000100:00100000:1.0:1789760780.091529:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15665 00000100:00100000:1.0:1789760780.091539:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723314642284 time 723327763788 state 0x2 ref 0 00000100:00100000:1.0:1789760780.096685:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760780.096776:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554769920 00000100:00100000:0.0:1789760780.096786:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554769920 00000100:00100000:0.0:1789760780.096836:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723314642284 time 723327763788 state 0x0 ref 1 00000100:00100000:0.0:1789760780.096851:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723328024204 time 723327763788 state 0x1 ref 1 00000100:00100000:0.0:1789760780.096884:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723328024204 time 723342502154 state 0x0 ref 1 00000100:00100000:0.0:1789760780.096895:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554769920:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760780.096943:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64122880, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e950 00010000:00010000:0.0:1789760780.096952:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c5647 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.096967:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760780.096971:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.096977:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c5647 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64122880->64126975] (req 64122880->64126975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.096991:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c5647 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64122880->64126975] (req 64122880->64126975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.102042:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982964 is committed 00002000:00010000:0.0:1789760780.102209:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64122880 -> 64126976 00002000:00010000:0.0:1789760780.102214:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125264 -> 125272 00010000:00010000:0.0:1789760780.102229:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c5647 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64122880->64126975] (req 64122880->64126975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.102248:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c5647 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64122880->64126975] (req 64122880->64126975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.102258:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c5647 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64122880->64126975] (req 64122880->64126975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.102265:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760780.102270:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c5647 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64122880->64126975] (req 64122880->64126975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.102279:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c5647 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64122880->64126975] (req 64122880->64126975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.102298:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c5647 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64122880->64126975] (req 64122880->64126975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.102306:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c5647 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64122880->64126975] (req 64122880->64126975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.102487:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554769920:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5588us (5809us total) trans 4294982964 rc 0/0 00000100:00100000:0.0:1789760780.102498:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15666 00000100:00100000:0.0:1789760780.102507:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723328024204 time 723342502154 state 0x2 ref 0 00000100:00100000:2.0:1789760780.106842:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760780.106931:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554770048 00000100:00100000:3.0:1789760780.106940:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554770048 00000100:00100000:3.0:1789760780.106995:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723328024204 time 723342502154 state 0x0 ref 1 00000100:00100000:3.0:1789760780.107010:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723342762570 time 723342502154 state 0x1 ref 1 00000100:00100000:3.0:1789760780.107085:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723342762570 time 723352694244 state 0x0 ref 1 00000100:00100000:3.0:1789760780.107098:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554770048:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760780.107144:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64126976, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e958 00010000:00010000:3.0:1789760780.107154:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c564e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.107170:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760780.107176:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.107182:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c564e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64126976->64131071] (req 64126976->64131071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.107198:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c564e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64126976->64131071] (req 64126976->64131071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.114079:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982965 is committed 00002000:00010000:3.0:1789760780.114257:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64126976 -> 64131072 00002000:00010000:3.0:1789760780.114262:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125272 -> 125280 00010000:00010000:3.0:1789760780.114275:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c564e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64126976->64131071] (req 64126976->64131071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.114288:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c564e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64126976->64131071] (req 64126976->64131071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.114297:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c564e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64126976->64131071] (req 64126976->64131071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.114304:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760780.114309:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c564e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64126976->64131071] (req 64126976->64131071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.114317:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c564e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64126976->64131071] (req 64126976->64131071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.114336:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c564e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64126976->64131071] (req 64126976->64131071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.114347:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c564e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64126976->64131071] (req 64126976->64131071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.114689:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554770048:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7587us (7850us total) trans 4294982965 rc 0/0 00000100:00100000:3.0:1789760780.114712:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15667 00000100:00100000:3.0:1789760780.114722:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723342762570 time 723352694244 state 0x2 ref 0 00000100:00100000:0.0:1789760780.119327:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760780.119954:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554770176 00000100:00100000:3.0:1789760780.119964:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554770176 00000100:00100000:3.0:1789760780.120018:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723342762570 time 723352694244 state 0x0 ref 1 00000100:00100000:3.0:1789760780.120033:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723352954660 time 723352694244 state 0x1 ref 1 00000100:00100000:3.0:1789760780.120054:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723352954660 time 723365673056 state 0x0 ref 1 00000100:00100000:3.0:1789760780.120065:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554770176:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760780.120175:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64131072, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e960 00010000:00010000:3.0:1789760780.120185:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5655 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.120200:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760780.120205:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.120212:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5655 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64131072->64135167] (req 64131072->64135167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.120231:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5655 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64131072->64135167] (req 64131072->64135167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.127282:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982966 is committed 00002000:00010000:0.0:1789760780.127368:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64131072 -> 64135168 00002000:00010000:0.0:1789760780.127374:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125280 -> 125288 00010000:00010000:0.0:1789760780.127411:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5655 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64131072->64135167] (req 64131072->64135167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.127425:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5655 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64131072->64135167] (req 64131072->64135167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.127435:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5655 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64131072->64135167] (req 64131072->64135167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.127442:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760780.127447:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5655 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64131072->64135167] (req 64131072->64135167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.127456:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5655 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64131072->64135167] (req 64131072->64135167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.127475:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5655 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64131072->64135167] (req 64131072->64135167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.127483:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5655 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64131072->64135167] (req 64131072->64135167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.127757:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554770176:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7688us (8432us total) trans 4294982966 rc 0/0 00000100:00100000:0.0:1789760780.127768:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15668 00000100:00100000:0.0:1789760780.127780:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723352954660 time 723365673056 state 0x2 ref 0 00000100:00100000:1.0:1789760780.133429:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760780.133568:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554770304 00000100:00100000:0.0:1789760780.133577:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554770304 00000100:00100000:0.0:1789760780.133629:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723352954660 time 723365673056 state 0x0 ref 1 00000100:00100000:0.0:1789760780.133644:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723365933472 time 723365673056 state 0x1 ref 1 00000100:00100000:0.0:1789760780.133673:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723365933472 time 723379292025 state 0x0 ref 1 00000100:00100000:0.0:1789760780.133694:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554770304:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760780.133749:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64135168, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e968 00010000:00010000:0.0:1789760780.133758:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c565c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.133774:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760780.133778:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.133784:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c565c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64135168->64139263] (req 64135168->64139263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.133798:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c565c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64135168->64139263] (req 64135168->64139263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.140317:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982967 is committed 00002000:00010000:0.0:1789760780.140633:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64135168 -> 64139264 00002000:00010000:0.0:1789760780.140639:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125288 -> 125296 00010000:00010000:0.0:1789760780.140652:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c565c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64135168->64139263] (req 64135168->64139263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.140669:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c565c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64135168->64139263] (req 64135168->64139263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.140679:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c565c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64135168->64139263] (req 64135168->64139263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.140686:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760780.140691:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c565c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64135168->64139263] (req 64135168->64139263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.142813:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c565c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64135168->64139263] (req 64135168->64139263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.142838:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c565c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64135168->64139263] (req 64135168->64139263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.142848:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c565c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64135168->64139263] (req 64135168->64139263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.142953:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554770304:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9256us (9528us total) trans 4294982967 rc 0/0 00000100:00100000:0.0:1789760780.142964:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15669 00000100:00100000:0.0:1789760780.142973:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723365933472 time 723379292025 state 0x2 ref 0 00000100:00100000:2.0:1789760780.152351:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760780.152509:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554770432 00000100:00100000:0.0:1789760780.152519:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554770432 00000100:00100000:0.0:1789760780.152572:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723365933472 time 723379292025 state 0x0 ref 1 00000100:00100000:0.0:1789760780.152591:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723379552441 time 723379292025 state 0x1 ref 1 00000100:00100000:0.0:1789760780.152623:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723379552441 time 723398241613 state 0x0 ref 1 00000100:00100000:0.0:1789760780.152637:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554770432:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760780.152681:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64139264, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e970 00010000:00010000:0.0:1789760780.152690:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c5663 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.152717:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760780.152722:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.152730:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c5663 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64139264->64143359] (req 64139264->64143359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.152744:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c5663 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64139264->64143359] (req 64139264->64143359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.158756:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982968 is committed 00002000:00010000:2.0:1789760780.159197:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64139264 -> 64143360 00002000:00010000:2.0:1789760780.159202:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125296 -> 125304 00010000:00010000:2.0:1789760780.159215:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c5663 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64139264->64143359] (req 64139264->64143359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.159228:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c5663 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64139264->64143359] (req 64139264->64143359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.159237:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c5663 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64139264->64143359] (req 64139264->64143359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.159244:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760780.159249:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c5663 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64139264->64143359] (req 64139264->64143359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.159258:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c5663 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64139264->64143359] (req 64139264->64143359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.159276:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c5663 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64139264->64143359] (req 64139264->64143359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.159290:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c5663 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64139264->64143359] (req 64139264->64143359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.161045:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554770432:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8402us (8693us total) trans 4294982968 rc 0/0 00000100:00100000:2.0:1789760780.161058:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15670 00000100:00100000:2.0:1789760780.161067:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723379552441 time 723398241613 state 0x2 ref 0 00000100:00100000:2.0:1789760780.168167:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760780.168305:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554770560 00000100:00100000:3.0:1789760780.168315:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554770560 00000100:00100000:3.0:1789760780.168368:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723379552441 time 723398241613 state 0x0 ref 1 00000100:00100000:3.0:1789760780.168397:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723398502029 time 723398241613 state 0x1 ref 1 00000100:00100000:3.0:1789760780.168417:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723398502029 time 723414036254 state 0x0 ref 1 00000100:00100000:3.0:1789760780.168428:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554770560:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760780.168479:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64143360, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e978 00010000:00010000:3.0:1789760780.168489:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c566a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.168505:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760780.168509:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.168515:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c566a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64143360->64147455] (req 64143360->64147455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.168529:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c566a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64143360->64147455] (req 64143360->64147455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760780.173860:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982969 is committed 00002000:00010000:3.0:1789760780.177176:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64143360 -> 64147456 00002000:00010000:3.0:1789760780.177186:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125304 -> 125312 00010000:00010000:3.0:1789760780.177200:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c566a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64143360->64147455] (req 64143360->64147455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.177340:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c566a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64143360->64147455] (req 64143360->64147455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.177351:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c566a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64143360->64147455] (req 64143360->64147455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.177358:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760780.177363:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c566a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64143360->64147455] (req 64143360->64147455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.177372:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c566a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64143360->64147455] (req 64143360->64147455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.177537:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c566a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64143360->64147455] (req 64143360->64147455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.177545:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c566a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64143360->64147455] (req 64143360->64147455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.177753:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554770560:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9321us (9588us total) trans 4294982969 rc 0/0 00000100:00100000:3.0:1789760780.177764:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15671 00000100:00100000:3.0:1789760780.177774:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723398502029 time 723414036254 state 0x2 ref 0 00000100:00100000:1.0:1789760780.183560:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760780.183638:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554770688 00000100:00100000:2.0:1789760780.183647:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554770688 00000100:00100000:2.0:1789760780.183694:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723398502029 time 723414036254 state 0x0 ref 1 00000100:00100000:2.0:1789760780.183721:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723414296670 time 723414036254 state 0x1 ref 1 00000100:00100000:2.0:1789760780.183822:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723414296670 time 723429438770 state 0x0 ref 1 00000100:00100000:2.0:1789760780.183835:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554770688:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760780.183882:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64147456, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e980 00010000:00010000:2.0:1789760780.183892:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5671 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.183907:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760780.183911:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.183917:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5671 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64147456->64151551] (req 64147456->64151551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.183931:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5671 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64147456->64151551] (req 64147456->64151551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.188432:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982970 is committed 00002000:00010000:2.0:1789760780.188557:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64147456 -> 64151552 00002000:00010000:2.0:1789760780.188562:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125312 -> 125320 00010000:00010000:2.0:1789760780.188574:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5671 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64147456->64151551] (req 64147456->64151551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.188587:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5671 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64147456->64151551] (req 64147456->64151551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.188596:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5671 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64147456->64151551] (req 64147456->64151551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.188603:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760780.188608:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5671 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64147456->64151551] (req 64147456->64151551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.188617:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5671 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64147456->64151551] (req 64147456->64151551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.188636:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5671 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64147456->64151551] (req 64147456->64151551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.188644:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5671 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64147456->64151551] (req 64147456->64151551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.188746:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554770688:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4908us (5191us total) trans 4294982970 rc 0/0 00000100:00100000:2.0:1789760780.188756:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15672 00000100:00100000:2.0:1789760780.188765:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723414296670 time 723429438770 state 0x2 ref 0 00000100:00100000:2.0:1789760780.192795:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760780.193078:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554770816 00000100:00100000:0.0:1789760780.193088:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554770816 00000100:00100000:0.0:1789760780.193142:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723414296670 time 723429438770 state 0x0 ref 1 00000100:00100000:0.0:1789760780.193157:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723429699186 time 723429438770 state 0x1 ref 1 00000100:00100000:0.0:1789760780.193191:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723429699186 time 723438808469 state 0x0 ref 1 00000100:00100000:0.0:1789760780.193206:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554770816:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760780.193250:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64151552, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e988 00010000:00010000:0.0:1789760780.193267:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5678 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.193284:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760780.193288:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.193295:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5678 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64151552->64155647] (req 64151552->64155647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.193309:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5678 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64151552->64155647] (req 64151552->64155647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.199326:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982971 is committed 00002000:00010000:0.0:1789760780.199584:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64151552 -> 64155648 00002000:00010000:0.0:1789760780.199588:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125320 -> 125328 00010000:00010000:0.0:1789760780.199600:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5678 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64151552->64155647] (req 64151552->64155647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.199614:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5678 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64151552->64155647] (req 64151552->64155647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.199624:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5678 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64151552->64155647] (req 64151552->64155647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.199631:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760780.199637:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5678 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64151552->64155647] (req 64151552->64155647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.199647:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5678 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64151552->64155647] (req 64151552->64155647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.199664:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5678 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64151552->64155647] (req 64151552->64155647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.199672:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5678 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64151552->64155647] (req 64151552->64155647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.199793:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554770816:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6578us (6997us total) trans 4294982971 rc 0/0 00000100:00100000:0.0:1789760780.199804:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15673 00000100:00100000:0.0:1789760780.199813:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723429699186 time 723438808469 state 0x2 ref 0 00000100:00100000:0.0:1789760780.205185:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760780.208552:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554770944 00000100:00100000:1.0:1789760780.208566:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554770944 00000100:00100000:1.0:1789760780.208639:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723429699186 time 723438808469 state 0x0 ref 1 00000100:00100000:1.0:1789760780.208652:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723439068885 time 723438808469 state 0x1 ref 1 00000100:00100000:1.0:1789760780.208682:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723439068885 time 723454301093 state 0x0 ref 1 00000100:00100000:1.0:1789760780.208710:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554770944:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760780.208758:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64155648, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e990 00010000:00010000:1.0:1789760780.208767:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c567f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.208882:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760780.208888:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.208895:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c567f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64155648->64159743] (req 64155648->64159743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.208910:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c567f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64155648->64159743] (req 64155648->64159743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.218607:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982972 is committed 00002000:00010000:1.0:1789760780.218733:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64155648 -> 64159744 00002000:00010000:1.0:1789760780.218738:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125328 -> 125336 00010000:00010000:1.0:1789760780.218750:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c567f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64155648->64159743] (req 64155648->64159743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.218763:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c567f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64155648->64159743] (req 64155648->64159743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.218772:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c567f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64155648->64159743] (req 64155648->64159743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.218779:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760780.218784:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c567f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64155648->64159743] (req 64155648->64159743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.218793:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c567f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64155648->64159743] (req 64155648->64159743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.218811:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c567f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64155648->64159743] (req 64155648->64159743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.218818:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c567f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64155648->64159743] (req 64155648->64159743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.218903:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554770944:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10202us (13721us total) trans 4294982972 rc 0/0 00000100:00100000:1.0:1789760780.218916:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15674 00000100:00100000:1.0:1789760780.218926:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723439068885 time 723454301093 state 0x2 ref 0 00000100:00100000:1.0:1789760780.223350:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760780.223750:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554771072 00000100:00100000:3.0:1789760780.223761:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554771072 00000100:00100000:3.0:1789760780.223813:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723439068885 time 723454301093 state 0x0 ref 1 00000100:00100000:3.0:1789760780.223828:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723454561509 time 723454301093 state 0x1 ref 1 00000100:00100000:3.0:1789760780.223850:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723454561509 time 723469468620 state 0x0 ref 1 00000100:00100000:3.0:1789760780.223864:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554771072:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760780.223909:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64159744, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e998 00010000:00010000:3.0:1789760780.223918:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5686 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.223933:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760780.223937:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.223943:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5686 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64159744->64163839] (req 64159744->64163839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.223956:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5686 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64159744->64163839] (req 64159744->64163839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.230013:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982973 is committed 00002000:00010000:1.0:1789760780.230182:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64159744 -> 64163840 00002000:00010000:1.0:1789760780.230188:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125336 -> 125344 00010000:00010000:1.0:1789760780.230201:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5686 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64159744->64163839] (req 64159744->64163839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.230215:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5686 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64159744->64163839] (req 64159744->64163839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.230225:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5686 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64159744->64163839] (req 64159744->64163839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.230234:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760780.230239:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5686 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64159744->64163839] (req 64159744->64163839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.230248:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5686 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64159744->64163839] (req 64159744->64163839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.230267:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5686 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64159744->64163839] (req 64159744->64163839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.230275:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5686 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64159744->64163839] (req 64159744->64163839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.230587:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554771072:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6720us (7241us total) trans 4294982973 rc 0/0 00000100:00100000:1.0:1789760780.230599:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15675 00000100:00100000:1.0:1789760780.230608:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723454561509 time 723469468620 state 0x2 ref 0 00000100:00100000:2.0:1789760780.236765:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760780.236861:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554771200 00000100:00100000:0.0:1789760780.236927:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554771200 00000100:00100000:0.0:1789760780.236971:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723454561509 time 723469468620 state 0x0 ref 1 00000100:00100000:0.0:1789760780.236983:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723469729036 time 723469468620 state 0x1 ref 1 00000100:00100000:0.0:1789760780.237042:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723469729036 time 723482661728 state 0x0 ref 1 00000100:00100000:0.0:1789760780.237052:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554771200:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760780.237097:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64163840, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e9a0 00010000:00010000:0.0:1789760780.237106:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c568d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.237121:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760780.237126:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.237132:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c568d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64163840->64167935] (req 64163840->64167935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.237146:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c568d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64163840->64167935] (req 64163840->64167935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.242552:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982974 is committed 00002000:00010000:0.0:1789760780.242821:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64163840 -> 64167936 00002000:00010000:0.0:1789760780.242826:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125344 -> 125352 00010000:00010000:0.0:1789760780.242849:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c568d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64163840->64167935] (req 64163840->64167935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.242862:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c568d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64163840->64167935] (req 64163840->64167935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.242872:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c568d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64163840->64167935] (req 64163840->64167935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.242879:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760780.242884:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c568d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64163840->64167935] (req 64163840->64167935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.242899:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c568d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64163840->64167935] (req 64163840->64167935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.243025:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c568d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64163840->64167935] (req 64163840->64167935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.243035:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c568d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64163840->64167935] (req 64163840->64167935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.243185:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554771200:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6128us (6424us total) trans 4294982974 rc 0/0 00000100:00100000:0.0:1789760780.243196:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15676 00000100:00100000:0.0:1789760780.243205:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723469729036 time 723482661728 state 0x2 ref 0 00000100:00100000:0.0:1789760780.248232:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760780.248371:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554771328 00000100:00100000:1.0:1789760780.248405:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554771328 00000100:00100000:1.0:1789760780.248460:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723469729036 time 723482661728 state 0x0 ref 1 00000100:00100000:1.0:1789760780.248477:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723482922144 time 723482661728 state 0x1 ref 1 00000100:00100000:1.0:1789760780.248522:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723482922144 time 723494134835 state 0x0 ref 1 00000100:00100000:1.0:1789760780.248541:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554771328:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760780.248599:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64167936, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e9a8 00010000:00010000:1.0:1789760780.248609:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5694 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.248634:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760780.248639:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.248646:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5694 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64167936->64172031] (req 64167936->64172031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.248659:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5694 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64167936->64172031] (req 64167936->64172031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.254669:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982975 is committed 00002000:00010000:1.0:1789760780.254916:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64167936 -> 64172032 00002000:00010000:1.0:1789760780.254921:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125352 -> 125360 00010000:00010000:1.0:1789760780.254934:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5694 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64167936->64172031] (req 64167936->64172031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.254948:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5694 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64167936->64172031] (req 64167936->64172031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.254956:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5694 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64167936->64172031] (req 64167936->64172031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.254964:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760780.254970:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5694 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64167936->64172031] (req 64167936->64172031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.254980:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5694 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64167936->64172031] (req 64167936->64172031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.254998:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5694 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64167936->64172031] (req 64167936->64172031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.255006:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5694 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64167936->64172031] (req 64167936->64172031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.255104:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554771328:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6560us (6876us total) trans 4294982975 rc 0/0 00000100:00100000:1.0:1789760780.255115:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15677 00000100:00100000:1.0:1789760780.255123:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723482922144 time 723494134835 state 0x2 ref 0 00000100:00100000:1.0:1789760780.262109:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760780.262245:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554771456 00000100:00100000:2.0:1789760780.262254:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554771456 00000100:00100000:2.0:1789760780.262304:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723482922144 time 723494134835 state 0x0 ref 1 00000100:00100000:2.0:1789760780.262319:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723494395251 time 723494134835 state 0x1 ref 1 00000100:00100000:2.0:1789760780.262349:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723494395251 time 723507968169 state 0x0 ref 1 00000100:00100000:2.0:1789760780.262362:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554771456:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760780.262535:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64172032, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e9b0 00010000:00010000:2.0:1789760780.262545:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c569b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.262561:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760780.262566:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.262573:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c569b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64172032->64176127] (req 64172032->64176127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.262591:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c569b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64172032->64176127] (req 64172032->64176127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.268307:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982976 is committed 00002000:00010000:2.0:1789760780.268583:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64172032 -> 64176128 00002000:00010000:2.0:1789760780.268587:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125360 -> 125368 00010000:00010000:2.0:1789760780.268600:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c569b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64172032->64176127] (req 64172032->64176127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.268612:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c569b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64172032->64176127] (req 64172032->64176127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.268620:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c569b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64172032->64176127] (req 64172032->64176127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.268626:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760780.268631:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c569b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64172032->64176127] (req 64172032->64176127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.268639:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c569b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64172032->64176127] (req 64172032->64176127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.268657:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c569b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64172032->64176127] (req 64172032->64176127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.268664:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c569b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64172032->64176127] (req 64172032->64176127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.268763:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554771456:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6399us (6658us total) trans 4294982976 rc 0/0 00000100:00100000:2.0:1789760780.268772:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15678 00000100:00100000:2.0:1789760780.268781:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723494395251 time 723507968169 state 0x2 ref 0 00000100:00100000:2.0:1789760780.272678:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760780.272838:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554771584 00000100:00100000:3.0:1789760780.272849:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554771584 00000100:00100000:3.0:1789760780.272914:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723494395251 time 723507968169 state 0x0 ref 1 00000100:00100000:3.0:1789760780.272929:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723508228585 time 723507968169 state 0x1 ref 1 00000100:00100000:3.0:1789760780.272963:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723508228585 time 723518581912 state 0x0 ref 1 00000100:00100000:3.0:1789760780.272976:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554771584:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760780.273021:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64176128, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e9b8 00010000:00010000:3.0:1789760780.273030:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56a2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.273047:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760780.273051:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.273059:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64176128->64180223] (req 64176128->64180223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.273083:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64176128->64180223] (req 64176128->64180223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.280001:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982977 is committed 00002000:00010000:3.0:1789760780.280216:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64176128 -> 64180224 00002000:00010000:3.0:1789760780.280221:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125368 -> 125376 00010000:00010000:3.0:1789760780.280234:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64176128->64180223] (req 64176128->64180223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.280246:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64176128->64180223] (req 64176128->64180223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.280256:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64176128->64180223] (req 64176128->64180223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.280263:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760780.280268:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64176128->64180223] (req 64176128->64180223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.280277:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56a2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64176128->64180223] (req 64176128->64180223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.280296:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56a2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64176128->64180223] (req 64176128->64180223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.280304:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56a2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64176128->64180223] (req 64176128->64180223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.280478:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554771584:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7499us (7804us total) trans 4294982977 rc 0/0 00000100:00100000:3.0:1789760780.280490:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15679 00000100:00100000:3.0:1789760780.280499:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723508228585 time 723518581912 state 0x2 ref 0 00000100:00100000:0.0:1789760780.284375:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760780.284593:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554771712 00000100:00100000:2.0:1789760780.284604:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554771712 00000100:00100000:2.0:1789760780.284671:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723508228585 time 723518581912 state 0x0 ref 1 00000100:00100000:2.0:1789760780.284687:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723518842328 time 723518581912 state 0x1 ref 1 00000100:00100000:2.0:1789760780.284727:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723518842328 time 723530345773 state 0x0 ref 1 00000100:00100000:2.0:1789760780.284855:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554771712:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760780.284904:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64180224, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e9c0 00010000:00010000:2.0:1789760780.284914:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c56a9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.284930:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760780.284935:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.284941:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c56a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64180224->64184319] (req 64180224->64184319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.284955:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c56a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64180224->64184319] (req 64180224->64184319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.291367:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982978 is committed 00002000:00010000:2.0:1789760780.291512:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64180224 -> 64184320 00002000:00010000:2.0:1789760780.291526:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125376 -> 125384 00010000:00010000:2.0:1789760780.291539:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c56a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64180224->64184319] (req 64180224->64184319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.291552:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c56a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64180224->64184319] (req 64180224->64184319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.291561:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c56a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64180224->64184319] (req 64180224->64184319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.291569:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760780.291575:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c56a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64180224->64184319] (req 64180224->64184319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.291584:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c56a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64180224->64184319] (req 64180224->64184319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.291605:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c56a9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64180224->64184319] (req 64180224->64184319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.291621:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c56a9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64180224->64184319] (req 64180224->64184319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.291725:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554771712:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6870us (7356us total) trans 4294982978 rc 0/0 00000100:00100000:2.0:1789760780.291804:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15680 00000100:00100000:2.0:1789760780.291814:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723518842328 time 723530345773 state 0x2 ref 0 00000100:00100000:1.0:1789760780.300829:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760780.301219:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554771840 00000100:00100000:0.0:1789760780.301230:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554771840 00000100:00100000:0.0:1789760780.301291:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723518842328 time 723530345773 state 0x0 ref 1 00000100:00100000:0.0:1789760780.301307:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723530606189 time 723530345773 state 0x1 ref 1 00000100:00100000:0.0:1789760780.301339:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723530606189 time 723546957185 state 0x0 ref 1 00000100:00100000:0.0:1789760780.301351:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554771840:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760780.301444:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64184320, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e9c8 00010000:00010000:0.0:1789760780.301455:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c56b0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.301473:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760780.301478:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.301485:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c56b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64184320->64188415] (req 64184320->64188415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.301499:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c56b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64184320->64188415] (req 64184320->64188415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.307419:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982979 is committed 00002000:00010000:0.0:1789760780.307591:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64184320 -> 64188416 00002000:00010000:0.0:1789760780.307596:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125384 -> 125392 00010000:00010000:0.0:1789760780.307607:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c56b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64184320->64188415] (req 64184320->64188415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.307629:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c56b0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64184320->64188415] (req 64184320->64188415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.307639:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c56b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64184320->64188415] (req 64184320->64188415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.307646:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760780.307651:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c56b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64184320->64188415] (req 64184320->64188415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.307661:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c56b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64184320->64188415] (req 64184320->64188415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.307680:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c56b0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64184320->64188415] (req 64184320->64188415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.307688:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c56b0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64184320->64188415] (req 64184320->64188415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.307879:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554771840:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6525us (7055us total) trans 4294982979 rc 0/0 00000100:00100000:0.0:1789760780.307890:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15681 00000100:00100000:0.0:1789760780.307899:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723530606189 time 723546957185 state 0x2 ref 0 00000100:00100000:0.0:1789760780.312148:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760780.312298:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554771968 00000100:00100000:3.0:1789760780.312307:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554771968 00000100:00100000:3.0:1789760780.312360:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723530606189 time 723546957185 state 0x0 ref 1 00000100:00100000:3.0:1789760780.312410:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723547217601 time 723546957185 state 0x1 ref 1 00000100:00100000:3.0:1789760780.312440:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723547217601 time 723558058665 state 0x0 ref 1 00000100:00100000:3.0:1789760780.312462:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554771968:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760780.312506:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64188416, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e9d0 00010000:00010000:3.0:1789760780.312515:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.312530:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760780.312535:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.312542:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64188416->64192511] (req 64188416->64192511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.312556:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64188416->64192511] (req 64188416->64192511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.323276:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982980 is committed 00002000:00010000:3.0:1789760780.323710:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64188416 -> 64192512 00002000:00010000:3.0:1789760780.323716:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125392 -> 125400 00010000:00010000:3.0:1789760780.323836:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64188416->64192511] (req 64188416->64192511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.323858:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64188416->64192511] (req 64188416->64192511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.323868:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64188416->64192511] (req 64188416->64192511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.323875:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760780.323880:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64188416->64192511] (req 64188416->64192511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.323889:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56b7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64188416->64192511] (req 64188416->64192511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.323911:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56b7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64188416->64192511] (req 64188416->64192511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.323918:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56b7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64188416->64192511] (req 64188416->64192511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.324020:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554771968:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11564us (11876us total) trans 4294982980 rc 0/0 00000100:00100000:3.0:1789760780.324031:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15682 00000100:00100000:3.0:1789760780.324041:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723547217601 time 723558058665 state 0x2 ref 0 00000100:00100000:2.0:1789760780.333142:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760780.333245:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554772096 00000100:00100000:3.0:1789760780.333256:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554772096 00000100:00100000:3.0:1789760780.333311:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723547217601 time 723558058665 state 0x0 ref 1 00000100:00100000:3.0:1789760780.333327:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723558319081 time 723558058665 state 0x1 ref 1 00000100:00100000:0.0:1789760780.333538:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723558319081 time 723579152985 state 0x0 ref 1 00000100:00100000:0.0:1789760780.333554:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554772096:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760780.333648:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64192512, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e9d8 00010000:00010000:0.0:1789760780.333659:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c56be lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.333676:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760780.333933:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.333940:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c56be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64192512->64196607] (req 64192512->64196607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.333956:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c56be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64192512->64196607] (req 64192512->64196607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.341892:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982981 is committed 00002000:00010000:0.0:1789760780.342006:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64192512 -> 64196608 00002000:00010000:0.0:1789760780.342011:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125400 -> 125408 00010000:00010000:0.0:1789760780.342024:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c56be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64192512->64196607] (req 64192512->64196607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.342036:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c56be lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64192512->64196607] (req 64192512->64196607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.342045:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c56be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64192512->64196607] (req 64192512->64196607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.342053:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760780.342058:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c56be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64192512->64196607] (req 64192512->64196607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.342076:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c56be lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64192512->64196607] (req 64192512->64196607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.342096:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c56be lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64192512->64196607] (req 64192512->64196607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.342108:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c56be lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64192512->64196607] (req 64192512->64196607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.342199:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554772096:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8645us (9063us total) trans 4294982981 rc 0/0 00000100:00100000:0.0:1789760780.342210:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15683 00000100:00100000:0.0:1789760780.342219:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723558319081 time 723579152985 state 0x2 ref 0 00000100:00100000:1.0:1789760780.349244:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760780.349356:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554772224 00000100:00100000:2.0:1789760780.349366:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554772224 00000100:00100000:2.0:1789760780.349501:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723558319081 time 723579152985 state 0x0 ref 1 00000100:00100000:2.0:1789760780.349516:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723579413401 time 723579152985 state 0x1 ref 1 00000100:00100000:2.0:1789760780.349576:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723579413401 time 723595194144 state 0x0 ref 1 00000100:00100000:2.0:1789760780.349589:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe1b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554772224:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760780.349632:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64196608, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e9e0 00010000:00010000:2.0:1789760780.349641:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c56c5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.349656:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760780.349660:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.349666:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c56c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64196608->64200703] (req 64196608->64200703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.349679:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c56c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64196608->64200703] (req 64196608->64200703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.355298:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982982 is committed 00002000:00010000:2.0:1789760780.355652:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64196608 -> 64200704 00002000:00010000:2.0:1789760780.355658:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125408 -> 125416 00010000:00010000:2.0:1789760780.355671:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c56c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64196608->64200703] (req 64196608->64200703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.355684:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c56c5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64196608->64200703] (req 64196608->64200703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.355703:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c56c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64196608->64200703] (req 64196608->64200703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.355711:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760780.355715:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c56c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64196608->64200703] (req 64196608->64200703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.355724:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c56c5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64196608->64200703] (req 64196608->64200703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.355742:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c56c5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64196608->64200703] (req 64196608->64200703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.355750:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c56c5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64196608->64200703] (req 64196608->64200703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.355977:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe1b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554772224:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6384us (6736us total) trans 4294982982 rc 0/0 00000100:00100000:2.0:1789760780.355988:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15684 00000100:00100000:2.0:1789760780.355998:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723579413401 time 723595194144 state 0x2 ref 0 00000100:00100000:2.0:1789760780.365290:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760780.365362:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554772352 00000100:00100000:1.0:1789760780.365372:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554772352 00000100:00100000:1.0:1789760780.365459:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723579413401 time 723595194144 state 0x0 ref 1 00000100:00100000:1.0:1789760780.365475:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723595454560 time 723595194144 state 0x1 ref 1 00000100:00100000:1.0:1789760780.365507:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723595454560 time 723611125143 state 0x0 ref 1 00000100:00100000:1.0:1789760780.365524:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554772352:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760780.365575:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64200704, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e9e8 00010000:00010000:1.0:1789760780.365584:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c56cc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.365599:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760780.365603:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.365609:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c56cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64200704->64204799] (req 64200704->64204799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.365623:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c56cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64200704->64204799] (req 64200704->64204799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.371898:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982983 is committed 00002000:00010000:1.0:1789760780.373000:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64200704 -> 64204800 00002000:00010000:1.0:1789760780.373007:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125416 -> 125424 00010000:00010000:1.0:1789760780.373021:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c56cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64200704->64204799] (req 64200704->64204799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.373035:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c56cc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64200704->64204799] (req 64200704->64204799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.373045:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c56cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64200704->64204799] (req 64200704->64204799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.373052:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760780.373059:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c56cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64200704->64204799] (req 64200704->64204799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.373068:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c56cc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64200704->64204799] (req 64200704->64204799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.373088:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c56cc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64200704->64204799] (req 64200704->64204799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.373097:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c56cc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64200704->64204799] (req 64200704->64204799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.373198:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554772352:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7673us (7912us total) trans 4294982983 rc 0/0 00000100:00100000:1.0:1789760780.373209:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15685 00000100:00100000:1.0:1789760780.373218:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723595454560 time 723611125143 state 0x2 ref 0 00000100:00100000:0.0:1789760780.383792:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760780.385122:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554772480 00000100:00100000:3.0:1789760780.385135:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554772480 00000100:00100000:3.0:1789760780.385193:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723595454560 time 723611125143 state 0x0 ref 1 00000100:00100000:3.0:1789760780.385280:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723611385559 time 723611125143 state 0x1 ref 1 00000100:00100000:3.0:1789760780.385374:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723611385559 time 723630991535 state 0x0 ref 1 00000100:00100000:3.0:1789760780.385413:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554772480:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760780.385463:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64204800, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e9f0 00010000:00010000:3.0:1789760780.385472:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56d3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.385488:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760780.385493:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.385499:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64204800->64208895] (req 64204800->64208895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.385513:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64204800->64208895] (req 64204800->64208895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.390344:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982984 is committed 00002000:00010000:3.0:1789760780.390578:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64204800 -> 64208896 00002000:00010000:3.0:1789760780.390583:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125424 -> 125432 00010000:00010000:3.0:1789760780.390596:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64204800->64208895] (req 64204800->64208895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.390608:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56d3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64204800->64208895] (req 64204800->64208895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.390618:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64204800->64208895] (req 64204800->64208895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.390625:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760780.390630:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64204800->64208895] (req 64204800->64208895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.390639:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56d3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64204800->64208895] (req 64204800->64208895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.390663:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56d3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64204800->64208895] (req 64204800->64208895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.390671:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56d3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64204800->64208895] (req 64204800->64208895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.390808:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554772480:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5393us (7020us total) trans 4294982984 rc 0/0 00000100:00100000:3.0:1789760780.390820:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15686 00000100:00100000:3.0:1789760780.390828:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723611385559 time 723630991535 state 0x2 ref 0 00000100:00100000:2.0:1789760780.395579:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760780.395651:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554772608 00000100:00100000:0.0:1789760780.395662:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554772608 00000100:00100000:0.0:1789760780.395729:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723611385559 time 723630991535 state 0x0 ref 1 00000100:00100000:0.0:1789760780.395745:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723631251951 time 723630991535 state 0x1 ref 1 00000100:00100000:0.0:1789760780.395775:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723631251951 time 723641393376 state 0x0 ref 1 00000100:00100000:0.0:1789760780.395789:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554772608:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760780.395836:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64208896, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1e9f8 00010000:00010000:0.0:1789760780.395846:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c56da lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.395864:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760780.395870:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.395877:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c56da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64208896->64212991] (req 64208896->64212991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.395895:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c56da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64208896->64212991] (req 64208896->64212991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.400784:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982985 is committed 00002000:00010000:0.0:1789760780.401092:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64208896 -> 64212992 00002000:00010000:0.0:1789760780.401097:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125432 -> 125440 00010000:00010000:0.0:1789760780.401111:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c56da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64208896->64212991] (req 64208896->64212991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.401125:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c56da lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64208896->64212991] (req 64208896->64212991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.401134:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c56da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64208896->64212991] (req 64208896->64212991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.401142:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760780.401147:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c56da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64208896->64212991] (req 64208896->64212991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.401156:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c56da lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64208896->64212991] (req 64208896->64212991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.401176:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c56da lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64208896->64212991] (req 64208896->64212991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.401184:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c56da lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64208896->64212991] (req 64208896->64212991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.401313:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554772608:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5522us (5738us total) trans 4294982985 rc 0/0 00000100:00100000:0.0:1789760780.401325:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15687 00000100:00100000:0.0:1789760780.401337:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723631251951 time 723641393376 state 0x2 ref 0 00000100:00100000:2.0:1789760780.405491:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760780.405582:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554772736 00000100:00100000:0.0:1789760780.405592:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554772736 00000100:00100000:0.0:1789760780.405646:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723631251951 time 723641393376 state 0x0 ref 1 00000100:00100000:0.0:1789760780.405660:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723641653792 time 723641393376 state 0x1 ref 1 00000100:00100000:0.0:1789760780.405691:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723641653792 time 723651309725 state 0x0 ref 1 00000100:00100000:0.0:1789760780.405716:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554772736:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760780.405761:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64212992, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1ea00 00010000:00010000:0.0:1789760780.405770:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c56e1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.405786:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760780.405790:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.405796:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c56e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64212992->64217087] (req 64212992->64217087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.405809:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c56e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64212992->64217087] (req 64212992->64217087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.411974:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982986 is committed 00002000:00010000:0.0:1789760780.413346:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64212992 -> 64217088 00002000:00010000:0.0:1789760780.413353:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125440 -> 125448 00010000:00010000:0.0:1789760780.413367:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c56e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64212992->64217087] (req 64212992->64217087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.413442:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c56e1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64212992->64217087] (req 64212992->64217087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.413451:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c56e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64212992->64217087] (req 64212992->64217087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.413458:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760780.413463:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c56e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64212992->64217087] (req 64212992->64217087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.413473:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c56e1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64212992->64217087] (req 64212992->64217087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.413493:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c56e1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64212992->64217087] (req 64212992->64217087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.413501:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c56e1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64212992->64217087] (req 64212992->64217087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.413650:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554772736:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7930us (8162us total) trans 4294982986 rc 0/0 00000100:00100000:0.0:1789760780.413662:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15688 00000100:00100000:0.0:1789760780.413672:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723641653792 time 723651309725 state 0x2 ref 0 00000100:00100000:2.0:1789760780.418631:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760780.419070:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554772864 00000100:00100000:1.0:1789760780.419082:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554772864 00000100:00100000:1.0:1789760780.419136:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723641653792 time 723651309725 state 0x0 ref 1 00000100:00100000:1.0:1789760780.419152:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723651570141 time 723651309725 state 0x1 ref 1 00000100:00100000:1.0:1789760780.419336:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723651570141 time 723664953113 state 0x0 ref 1 00000100:00100000:1.0:1789760780.419349:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554772864:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760780.419416:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64217088, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1ea08 00010000:00010000:1.0:1789760780.419426:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c56e8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.419443:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760780.419447:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.419453:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c56e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64217088->64221183] (req 64217088->64221183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.419467:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c56e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64217088->64221183] (req 64217088->64221183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.425013:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982987 is committed 00002000:00010000:1.0:1789760780.426724:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64217088 -> 64221184 00002000:00010000:1.0:1789760780.426733:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125448 -> 125456 00010000:00010000:1.0:1789760780.426745:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c56e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64217088->64221183] (req 64217088->64221183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.426757:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c56e8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64217088->64221183] (req 64217088->64221183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.426766:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c56e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64217088->64221183] (req 64217088->64221183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.426773:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760780.426777:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c56e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64217088->64221183] (req 64217088->64221183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.426786:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c56e8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64217088->64221183] (req 64217088->64221183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.426807:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c56e8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64217088->64221183] (req 64217088->64221183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.426814:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c56e8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64217088->64221183] (req 64217088->64221183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.426918:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554772864:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7566us (8291us total) trans 4294982987 rc 0/0 00000100:00100000:1.0:1789760780.426929:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15689 00000100:00100000:1.0:1789760780.426938:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723651570141 time 723664953113 state 0x2 ref 0 00000100:00100000:2.0:1789760780.430925:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760780.431643:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554772992 00000100:00100000:3.0:1789760780.431654:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554772992 00000100:00100000:3.0:1789760780.431717:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723651570141 time 723664953113 state 0x0 ref 1 00000100:00100000:3.0:1789760780.431733:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723665213529 time 723664953113 state 0x1 ref 1 00000100:00100000:3.0:1789760780.431798:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723665213529 time 723677416251 state 0x0 ref 1 00000100:00100000:3.0:1789760780.431812:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554772992:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760780.431857:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64221184, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1ea10 00010000:00010000:3.0:1789760780.431867:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56ef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.431882:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760780.431895:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.431902:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64221184->64225279] (req 64221184->64225279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.431915:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64221184->64225279] (req 64221184->64225279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.436739:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982988 is committed 00002000:00010000:2.0:1789760780.436908:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64221184 -> 64225280 00002000:00010000:2.0:1789760780.436914:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125456 -> 125464 00010000:00010000:2.0:1789760780.436927:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64221184->64225279] (req 64221184->64225279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.436940:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56ef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64221184->64225279] (req 64221184->64225279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.436950:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64221184->64225279] (req 64221184->64225279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.436958:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760780.436963:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64221184->64225279] (req 64221184->64225279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.436978:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56ef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64221184->64225279] (req 64221184->64225279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.436999:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56ef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64221184->64225279] (req 64221184->64225279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.437007:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c56ef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64221184->64225279] (req 64221184->64225279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.437321:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554772992:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5506us (6401us total) trans 4294982988 rc 0/0 00000100:00100000:2.0:1789760780.437332:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15690 00000100:00100000:2.0:1789760780.437342:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723665213529 time 723677416251 state 0x2 ref 0 00000100:00100000:2.0:1789760780.441964:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760780.442105:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554773120 00000100:00100000:0.0:1789760780.442114:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554773120 00000100:00100000:0.0:1789760780.442170:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723665213529 time 723677416251 state 0x0 ref 1 00000100:00100000:0.0:1789760780.442185:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723677676667 time 723677416251 state 0x1 ref 1 00000100:00100000:0.0:1789760780.442218:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723677676667 time 723687835690 state 0x0 ref 1 00000100:00100000:0.0:1789760780.442230:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554773120:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760780.442275:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64225280, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1ea18 00010000:00010000:0.0:1789760780.442284:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c56f6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.442299:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760780.442303:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.442309:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c56f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64225280->64229375] (req 64225280->64229375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.442324:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c56f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64225280->64229375] (req 64225280->64229375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.446947:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982989 is committed 00002000:00010000:0.0:1789760780.447057:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64225280 -> 64229376 00002000:00010000:0.0:1789760780.447062:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125464 -> 125472 00010000:00010000:0.0:1789760780.447084:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c56f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64225280->64229375] (req 64225280->64229375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.447095:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c56f6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64225280->64229375] (req 64225280->64229375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.447103:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c56f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64225280->64229375] (req 64225280->64229375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.447110:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760780.447115:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c56f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64225280->64229375] (req 64225280->64229375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.447123:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c56f6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64225280->64229375] (req 64225280->64229375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.447144:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c56f6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64225280->64229375] (req 64225280->64229375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.447152:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c56f6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64225280->64229375] (req 64225280->64229375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.447290:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554773120:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5057us (5331us total) trans 4294982989 rc 0/0 00000100:00100000:0.0:1789760780.447301:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15691 00000100:00100000:0.0:1789760780.447310:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723677676667 time 723687835690 state 0x2 ref 0 00000100:00100000:0.0:1789760780.453297:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760780.453415:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554773248 00000100:00100000:2.0:1789760780.453424:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554773248 00000100:00100000:2.0:1789760780.453484:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723677676667 time 723687835690 state 0x0 ref 1 00000100:00100000:2.0:1789760780.453499:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723688096106 time 723687835690 state 0x1 ref 1 00000100:00100000:2.0:1789760780.453528:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723688096106 time 723699146478 state 0x0 ref 1 00000100:00100000:2.0:1789760780.453540:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554773248:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760780.453595:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64229376, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1ea20 00010000:00010000:2.0:1789760780.453605:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c56fd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.453620:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:2.0:1789760780.453624:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.453630:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c56fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64229376->64233471] (req 64229376->64233471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.453644:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c56fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64229376->64233471] (req 64229376->64233471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.458132:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982990 is committed 00002000:00010000:2.0:1789760780.458237:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64229376 -> 64233472 00002000:00010000:2.0:1789760780.458243:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125472 -> 125480 00010000:00010000:2.0:1789760780.458256:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c56fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64229376->64233471] (req 64229376->64233471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.458270:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c56fd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64229376->64233471] (req 64229376->64233471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.458280:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c56fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64229376->64233471] (req 64229376->64233471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.458288:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760780.458293:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c56fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64229376->64233471] (req 64229376->64233471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.458302:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c56fd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64229376->64233471] (req 64229376->64233471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.458322:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c56fd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64229376->64233471] (req 64229376->64233471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.458330:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c56fd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64229376->64233471] (req 64229376->64233471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.458610:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554773248:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5067us (5318us total) trans 4294982990 rc 0/0 00000100:00100000:2.0:1789760780.458621:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15692 00000100:00100000:2.0:1789760780.458631:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723688096106 time 723699146478 state 0x2 ref 0 00000100:00100000:2.0:1789760780.463132:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760780.463371:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554773376 00000100:00100000:1.0:1789760780.463421:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554773376 00000100:00100000:1.0:1789760780.463477:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723688096106 time 723699146478 state 0x0 ref 1 00000100:00100000:1.0:1789760780.463494:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723699406894 time 723699146478 state 0x1 ref 1 00000100:00100000:1.0:1789760780.463515:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723699406894 time 723709133592 state 0x0 ref 1 00000100:00100000:1.0:1789760780.463528:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554773376:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760780.463585:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64233472, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1ea28 00010000:00010000:1.0:1789760780.463595:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5704 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.463610:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760780.463615:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.463622:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5704 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64233472->64237567] (req 64233472->64237567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.463636:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5704 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64233472->64237567] (req 64233472->64237567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.467901:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982991 is committed 00002000:00010000:1.0:1789760780.468058:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64233472 -> 64237568 00002000:00010000:1.0:1789760780.468063:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125480 -> 125488 00010000:00010000:1.0:1789760780.468152:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5704 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64233472->64237567] (req 64233472->64237567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.468166:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5704 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64233472->64237567] (req 64233472->64237567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.468176:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5704 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64233472->64237567] (req 64233472->64237567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.468183:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760780.468187:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5704 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64233472->64237567] (req 64233472->64237567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.468196:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5704 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64233472->64237567] (req 64233472->64237567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.468217:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5704 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64233472->64237567] (req 64233472->64237567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.468232:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5704 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64233472->64237567] (req 64233472->64237567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.468364:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554773376:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4833us (5236us total) trans 4294982991 rc 0/0 00000100:00100000:1.0:1789760780.468411:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15693 00000100:00100000:1.0:1789760780.468422:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723699406894 time 723709133592 state 0x2 ref 0 00000100:00100000:0.0:1789760780.473789:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760780.473914:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554773504 00000100:00100000:3.0:1789760780.473922:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554773504 00000100:00100000:3.0:1789760780.473973:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723699406894 time 723709133592 state 0x0 ref 1 00000100:00100000:3.0:1789760780.473988:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723709394008 time 723709133592 state 0x1 ref 1 00000100:00100000:3.0:1789760780.474020:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723709394008 time 723719638432 state 0x0 ref 1 00000100:00100000:3.0:1789760780.474031:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554773504:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760780.474086:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64237568, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1ea30 00010000:00010000:3.0:1789760780.474095:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c570b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.474110:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760780.474115:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.474121:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c570b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64237568->64241663] (req 64237568->64241663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.474135:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c570b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64237568->64241663] (req 64237568->64241663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.478128:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982992 is committed 00002000:00010000:3.0:1789760780.478254:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64237568 -> 64241664 00002000:00010000:3.0:1789760780.478262:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125488 -> 125496 00010000:00010000:3.0:1789760780.478275:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c570b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64237568->64241663] (req 64237568->64241663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.478293:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c570b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64237568->64241663] (req 64237568->64241663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.478302:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c570b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64237568->64241663] (req 64237568->64241663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.478309:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760780.478314:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c570b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64237568->64241663] (req 64237568->64241663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.478323:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c570b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64237568->64241663] (req 64237568->64241663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.478343:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c570b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64237568->64241663] (req 64237568->64241663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.478351:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c570b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64237568->64241663] (req 64237568->64241663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.478519:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554773504:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4480us (4729us total) trans 4294982992 rc 0/0 00000100:00100000:3.0:1789760780.478530:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15694 00000100:00100000:3.0:1789760780.478540:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723709394008 time 723719638432 state 0x2 ref 0 00000100:00100000:0.0:1789760780.484781:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760780.484881:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554773632 00000100:00100000:1.0:1789760780.484899:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554773632 00000100:00100000:1.0:1789760780.484961:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723709394008 time 723719638432 state 0x0 ref 1 00000100:00100000:1.0:1789760780.484977:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723719898848 time 723719638432 state 0x1 ref 1 00000100:00100000:1.0:1789760780.485014:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723719898848 time 723730631861 state 0x0 ref 1 00000100:00100000:1.0:1789760780.485027:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554773632:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760780.485089:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64241664, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1ea38 00010000:00010000:1.0:1789760780.485099:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5712 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.485115:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760780.485119:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.485126:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5712 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64241664->64245759] (req 64241664->64245759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.485145:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5712 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64241664->64245759] (req 64241664->64245759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.493988:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982993 is committed 00002000:00010000:1.0:1789760780.494445:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64241664 -> 64245760 00002000:00010000:1.0:1789760780.494450:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125496 -> 125504 00010000:00010000:1.0:1789760780.494795:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5712 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64241664->64245759] (req 64241664->64245759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.494808:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5712 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64241664->64245759] (req 64241664->64245759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.494817:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5712 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64241664->64245759] (req 64241664->64245759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.494824:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760780.494828:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5712 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64241664->64245759] (req 64241664->64245759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.494838:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5712 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64241664->64245759] (req 64241664->64245759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.494857:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5712 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64241664->64245759] (req 64241664->64245759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.494866:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5712 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64241664->64245759] (req 64241664->64245759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.494950:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554773632:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9921us (10174us total) trans 4294982993 rc 0/0 00000100:00100000:1.0:1789760780.494961:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15695 00000100:00100000:1.0:1789760780.494971:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723719898848 time 723730631861 state 0x2 ref 0 00000100:00100000:1.0:1789760780.506718:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760780.506903:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554773760 00000100:00100000:2.0:1789760780.506913:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554773760 00000100:00100000:3.0:1789760780.507223:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723719898848 time 723730631861 state 0x0 ref 1 00000100:00100000:3.0:1789760780.507240:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723730892277 time 723730631861 state 0x1 ref 1 00000100:00100000:3.0:1789760780.507305:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723730892277 time 723752923521 state 0x0 ref 1 00000100:00100000:3.0:1789760780.507318:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554773760:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760780.507365:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64245760, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1ea40 00010000:00010000:3.0:1789760780.507374:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5719 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.507413:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760780.507416:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.507423:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5719 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64245760->64249855] (req 64245760->64249855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.507436:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5719 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64245760->64249855] (req 64245760->64249855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.511424:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982994 is committed 00002000:00010000:3.0:1789760780.511584:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64245760 -> 64249856 00002000:00010000:3.0:1789760780.511590:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125504 -> 125512 00010000:00010000:3.0:1789760780.511605:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5719 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64245760->64249855] (req 64245760->64249855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.511618:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5719 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64245760->64249855] (req 64245760->64249855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.511626:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5719 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64245760->64249855] (req 64245760->64249855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.511635:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760780.511639:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5719 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64245760->64249855] (req 64245760->64249855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.511648:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5719 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64245760->64249855] (req 64245760->64249855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.511667:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5719 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64245760->64249855] (req 64245760->64249855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.511675:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5719 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64245760->64249855] (req 64245760->64249855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.511822:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554773760:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4500us (5107us total) trans 4294982994 rc 0/0 00000100:00100000:3.0:1789760780.511833:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15696 00000100:00100000:3.0:1789760780.511842:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723730892277 time 723752923521 state 0x2 ref 0 00000100:00100000:0.0:1789760780.520162:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760780.520299:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554773888 00000100:00100000:2.0:1789760780.520309:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554773888 00000100:00100000:2.0:1789760780.520599:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723730892277 time 723752923521 state 0x0 ref 1 00000100:00100000:2.0:1789760780.520616:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723753183937 time 723752923521 state 0x1 ref 1 00000100:00100000:1.0:1789760780.520690:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723753183937 time 723766305951 state 0x0 ref 1 00000100:00100000:1.0:1789760780.520716:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554773888:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760780.520762:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64249856, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1ea48 00010000:00010000:1.0:1789760780.520772:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5720 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.520787:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760780.520791:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.520797:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5720 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64249856->64253951] (req 64249856->64253951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.520810:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5720 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64249856->64253951] (req 64249856->64253951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.525583:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982995 is committed 00002000:00010000:1.0:1789760780.525706:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64249856 -> 64253952 00002000:00010000:1.0:1789760780.525711:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125512 -> 125520 00010000:00010000:1.0:1789760780.525723:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5720 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64249856->64253951] (req 64249856->64253951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.525743:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5720 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64249856->64253951] (req 64249856->64253951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.525751:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5720 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64249856->64253951] (req 64249856->64253951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.525758:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760780.525763:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5720 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64249856->64253951] (req 64249856->64253951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.525773:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5720 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64249856->64253951] (req 64249856->64253951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.525804:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5720 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64249856->64253951] (req 64249856->64253951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.525812:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5720 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64249856->64253951] (req 64249856->64253951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.525959:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554773888:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5242us (5802us total) trans 4294982995 rc 0/0 00000100:00100000:1.0:1789760780.525970:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15697 00000100:00100000:1.0:1789760780.525979:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723753183937 time 723766305951 state 0x2 ref 0 00000100:00100000:0.0:1789760780.535329:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760780.537023:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554774016 00000100:00100000:2.0:1789760780.537037:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554774016 00000100:00100000:0.0:1789760780.537281:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723753183937 time 723766305951 state 0x0 ref 1 00000100:00100000:0.0:1789760780.537297:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723766566367 time 723766305951 state 0x1 ref 1 00000100:00100000:0.0:1789760780.537373:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723766566367 time 723782990708 state 0x0 ref 1 00000100:00100000:0.0:1789760780.537414:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554774016:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760780.537461:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64253952, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1ea50 00010000:00010000:0.0:1789760780.537469:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5727 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.537484:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760780.537488:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.537494:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5727 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64253952->64258047] (req 64253952->64258047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.537509:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5727 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64253952->64258047] (req 64253952->64258047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.541164:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982996 is committed 00002000:00010000:0.0:1789760780.541419:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64253952 -> 64258048 00002000:00010000:0.0:1789760780.541424:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125520 -> 125528 00010000:00010000:0.0:1789760780.541437:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5727 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64253952->64258047] (req 64253952->64258047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.541450:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5727 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64253952->64258047] (req 64253952->64258047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.541460:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5727 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64253952->64258047] (req 64253952->64258047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.541468:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760780.541473:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5727 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64253952->64258047] (req 64253952->64258047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.541481:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5727 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64253952->64258047] (req 64253952->64258047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.541499:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5727 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64253952->64258047] (req 64253952->64258047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.541506:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5727 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64253952->64258047] (req 64253952->64258047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.541793:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554774016:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4377us (6468us total) trans 4294982996 rc 0/0 00000100:00100000:0.0:1789760780.541803:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15698 00000100:00100000:0.0:1789760780.541813:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723766566367 time 723782990708 state 0x2 ref 0 00000100:00100000:1.0:1789760780.546301:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760780.546421:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554774144 00000100:00100000:3.0:1789760780.546431:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554774144 00000100:00100000:3.0:1789760780.546643:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723766566367 time 723782990708 state 0x0 ref 1 00000100:00100000:3.0:1789760780.546659:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723783251124 time 723782990708 state 0x1 ref 1 00000100:00100000:3.0:1789760780.546689:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723783251124 time 723792308382 state 0x0 ref 1 00000100:00100000:3.0:1789760780.546711:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554774144:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760780.546758:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64258048, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1ea58 00010000:00010000:3.0:1789760780.546766:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c572e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.546780:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760780.546785:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.546791:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c572e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64258048->64262143] (req 64258048->64262143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.546804:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c572e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64258048->64262143] (req 64258048->64262143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.551146:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982997 is committed 00002000:00010000:3.0:1789760780.551375:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64258048 -> 64262144 00002000:00010000:3.0:1789760780.551587:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125528 -> 125536 00010000:00010000:3.0:1789760780.551602:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c572e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64258048->64262143] (req 64258048->64262143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.551614:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c572e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64258048->64262143] (req 64258048->64262143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.551623:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c572e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64258048->64262143] (req 64258048->64262143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.551631:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760780.551636:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c572e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64258048->64262143] (req 64258048->64262143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.551646:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c572e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64258048->64262143] (req 64258048->64262143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.551667:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c572e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64258048->64262143] (req 64258048->64262143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.551675:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c572e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64258048->64262143] (req 64258048->64262143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.551809:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554774144:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5095us (5512us total) trans 4294982997 rc 0/0 00000100:00100000:3.0:1789760780.551820:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15699 00000100:00100000:3.0:1789760780.551829:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723783251124 time 723792308382 state 0x2 ref 0 00000100:00100000:0.0:1789760780.557146:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760780.557483:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554774272 00000100:00100000:3.0:1789760780.557492:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554774272 00000100:00100000:3.0:1789760780.557555:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723783251124 time 723792308382 state 0x0 ref 1 00000100:00100000:3.0:1789760780.557570:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723792568798 time 723792308382 state 0x1 ref 1 00000100:00100000:3.0:1789760780.557601:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723792568798 time 723803219442 state 0x0 ref 1 00000100:00100000:3.0:1789760780.557768:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554774272:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760780.557819:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64262144, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1ea60 00010000:00010000:3.0:1789760780.557829:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5735 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.557843:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760780.557847:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.557853:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5735 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64262144->64266239] (req 64262144->64266239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.557866:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5735 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64262144->64266239] (req 64262144->64266239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.564102:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982998 is committed 00002000:00010000:0.0:1789760780.564314:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64262144 -> 64266240 00002000:00010000:0.0:1789760780.564320:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125536 -> 125544 00010000:00010000:0.0:1789760780.564334:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5735 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64262144->64266239] (req 64262144->64266239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.564352:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5735 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64262144->64266239] (req 64262144->64266239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.564366:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5735 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64262144->64266239] (req 64262144->64266239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.564374:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760780.564413:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5735 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64262144->64266239] (req 64262144->64266239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.564424:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5735 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64262144->64266239] (req 64262144->64266239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.564443:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5735 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64262144->64266239] (req 64262144->64266239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.564450:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5735 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64262144->64266239] (req 64262144->64266239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.564538:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554774272:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6922us (7397us total) trans 4294982998 rc 0/0 00000100:00100000:0.0:1789760780.564552:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15700 00000100:00100000:0.0:1789760780.564562:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723792568798 time 723803219442 state 0x2 ref 0 00000100:00100000:0.0:1789760780.568656:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760780.568841:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554774400 00000100:00100000:1.0:1789760780.568851:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554774400 00000100:00100000:1.0:1789760780.568918:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723792568798 time 723803219442 state 0x0 ref 1 00000100:00100000:1.0:1789760780.568939:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723803479858 time 723803219442 state 0x1 ref 1 00000100:00100000:1.0:1789760780.568969:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723803479858 time 723814587851 state 0x0 ref 1 00000100:00100000:1.0:1789760780.568982:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554774400:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760780.569028:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64266240, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1ea68 00010000:00010000:1.0:1789760780.569038:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c573c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.569052:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760780.569057:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.569063:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c573c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64266240->64270335] (req 64266240->64270335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.569077:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c573c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64266240->64270335] (req 64266240->64270335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.573326:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294982999 is committed 00002000:00010000:1.0:1789760780.573594:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64266240 -> 64270336 00002000:00010000:1.0:1789760780.573598:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125544 -> 125552 00010000:00010000:1.0:1789760780.573612:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c573c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64266240->64270335] (req 64266240->64270335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.573623:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c573c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64266240->64270335] (req 64266240->64270335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.573632:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c573c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64266240->64270335] (req 64266240->64270335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.573638:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760780.573642:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c573c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64266240->64270335] (req 64266240->64270335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.573650:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c573c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64266240->64270335] (req 64266240->64270335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.573666:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c573c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64266240->64270335] (req 64266240->64270335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.573673:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c573c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64266240->64270335] (req 64266240->64270335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.573956:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554774400:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4972us (5304us total) trans 4294982999 rc 0/0 00000100:00100000:1.0:1789760780.573967:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15701 00000100:00100000:1.0:1789760780.573976:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723803479858 time 723814587851 state 0x2 ref 0 00000100:00100000:1.0:1789760780.580807:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760780.581016:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554774528 00000100:00100000:0.0:1789760780.581026:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554774528 00000100:00100000:0.0:1789760780.581093:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723803479858 time 723814587851 state 0x0 ref 1 00000100:00100000:0.0:1789760780.581108:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723814848267 time 723814587851 state 0x1 ref 1 00000100:00100000:0.0:1789760780.581137:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723814848267 time 723826755776 state 0x0 ref 1 00000100:00100000:0.0:1789760780.581151:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554774528:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760780.581196:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64270336, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1ea70 00010000:00010000:0.0:1789760780.581205:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5743 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.581221:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760780.581225:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.581231:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5743 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64270336->64274431] (req 64270336->64274431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.581244:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5743 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64270336->64274431] (req 64270336->64274431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.585191:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983000 is committed 00002000:00010000:0.0:1789760780.585429:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64270336 -> 64274432 00002000:00010000:0.0:1789760780.585435:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125552 -> 125560 00010000:00010000:0.0:1789760780.585460:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5743 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64270336->64274431] (req 64270336->64274431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.585473:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5743 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64270336->64274431] (req 64270336->64274431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.585482:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5743 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64270336->64274431] (req 64270336->64274431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.585489:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760780.585494:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5743 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64270336->64274431] (req 64270336->64274431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.585504:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5743 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64270336->64274431] (req 64270336->64274431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.585524:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5743 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64270336->64274431] (req 64270336->64274431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.585811:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5743 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64270336->64274431] (req 64270336->64274431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.585917:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554774528:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4764us (5116us total) trans 4294983000 rc 0/0 00000100:00100000:0.0:1789760780.585928:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15702 00000100:00100000:0.0:1789760780.585938:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723814848267 time 723826755776 state 0x2 ref 0 00000100:00100000:0.0:1789760780.591322:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760780.591571:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554774656 00000100:00100000:3.0:1789760780.591582:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554774656 00000100:00100000:3.0:1789760780.591636:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723814848267 time 723826755776 state 0x0 ref 1 00000100:00100000:3.0:1789760780.591651:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723827016192 time 723826755776 state 0x1 ref 1 00000100:00100000:3.0:1789760780.591724:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723827016192 time 723837342198 state 0x0 ref 1 00000100:00100000:3.0:1789760780.591741:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554774656:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760780.591846:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64274432, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1ea78 00010000:00010000:3.0:1789760780.591855:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c574a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.591867:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760780.591871:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.591875:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c574a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64274432->64278527] (req 64274432->64278527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.591885:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c574a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64274432->64278527] (req 64274432->64278527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.598277:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983001 is committed 00002000:00010000:1.0:1789760780.598638:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64274432 -> 64278528 00002000:00010000:1.0:1789760780.598643:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125560 -> 125568 00010000:00010000:1.0:1789760780.598656:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c574a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64274432->64278527] (req 64274432->64278527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.598670:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c574a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64274432->64278527] (req 64274432->64278527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.598680:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c574a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64274432->64278527] (req 64274432->64278527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.598687:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760780.598691:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c574a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64274432->64278527] (req 64274432->64278527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.598709:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c574a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64274432->64278527] (req 64274432->64278527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.598728:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c574a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64274432->64278527] (req 64274432->64278527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.598736:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c574a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64274432->64278527] (req 64274432->64278527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.598894:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554774656:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7150us (7575us total) trans 4294983001 rc 0/0 00000100:00100000:1.0:1789760780.598904:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15703 00000100:00100000:1.0:1789760780.598912:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723827016192 time 723837342198 state 0x2 ref 0 00000100:00100000:2.0:1789760780.605863:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760780.606280:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554774784 00000100:00100000:3.0:1789760780.606289:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554774784 00000100:00100000:3.0:1789760780.606342:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723827016192 time 723837342198 state 0x0 ref 1 00000100:00100000:3.0:1789760780.606358:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723837602614 time 723837342198 state 0x1 ref 1 00000100:00100000:3.0:1789760780.606417:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723837602614 time 723852034810 state 0x0 ref 1 00000100:00100000:3.0:1789760780.606429:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554774784:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760780.606483:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64278528, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1ea80 00010000:00010000:3.0:1789760780.606492:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5751 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.606508:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760780.606512:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.606634:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5751 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64278528->64282623] (req 64278528->64282623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.606649:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5751 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64278528->64282623] (req 64278528->64282623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.611980:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983002 is committed 00002000:00010000:3.0:1789760780.612197:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64278528 -> 64282624 00002000:00010000:3.0:1789760780.612202:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125568 -> 125576 00010000:00010000:3.0:1789760780.612215:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5751 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64278528->64282623] (req 64278528->64282623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.612228:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5751 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64278528->64282623] (req 64278528->64282623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.612236:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5751 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64278528->64282623] (req 64278528->64282623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.612244:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760780.612249:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5751 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64278528->64282623] (req 64278528->64282623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.612258:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5751 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64278528->64282623] (req 64278528->64282623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.612750:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5751 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64278528->64282623] (req 64278528->64282623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.612825:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5751 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64278528->64282623] (req 64278528->64282623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.612955:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554774784:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6524us (7097us total) trans 4294983002 rc 0/0 00000100:00100000:3.0:1789760780.612967:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15704 00000100:00100000:3.0:1789760780.612976:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723837602614 time 723852034810 state 0x2 ref 0 00000100:00100000:2.0:1789760780.617817:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760780.617891:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554774912 00000100:00100000:0.0:1789760780.617904:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554774912 00000100:00100000:0.0:1789760780.617957:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723837602614 time 723852034810 state 0x0 ref 1 00000100:00100000:0.0:1789760780.617971:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723852295226 time 723852034810 state 0x1 ref 1 00000100:00100000:0.0:1789760780.618004:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723852295226 time 723863623076 state 0x0 ref 1 00000100:00100000:0.0:1789760780.618016:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554774912:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760780.618063:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64282624, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1ea88 00010000:00010000:0.0:1789760780.618081:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5758 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.618098:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760780.618102:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.618109:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5758 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64282624->64286719] (req 64282624->64286719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.618122:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5758 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64282624->64286719] (req 64282624->64286719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.623322:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983003 is committed 00002000:00010000:0.0:1789760780.623654:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64282624 -> 64286720 00002000:00010000:0.0:1789760780.623659:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125576 -> 125584 00010000:00010000:0.0:1789760780.623672:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5758 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64282624->64286719] (req 64282624->64286719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.623688:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5758 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64282624->64286719] (req 64282624->64286719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.623706:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5758 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64282624->64286719] (req 64282624->64286719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.623713:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760780.623717:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5758 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64282624->64286719] (req 64282624->64286719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.623726:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5758 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64282624->64286719] (req 64282624->64286719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.623744:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5758 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64282624->64286719] (req 64282624->64286719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.623751:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5758 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64282624->64286719] (req 64282624->64286719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.624917:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554774912:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6894us (7102us total) trans 4294983003 rc 0/0 00000100:00100000:0.0:1789760780.624931:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15705 00000100:00100000:0.0:1789760780.624942:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723852295226 time 723863623076 state 0x2 ref 0 00000100:00100000:0.0:1789760780.630624:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760780.630722:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554775040 00000100:00100000:3.0:1789760780.630732:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554775040 00000100:00100000:3.0:1789760780.630782:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723852295226 time 723863623076 state 0x0 ref 1 00000100:00100000:3.0:1789760780.630797:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723863883492 time 723863623076 state 0x1 ref 1 00000100:00100000:3.0:1789760780.630863:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723863883492 time 723876481454 state 0x0 ref 1 00000100:00100000:3.0:1789760780.630877:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554775040:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760780.630922:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64286720, mtime: 0x6aad950b, atime: 0x0, ctime: 0x6aad950b, blocks: 0x1ea90 00010000:00010000:3.0:1789760780.630931:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c575f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.630958:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760780.630963:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.630969:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c575f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64286720->64290815] (req 64286720->64290815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.630983:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c575f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64286720->64290815] (req 64286720->64290815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.634821:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983004 is committed 00002000:00010000:3.0:1789760780.634998:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64286720 -> 64290816 00002000:00010000:3.0:1789760780.635004:0:8443:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760779 -> 1789760780 00002000:00010000:3.0:1789760780.635007:0:8443:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760779 -> 1789760780 00002000:00010000:3.0:1789760780.635010:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125584 -> 125592 00010000:00010000:3.0:1789760780.635024:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c575f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64286720->64290815] (req 64286720->64290815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.635038:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c575f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64286720->64290815] (req 64286720->64290815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.635047:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c575f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64286720->64290815] (req 64286720->64290815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.635054:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760780.635059:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c575f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64286720->64290815] (req 64286720->64290815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.635125:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c575f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64286720->64290815] (req 64286720->64290815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.635151:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c575f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64286720->64290815] (req 64286720->64290815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.635160:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c575f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64286720->64290815] (req 64286720->64290815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.635322:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554775040:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4442us (4701us total) trans 4294983004 rc 0/0 00000100:00100000:3.0:1789760780.635332:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15706 00000100:00100000:3.0:1789760780.635342:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723863883492 time 723876481454 state 0x2 ref 0 00000100:00100000:2.0:1789760780.639505:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760780.639757:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554775168 00000100:00100000:3.0:1789760780.639767:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554775168 00000100:00100000:3.0:1789760780.639823:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723863883492 time 723876481454 state 0x0 ref 1 00000100:00100000:3.0:1789760780.639837:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723876741870 time 723876481454 state 0x1 ref 1 00000100:00100000:3.0:1789760780.639858:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723876741870 time 723885477118 state 0x0 ref 1 00000100:00100000:3.0:1789760780.639869:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554775168:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760780.639915:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64290816, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ea98 00010000:00010000:3.0:1789760780.639923:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5766 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.639937:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760780.639941:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.639947:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5766 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64290816->64294911] (req 64290816->64294911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.639960:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5766 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64290816->64294911] (req 64290816->64294911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.644560:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983005 is committed 00002000:00010000:1.0:1789760780.644762:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64290816 -> 64294912 00002000:00010000:1.0:1789760780.644768:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125592 -> 125600 00010000:00010000:1.0:1789760780.644781:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5766 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64290816->64294911] (req 64290816->64294911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.644794:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5766 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64290816->64294911] (req 64290816->64294911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.644803:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5766 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64290816->64294911] (req 64290816->64294911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.644812:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760780.644816:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5766 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64290816->64294911] (req 64290816->64294911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.644825:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5766 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64290816->64294911] (req 64290816->64294911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.644843:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5766 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64290816->64294911] (req 64290816->64294911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.644851:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5766 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64290816->64294911] (req 64290816->64294911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.645015:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554775168:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5136us (5508us total) trans 4294983005 rc 0/0 00000100:00100000:1.0:1789760780.645024:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15707 00000100:00100000:1.0:1789760780.645032:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723876741870 time 723885477118 state 0x2 ref 0 00000100:00100000:2.0:1789760780.650027:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760780.650553:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554775296 00000100:00100000:0.0:1789760780.650563:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554775296 00000100:00100000:0.0:1789760780.650616:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723876741870 time 723885477118 state 0x0 ref 1 00000100:00100000:0.0:1789760780.650631:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723885737534 time 723885477118 state 0x1 ref 1 00000100:00100000:0.0:1789760780.650661:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723885737534 time 723896279867 state 0x0 ref 1 00000100:00100000:0.0:1789760780.650672:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554775296:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760780.650726:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64294912, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eaa0 00010000:00010000:0.0:1789760780.650735:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c576d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.650752:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760780.650765:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.650772:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c576d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64294912->64299007] (req 64294912->64299007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.650787:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c576d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64294912->64299007] (req 64294912->64299007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.654344:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983006 is committed 00002000:00010000:0.0:1789760780.654486:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64294912 -> 64299008 00002000:00010000:0.0:1789760780.654491:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125600 -> 125608 00010000:00010000:0.0:1789760780.654503:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c576d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64294912->64299007] (req 64294912->64299007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.654514:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c576d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64294912->64299007] (req 64294912->64299007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.654523:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c576d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64294912->64299007] (req 64294912->64299007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.654530:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760780.654535:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c576d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64294912->64299007] (req 64294912->64299007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.654544:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c576d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64294912->64299007] (req 64294912->64299007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.654562:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c576d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64294912->64299007] (req 64294912->64299007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.654570:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c576d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64294912->64299007] (req 64294912->64299007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.654667:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554775296:12345-192.168.203.22@tcp:4:dd.0 Request processed in 3991us (4643us total) trans 4294983006 rc 0/0 00000100:00100000:0.0:1789760780.654677:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15708 00000100:00100000:0.0:1789760780.654686:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723885737534 time 723896279867 state 0x2 ref 0 00000100:00100000:0.0:1789760780.659900:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760780.660060:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554775424 00000100:00100000:1.0:1789760780.660124:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554775424 00000100:00100000:1.0:1789760780.660179:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723885737534 time 723896279867 state 0x0 ref 1 00000100:00100000:1.0:1789760780.660194:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723896540283 time 723896279867 state 0x1 ref 1 00000100:00100000:1.0:1789760780.660258:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723896540283 time 723905875511 state 0x0 ref 1 00000100:00100000:1.0:1789760780.660270:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554775424:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760780.660320:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64299008, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eaa8 00010000:00010000:1.0:1789760780.660329:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5774 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.660346:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760780.660350:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.660357:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5774 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64299008->64303103] (req 64299008->64303103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.660369:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5774 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64299008->64303103] (req 64299008->64303103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.664681:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983007 is committed 00002000:00010000:1.0:1789760780.664914:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64299008 -> 64303104 00002000:00010000:1.0:1789760780.664918:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125608 -> 125616 00010000:00010000:1.0:1789760780.665001:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5774 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64299008->64303103] (req 64299008->64303103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.665018:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5774 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64299008->64303103] (req 64299008->64303103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.665027:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5774 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64299008->64303103] (req 64299008->64303103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.665034:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760780.665138:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5774 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64299008->64303103] (req 64299008->64303103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.665150:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5774 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64299008->64303103] (req 64299008->64303103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.665169:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5774 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64299008->64303103] (req 64299008->64303103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.665177:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5774 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64299008->64303103] (req 64299008->64303103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.665315:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554775424:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5042us (5419us total) trans 4294983007 rc 0/0 00000100:00100000:1.0:1789760780.665326:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15709 00000100:00100000:1.0:1789760780.665336:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723896540283 time 723905875511 state 0x2 ref 0 00000100:00100000:2.0:1789760780.672719:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760780.672837:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554775552 00000100:00100000:3.0:1789760780.672846:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554775552 00000100:00100000:3.0:1789760780.672896:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723896540283 time 723905875511 state 0x0 ref 1 00000100:00100000:3.0:1789760780.672909:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723906135927 time 723905875511 state 0x1 ref 1 00000100:00100000:3.0:1789760780.673043:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723906135927 time 723918654615 state 0x0 ref 1 00000100:00100000:3.0:1789760780.673056:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554775552:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760780.673098:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64303104, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eab0 00010000:00010000:3.0:1789760780.673107:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c577b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.673122:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760780.673126:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.673133:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c577b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64303104->64307199] (req 64303104->64307199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.673145:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c577b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64303104->64307199] (req 64303104->64307199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.678632:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983008 is committed 00002000:00010000:3.0:1789760780.679712:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64303104 -> 64307200 00002000:00010000:3.0:1789760780.679726:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125616 -> 125624 00010000:00010000:3.0:1789760780.679742:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c577b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64303104->64307199] (req 64303104->64307199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.679757:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c577b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64303104->64307199] (req 64303104->64307199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.679771:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c577b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64303104->64307199] (req 64303104->64307199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.679778:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760780.679783:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c577b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64303104->64307199] (req 64303104->64307199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.679792:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c577b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64303104->64307199] (req 64303104->64307199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.679813:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c577b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64303104->64307199] (req 64303104->64307199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.679820:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c577b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64303104->64307199] (req 64303104->64307199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.679974:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554775552:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6913us (7257us total) trans 4294983008 rc 0/0 00000100:00100000:3.0:1789760780.679985:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15710 00000100:00100000:3.0:1789760780.679994:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723906135927 time 723918654615 state 0x2 ref 0 00000100:00100000:2.0:1789760780.688214:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760780.688330:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554775680 00000100:00100000:1.0:1789760780.688340:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554775680 00000100:00100000:1.0:1789760780.688411:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723906135927 time 723918654615 state 0x0 ref 1 00000100:00100000:1.0:1789760780.688426:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723918915031 time 723918654615 state 0x1 ref 1 00000100:00100000:1.0:1789760780.688465:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723918915031 time 723934083238 state 0x0 ref 1 00000100:00100000:1.0:1789760780.688477:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554775680:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760780.688523:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64307200, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eab8 00010000:00010000:1.0:1789760780.688532:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5782 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.688547:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760780.688551:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.688557:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5782 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64307200->64311295] (req 64307200->64311295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.688569:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5782 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64307200->64311295] (req 64307200->64311295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.689826:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb054cb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3648:1876699573916544:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789760780.689888:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760780.689979:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573916544 00000100:00100000:1.0:1789760780.689984:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573916544 00000100:00100000:1.0:1789760780.690021:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 385 00000100:00100000:1.0:1789760780.690029:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f87800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3648:x1876699573916544:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1789760780.690054:0:6525:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446383616 avail 1300144128 00000100:00100000:1.0:1789760780.690316:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f87800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3648:x1876699573916544:12345-0@lo:41:osp-pre-1-0.0 Request processed in 286us (426us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760780.690326:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 385 00000100:00100000:0.0:1789760780.690417:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb054cb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3648:1876699573916544:0@lo:41:osp-pre-1-0.0 00000001:00080000:0.0:1789760780.692474:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983009 is committed 00002000:00010000:1.0:1789760780.692627:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64307200 -> 64311296 00002000:00010000:1.0:1789760780.692632:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125624 -> 125632 00010000:00010000:1.0:1789760780.692645:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5782 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64307200->64311295] (req 64307200->64311295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.692658:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5782 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64307200->64311295] (req 64307200->64311295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.692667:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5782 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64307200->64311295] (req 64307200->64311295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.692674:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760780.692679:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5782 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64307200->64311295] (req 64307200->64311295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.692712:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5782 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64307200->64311295] (req 64307200->64311295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.692733:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5782 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64307200->64311295] (req 64307200->64311295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.692742:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5782 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64307200->64311295] (req 64307200->64311295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.692832:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554775680:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4353us (4623us total) trans 4294983009 rc 0/0 00000100:00100000:1.0:1789760780.692841:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15711 00000100:00100000:1.0:1789760780.692851:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723918915031 time 723934083238 state 0x2 ref 0 00000100:00100000:0.0:1789760780.696476:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760780.696626:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554775808 00000100:00100000:3.0:1789760780.696636:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554775808 00000100:00100000:3.0:1789760780.696699:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723918915031 time 723934083238 state 0x0 ref 1 00000100:00100000:3.0:1789760780.696714:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723934343654 time 723934083238 state 0x1 ref 1 00000100:00100000:3.0:1789760780.696781:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723934343654 time 723942398615 state 0x0 ref 1 00000100:00100000:3.0:1789760780.696796:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554775808:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760780.696841:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64311296, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eac0 00010000:00010000:3.0:1789760780.696851:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5789 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.696866:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760780.696870:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.696877:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5789 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64311296->64315391] (req 64311296->64315391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.696889:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5789 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64311296->64315391] (req 64311296->64315391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.700239:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983010 is committed 00002000:00010000:3.0:1789760780.700440:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64311296 -> 64315392 00002000:00010000:3.0:1789760780.700445:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125632 -> 125640 00010000:00010000:3.0:1789760780.700524:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5789 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64311296->64315391] (req 64311296->64315391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.700536:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5789 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64311296->64315391] (req 64311296->64315391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.700545:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5789 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64311296->64315391] (req 64311296->64315391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.700555:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760780.700558:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5789 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64311296->64315391] (req 64311296->64315391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.700565:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5789 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64311296->64315391] (req 64311296->64315391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.700585:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5789 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64311296->64315391] (req 64311296->64315391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.700593:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5789 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64311296->64315391] (req 64311296->64315391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.700743:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554775808:12345-192.168.203.22@tcp:4:dd.0 Request processed in 3946us (4274us total) trans 4294983010 rc 0/0 00000100:00100000:3.0:1789760780.700754:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15712 00000100:00100000:3.0:1789760780.700764:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723934343654 time 723942398615 state 0x2 ref 0 00000100:00100000:2.0:1789760780.704167:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760780.704226:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554775936 00000100:00100000:0.0:1789760780.704237:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554775936 00000100:00100000:0.0:1789760780.704292:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723934343654 time 723942398615 state 0x0 ref 1 00000100:00100000:0.0:1789760780.704306:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723942659031 time 723942398615 state 0x1 ref 1 00000100:00100000:0.0:1789760780.704340:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723942659031 time 723949958933 state 0x0 ref 1 00000100:00100000:0.0:1789760780.704351:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554775936:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760780.704425:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64315392, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eac8 00010000:00010000:0.0:1789760780.704434:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5790 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.704451:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760780.704569:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.704576:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5790 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64315392->64319487] (req 64315392->64319487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.704590:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5790 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64315392->64319487] (req 64315392->64319487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760780.708014:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983011 is committed 00002000:00010000:0.0:1789760780.708117:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64315392 -> 64319488 00002000:00010000:0.0:1789760780.708122:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125640 -> 125648 00010000:00010000:0.0:1789760780.708137:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5790 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64315392->64319487] (req 64315392->64319487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.708150:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5790 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64315392->64319487] (req 64315392->64319487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.708158:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5790 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64315392->64319487] (req 64315392->64319487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.708166:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760780.708171:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5790 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64315392->64319487] (req 64315392->64319487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.708179:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5790 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64315392->64319487] (req 64315392->64319487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.708197:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5790 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64315392->64319487] (req 64315392->64319487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.708205:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5790 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64315392->64319487] (req 64315392->64319487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.708300:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554775936:12345-192.168.203.22@tcp:4:dd.0 Request processed in 3945us (4273us total) trans 4294983011 rc 0/0 00000100:00100000:0.0:1789760780.708310:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15713 00000100:00100000:0.0:1789760780.708320:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723942659031 time 723949958933 state 0x2 ref 0 00000100:00100000:2.0:1789760780.714029:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760780.714607:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554776064 00000100:00100000:0.0:1789760780.714617:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554776064 00000100:00100000:0.0:1789760780.714671:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723942659031 time 723949958933 state 0x0 ref 1 00000100:00100000:0.0:1789760780.714686:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723950219349 time 723949958933 state 0x1 ref 1 00000100:00100000:0.0:1789760780.714761:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723950219349 time 723960379072 state 0x0 ref 1 00000100:00100000:0.0:1789760780.714774:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554776064:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760780.714818:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64319488, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ead0 00010000:00010000:0.0:1789760780.714826:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5797 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.714842:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760780.714846:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.714852:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5797 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64319488->64323583] (req 64319488->64323583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.714865:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5797 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64319488->64323583] (req 64319488->64323583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.718752:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983012 is committed 00002000:00010000:0.0:1789760780.718912:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64319488 -> 64323584 00002000:00010000:0.0:1789760780.718917:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125648 -> 125656 00010000:00010000:0.0:1789760780.718929:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5797 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64319488->64323583] (req 64319488->64323583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.718948:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5797 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64319488->64323583] (req 64319488->64323583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.718957:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5797 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64319488->64323583] (req 64319488->64323583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.718965:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760780.718970:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5797 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64319488->64323583] (req 64319488->64323583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.718979:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5797 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64319488->64323583] (req 64319488->64323583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.718999:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5797 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64319488->64323583] (req 64319488->64323583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.719007:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5797 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64319488->64323583] (req 64319488->64323583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.719338:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554776064:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4560us (5313us total) trans 4294983012 rc 0/0 00000100:00100000:0.0:1789760780.719349:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15714 00000100:00100000:0.0:1789760780.719358:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723950219349 time 723960379072 state 0x2 ref 0 00000100:00100000:2.0:1789760780.723755:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760780.723829:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554776192 00000100:00100000:1.0:1789760780.723838:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554776192 00000100:00100000:1.0:1789760780.723895:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723950219349 time 723960379072 state 0x0 ref 1 00000100:00100000:1.0:1789760780.723911:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723960639488 time 723960379072 state 0x1 ref 1 00000100:00100000:1.0:1789760780.723978:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723960639488 time 723969595626 state 0x0 ref 1 00000100:00100000:1.0:1789760780.723993:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554776192:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760780.724048:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64323584, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ead8 00010000:00010000:1.0:1789760780.724060:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c579e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.724087:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:1.0:1789760780.724092:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.724099:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c579e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64323584->64327679] (req 64323584->64327679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.724113:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c579e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64323584->64327679] (req 64323584->64327679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.728256:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983013 is committed 00002000:00010000:1.0:1789760780.728374:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64323584 -> 64327680 00002000:00010000:1.0:1789760780.728414:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125656 -> 125664 00010000:00010000:1.0:1789760780.728431:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c579e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64323584->64327679] (req 64323584->64327679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.728445:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c579e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64323584->64327679] (req 64323584->64327679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.728467:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c579e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64323584->64327679] (req 64323584->64327679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.728475:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760780.728481:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c579e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64323584->64327679] (req 64323584->64327679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.728490:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c579e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64323584->64327679] (req 64323584->64327679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.728514:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c579e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64323584->64327679] (req 64323584->64327679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.728522:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c579e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64323584->64327679] (req 64323584->64327679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.728627:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554776192:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4632us (4875us total) trans 4294983013 rc 0/0 00000100:00100000:1.0:1789760780.728639:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15715 00000100:00100000:1.0:1789760780.728649:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723960639488 time 723969595626 state 0x2 ref 0 00000100:00100000:2.0:1789760780.732410:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760780.732551:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554776320 00000100:00100000:3.0:1789760780.732562:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554776320 00000100:00100000:3.0:1789760780.732628:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723960639488 time 723969595626 state 0x0 ref 1 00000100:00100000:3.0:1789760780.732646:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723969856042 time 723969595626 state 0x1 ref 1 00000100:00100000:3.0:1789760780.732669:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723969856042 time 723978287911 state 0x0 ref 1 00000100:00100000:3.0:1789760780.732684:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554776320:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760780.732743:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64327680, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eae0 00010000:00010000:3.0:1789760780.732753:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c57a5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.732769:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:3.0:1789760780.732773:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.732780:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c57a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64327680->64331775] (req 64327680->64331775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.732795:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c57a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64327680->64331775] (req 64327680->64331775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.737896:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983014 is committed 00002000:00010000:3.0:1789760780.738026:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64327680 -> 64331776 00002000:00010000:3.0:1789760780.738032:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125664 -> 125672 00010000:00010000:3.0:1789760780.738046:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c57a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64327680->64331775] (req 64327680->64331775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.738058:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c57a5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64327680->64331775] (req 64327680->64331775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.738080:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c57a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64327680->64331775] (req 64327680->64331775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.738088:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760780.738093:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c57a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64327680->64331775] (req 64327680->64331775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.738101:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c57a5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64327680->64331775] (req 64327680->64331775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.738124:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c57a5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64327680->64331775] (req 64327680->64331775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.738130:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c57a5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64327680->64331775] (req 64327680->64331775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.738221:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554776320:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5536us (5849us total) trans 4294983014 rc 0/0 00000100:00100000:3.0:1789760780.738231:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15716 00000100:00100000:3.0:1789760780.738240:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723969856042 time 723978287911 state 0x2 ref 0 00000100:00100000:3.0:1789760780.747925:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760780.748035:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554776448 00000100:00100000:0.0:1789760780.748046:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554776448 00000100:00100000:0.0:1789760780.748114:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723969856042 time 723978287911 state 0x0 ref 1 00000100:00100000:0.0:1789760780.748130:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723978548327 time 723978287911 state 0x1 ref 1 00000100:00100000:0.0:1789760780.748162:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723978548327 time 723993780151 state 0x0 ref 1 00000100:00100000:0.0:1789760780.748173:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06491c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554776448:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760780.748219:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64331776, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eae8 00010000:00010000:0.0:1789760780.748227:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c57ac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.748243:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=723s v=(0 0) 00010000:00010000:0.0:1789760780.748247:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.748252:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c57ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64331776->64335871] (req 64331776->64335871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.748264:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c57ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64331776->64335871] (req 64331776->64335871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.753198:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983015 is committed 00002000:00010000:0.0:1789760780.753513:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64331776 -> 64335872 00002000:00010000:0.0:1789760780.753518:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125672 -> 125680 00010000:00010000:0.0:1789760780.753534:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c57ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64331776->64335871] (req 64331776->64335871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.753547:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c57ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64331776->64335871] (req 64331776->64335871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.753561:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c57ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64331776->64335871] (req 64331776->64335871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.753568:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760780.753573:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c57ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64331776->64335871] (req 64331776->64335871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.753581:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c57ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64331776->64335871] (req 64331776->64335871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.753599:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c57ac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64331776->64335871] (req 64331776->64335871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.753607:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c57ac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64331776->64335871] (req 64331776->64335871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.753717:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06491c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554776448:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5539us (5795us total) trans 4294983015 rc 0/0 00000100:00100000:0.0:1789760780.753728:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15717 00000100:00100000:0.0:1789760780.753737:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723978548327 time 723993780151 state 0x2 ref 0 00000100:00100000:2.0:1789760780.757254:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760780.757358:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554776576 00000100:00100000:1.0:1789760780.757368:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554776576 00000100:00100000:1.0:1789760780.757461:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723978548327 time 723993780151 state 0x0 ref 1 00000100:00100000:1.0:1789760780.757476:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723994040567 time 723993780151 state 0x1 ref 1 00000100:00100000:1.0:1789760780.757504:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723994040567 time 724003122767 state 0x0 ref 1 00000100:00100000:1.0:1789760780.757516:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554776576:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760780.757567:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64335872, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eaf0 00010000:00010000:1.0:1789760780.757576:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57b3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.757592:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760780.757597:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.757603:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64335872->64339967] (req 64335872->64339967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.757618:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64335872->64339967] (req 64335872->64339967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.761720:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983016 is committed 00002000:00010000:1.0:1789760780.761915:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64335872 -> 64339968 00002000:00010000:1.0:1789760780.761919:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125680 -> 125688 00010000:00010000:1.0:1789760780.761932:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64335872->64339967] (req 64335872->64339967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.761948:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64335872->64339967] (req 64335872->64339967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.761957:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64335872->64339967] (req 64335872->64339967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.761963:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760780.761968:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64335872->64339967] (req 64335872->64339967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.761975:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64335872->64339967] (req 64335872->64339967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.761993:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57b3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64335872->64339967] (req 64335872->64339967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.762000:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57b3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64335872->64339967] (req 64335872->64339967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.762189:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554776576:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4669us (4939us total) trans 4294983016 rc 0/0 00000100:00100000:1.0:1789760780.762200:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15718 00000100:00100000:1.0:1789760780.762212:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723994040567 time 724003122767 state 0x2 ref 0 00000100:00100000:2.0:1789760780.771084:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760780.771651:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554776704 00000100:00100000:3.0:1789760780.771661:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554776704 00000100:00100000:2.0:1789760780.771853:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 723994040567 time 724003122767 state 0x0 ref 1 00000100:00100000:2.0:1789760780.771868:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724003383183 time 724003122767 state 0x1 ref 1 00000100:00100000:2.0:1789760780.771898:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724003383183 time 724017516861 state 0x0 ref 1 00000100:00100000:2.0:1789760780.771912:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554776704:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760780.771955:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64339968, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eaf8 00010000:00010000:2.0:1789760780.771964:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c57ba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.771979:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760780.771984:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.771990:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c57ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64339968->64344063] (req 64339968->64344063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.772003:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c57ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64339968->64344063] (req 64339968->64344063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.775564:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983017 is committed 00002000:00010000:2.0:1789760780.775678:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64339968 -> 64344064 00002000:00010000:2.0:1789760780.775684:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125688 -> 125696 00010000:00010000:2.0:1789760780.775706:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c57ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64339968->64344063] (req 64339968->64344063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.775719:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c57ba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64339968->64344063] (req 64339968->64344063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.775728:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c57ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64339968->64344063] (req 64339968->64344063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.775735:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760780.775740:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c57ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64339968->64344063] (req 64339968->64344063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.775748:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c57ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64339968->64344063] (req 64339968->64344063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.775768:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c57ba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64339968->64344063] (req 64339968->64344063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.775775:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c57ba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64339968->64344063] (req 64339968->64344063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.775868:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554776704:12345-192.168.203.22@tcp:4:dd.0 Request processed in 3954us (4788us total) trans 4294983017 rc 0/0 00000100:00100000:2.0:1789760780.775877:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15719 00000100:00100000:2.0:1789760780.775886:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724003383183 time 724017516861 state 0x2 ref 0 00000100:00100000:3.0:1789760780.781327:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760780.781486:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554776832 00000100:00100000:0.0:1789760780.781497:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554776832 00000100:00100000:0.0:1789760780.781657:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724003383183 time 724017516861 state 0x0 ref 1 00000100:00100000:0.0:1789760780.781673:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724017777277 time 724017516861 state 0x1 ref 1 00000100:00100000:0.0:1789760780.781715:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724017777277 time 724027332850 state 0x0 ref 1 00000100:00100000:0.0:1789760780.781727:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06490000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554776832:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760780.781777:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64344064, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eb00 00010000:00010000:0.0:1789760780.781786:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c57c1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.781800:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760780.781804:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.781810:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c57c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64344064->64348159] (req 64344064->64348159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.781823:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c57c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64344064->64348159] (req 64344064->64348159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.789403:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983018 is committed 00002000:00010000:0.0:1789760780.789492:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64344064 -> 64348160 00002000:00010000:0.0:1789760780.789498:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125696 -> 125704 00010000:00010000:0.0:1789760780.789510:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c57c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64344064->64348159] (req 64344064->64348159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.789523:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c57c1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64344064->64348159] (req 64344064->64348159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.789533:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c57c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64344064->64348159] (req 64344064->64348159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.789541:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760780.789545:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c57c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64344064->64348159] (req 64344064->64348159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.789553:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c57c1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64344064->64348159] (req 64344064->64348159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.789571:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c57c1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64344064->64348159] (req 64344064->64348159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.789579:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c57c1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64344064->64348159] (req 64344064->64348159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.789675:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06490000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554776832:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7946us (8353us total) trans 4294983018 rc 0/0 00000100:00100000:0.0:1789760780.789686:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15720 00000100:00100000:0.0:1789760780.789706:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724017777277 time 724027332850 state 0x2 ref 0 00000100:00100000:0.0:1789760780.793511:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760780.793650:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554776960 00000100:00100000:1.0:1789760780.793660:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554776960 00000100:00100000:1.0:1789760780.793720:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724017777277 time 724027332850 state 0x0 ref 1 00000100:00100000:1.0:1789760780.793735:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724027593266 time 724027332850 state 0x1 ref 1 00000100:00100000:1.0:1789760780.793872:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724027593266 time 724039489540 state 0x0 ref 1 00000100:00100000:1.0:1789760780.793885:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554776960:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760780.793934:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64348160, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eb08 00010000:00010000:1.0:1789760780.793943:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57c8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.793958:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760780.793962:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.793969:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64348160->64352255] (req 64348160->64352255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.793984:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64348160->64352255] (req 64348160->64352255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.798081:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983019 is committed 00002000:00010000:1.0:1789760780.798222:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64348160 -> 64352256 00002000:00010000:1.0:1789760780.798228:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125704 -> 125712 00010000:00010000:1.0:1789760780.798242:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64348160->64352255] (req 64348160->64352255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.798255:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57c8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64348160->64352255] (req 64348160->64352255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.798270:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64348160->64352255] (req 64348160->64352255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.798277:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760780.798282:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64348160->64352255] (req 64348160->64352255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.798292:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57c8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64348160->64352255] (req 64348160->64352255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.798317:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57c8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64348160->64352255] (req 64348160->64352255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.798324:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57c8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64348160->64352255] (req 64348160->64352255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.798687:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554776960:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4799us (5181us total) trans 4294983019 rc 0/0 00000100:00100000:1.0:1789760780.798710:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15721 00000100:00100000:1.0:1789760780.798719:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724027593266 time 724039489540 state 0x2 ref 0 00000100:00100000:2.0:1789760780.805117:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760780.805334:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554777088 00000100:00100000:1.0:1789760780.805492:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554777088 00000100:00100000:1.0:1789760780.805547:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724027593266 time 724039489540 state 0x0 ref 1 00000100:00100000:1.0:1789760780.805563:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724039749956 time 724039489540 state 0x1 ref 1 00000100:00100000:1.0:1789760780.805626:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724039749956 time 724051244124 state 0x0 ref 1 00000100:00100000:1.0:1789760780.805639:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554777088:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760780.805684:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64352256, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eb10 00010000:00010000:1.0:1789760780.805704:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57cf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.805728:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760780.805733:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.805741:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64352256->64356351] (req 64352256->64356351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.805756:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64352256->64356351] (req 64352256->64356351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.809721:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983020 is committed 00002000:00010000:1.0:1789760780.809909:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64352256 -> 64356352 00002000:00010000:1.0:1789760780.809914:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125712 -> 125720 00010000:00010000:1.0:1789760780.809927:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64352256->64356351] (req 64352256->64356351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.809940:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64352256->64356351] (req 64352256->64356351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.809950:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64352256->64356351] (req 64352256->64356351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.809962:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760780.809967:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64352256->64356351] (req 64352256->64356351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.809976:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64352256->64356351] (req 64352256->64356351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.809996:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57cf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64352256->64356351] (req 64352256->64356351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.810004:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64352256->64356351] (req 64352256->64356351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.810223:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554777088:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4581us (5110us total) trans 4294983020 rc 0/0 00000100:00100000:1.0:1789760780.810234:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15722 00000100:00100000:1.0:1789760780.810244:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724039749956 time 724051244124 state 0x2 ref 0 00000100:00100000:3.0:1789760780.816943:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760780.817021:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554777216 00000100:00100000:2.0:1789760780.817031:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554777216 00000100:00100000:2.0:1789760780.817085:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724039749956 time 724051244124 state 0x0 ref 1 00000100:00100000:2.0:1789760780.817101:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724051504540 time 724051244124 state 0x1 ref 1 00000100:00100000:2.0:1789760780.817176:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724051504540 time 724062793789 state 0x0 ref 1 00000100:00100000:2.0:1789760780.817189:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554777216:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760780.817245:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64356352, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eb18 00010000:00010000:2.0:1789760780.817255:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c57d6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.817276:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760780.817280:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.817287:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c57d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64356352->64360447] (req 64356352->64360447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.817299:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c57d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64356352->64360447] (req 64356352->64360447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.821973:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983021 is committed 00002000:00010000:2.0:1789760780.822096:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64356352 -> 64360448 00002000:00010000:2.0:1789760780.822101:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125720 -> 125728 00010000:00010000:2.0:1789760780.822114:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c57d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64356352->64360447] (req 64356352->64360447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.822127:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c57d6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64356352->64360447] (req 64356352->64360447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.822137:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c57d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64356352->64360447] (req 64356352->64360447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.822145:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760780.822149:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c57d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64356352->64360447] (req 64356352->64360447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.822158:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c57d6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64356352->64360447] (req 64356352->64360447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.822182:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c57d6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64356352->64360447] (req 64356352->64360447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.822190:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c57d6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64356352->64360447] (req 64356352->64360447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.822286:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554777216:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5094us (5348us total) trans 4294983021 rc 0/0 00000100:00100000:2.0:1789760780.822296:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15723 00000100:00100000:2.0:1789760780.822305:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724051504540 time 724062793789 state 0x2 ref 0 00000100:00100000:0.0:1789760780.827557:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760780.828216:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554777344 00000100:00100000:3.0:1789760780.828226:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554777344 00000100:00100000:3.0:1789760780.828280:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724051504540 time 724062793789 state 0x0 ref 1 00000100:00100000:3.0:1789760780.828296:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724063054205 time 724062793789 state 0x1 ref 1 00000100:00100000:3.0:1789760780.828333:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724063054205 time 724073951499 state 0x0 ref 1 00000100:00100000:3.0:1789760780.828346:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554777344:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760780.828434:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64360448, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eb20 00010000:00010000:3.0:1789760780.828445:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c57dd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.828558:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760780.828562:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.828569:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c57dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64360448->64364543] (req 64360448->64364543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.828583:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c57dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64360448->64364543] (req 64360448->64364543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.832136:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983022 is committed 00002000:00010000:3.0:1789760780.832427:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64360448 -> 64364544 00002000:00010000:3.0:1789760780.832433:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125728 -> 125736 00010000:00010000:3.0:1789760780.832445:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c57dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64360448->64364543] (req 64360448->64364543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.832459:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c57dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64360448->64364543] (req 64360448->64364543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.832469:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c57dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64360448->64364543] (req 64360448->64364543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.832482:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760780.832487:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c57dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64360448->64364543] (req 64360448->64364543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.832508:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c57dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64360448->64364543] (req 64360448->64364543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.832529:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c57dd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64360448->64364543] (req 64360448->64364543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.832536:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c57dd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64360448->64364543] (req 64360448->64364543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.832832:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554777344:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4482us (5280us total) trans 4294983022 rc 0/0 00000100:00100000:3.0:1789760780.832842:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15724 00000100:00100000:3.0:1789760780.832852:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724063054205 time 724073951499 state 0x2 ref 0 00000100:00100000:0.0:1789760780.839751:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760780.839878:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554777472 00000100:00100000:1.0:1789760780.839887:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554777472 00000100:00100000:1.0:1789760780.839940:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724063054205 time 724073951499 state 0x0 ref 1 00000100:00100000:1.0:1789760780.839954:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724074211915 time 724073951499 state 0x1 ref 1 00000100:00100000:1.0:1789760780.839988:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724074211915 time 724085606443 state 0x0 ref 1 00000100:00100000:1.0:1789760780.840530:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554777472:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760780.840583:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64364544, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eb28 00010000:00010000:1.0:1789760780.840592:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57e4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.840609:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760780.840614:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.840620:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64364544->64368639] (req 64364544->64368639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.840636:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64364544->64368639] (req 64364544->64368639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.845154:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983023 is committed 00002000:00010000:1.0:1789760780.845372:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64364544 -> 64368640 00002000:00010000:1.0:1789760780.845436:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125736 -> 125744 00010000:00010000:1.0:1789760780.845514:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64364544->64368639] (req 64364544->64368639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.845527:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64364544->64368639] (req 64364544->64368639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.845536:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64364544->64368639] (req 64364544->64368639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.845543:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760780.845548:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64364544->64368639] (req 64364544->64368639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.845556:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57e4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64364544->64368639] (req 64364544->64368639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.845575:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57e4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64364544->64368639] (req 64364544->64368639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.845583:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57e4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64364544->64368639] (req 64364544->64368639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.845711:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554777472:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5705us (5966us total) trans 4294983023 rc 0/0 00000100:00100000:1.0:1789760780.845722:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15725 00000100:00100000:1.0:1789760780.845730:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724074211915 time 724085606443 state 0x2 ref 0 00000100:00100000:2.0:1789760780.852612:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760780.852676:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554777600 00000100:00100000:1.0:1789760780.852688:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554777600 00000100:00100000:1.0:1789760780.852760:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724074211915 time 724085606443 state 0x0 ref 1 00000100:00100000:1.0:1789760780.852776:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724085866859 time 724085606443 state 0x1 ref 1 00000100:00100000:1.0:1789760780.852808:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724085866859 time 724098425772 state 0x0 ref 1 00000100:00100000:1.0:1789760780.852820:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554777600:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760780.852865:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64368640, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eb30 00010000:00010000:1.0:1789760780.852874:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57eb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.852890:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760780.852894:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.852900:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64368640->64372735] (req 64368640->64372735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.852914:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64368640->64372735] (req 64368640->64372735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.856673:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983024 is committed 00002000:00010000:1.0:1789760780.856786:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64368640 -> 64372736 00002000:00010000:1.0:1789760780.856791:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125744 -> 125752 00010000:00010000:1.0:1789760780.856804:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64368640->64372735] (req 64368640->64372735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.856816:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64368640->64372735] (req 64368640->64372735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.856825:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64368640->64372735] (req 64368640->64372735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.856844:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760780.856849:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64368640->64372735] (req 64368640->64372735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.856858:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64368640->64372735] (req 64368640->64372735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.856878:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57eb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64368640->64372735] (req 64368640->64372735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.856886:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57eb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64368640->64372735] (req 64368640->64372735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.857271:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554777600:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4447us (4886us total) trans 4294983024 rc 0/0 00000100:00100000:1.0:1789760780.857282:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15726 00000100:00100000:1.0:1789760780.857291:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724085866859 time 724098425772 state 0x2 ref 0 00000100:00100000:0.0:1789760780.862055:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760780.862300:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554777728 00000100:00100000:2.0:1789760780.862311:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554777728 00000100:00100000:2.0:1789760780.862363:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724085866859 time 724098425772 state 0x0 ref 1 00000100:00100000:2.0:1789760780.862411:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724098686188 time 724098425772 state 0x1 ref 1 00000100:00100000:2.0:1789760780.862435:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724098686188 time 724108053384 state 0x0 ref 1 00000100:00100000:2.0:1789760780.862449:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04562d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554777728:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760780.862495:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64372736, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eb38 00010000:00010000:2.0:1789760780.862509:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c57f2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.862530:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760780.862535:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.862545:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c57f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64372736->64376831] (req 64372736->64376831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.862558:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c57f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64372736->64376831] (req 64372736->64376831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.866511:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983025 is committed 00002000:00010000:0.0:1789760780.866953:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64372736 -> 64376832 00002000:00010000:0.0:1789760780.866959:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125752 -> 125760 00010000:00010000:0.0:1789760780.866972:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c57f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64372736->64376831] (req 64372736->64376831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.866985:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c57f2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64372736->64376831] (req 64372736->64376831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.866994:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c57f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64372736->64376831] (req 64372736->64376831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.867003:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760780.867007:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c57f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64372736->64376831] (req 64372736->64376831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.867017:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c57f2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64372736->64376831] (req 64372736->64376831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.867037:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c57f2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64372736->64376831] (req 64372736->64376831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.867044:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c57f2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64372736->64376831] (req 64372736->64376831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.867644:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04562d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554777728:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5192us (5596us total) trans 4294983025 rc 0/0 00000100:00100000:0.0:1789760780.867659:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15727 00000100:00100000:0.0:1789760780.867667:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724098686188 time 724108053384 state 0x2 ref 0 00000100:00100000:0.0:1789760780.875158:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760780.875256:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554777856 00000100:00100000:1.0:1789760780.875275:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554777856 00000100:00100000:1.0:1789760780.875326:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724098686188 time 724108053384 state 0x0 ref 1 00000100:00100000:1.0:1789760780.875342:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724108313800 time 724108053384 state 0x1 ref 1 00000100:00100000:1.0:1789760780.875374:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724108313800 time 724120993251 state 0x0 ref 1 00000100:00100000:1.0:1789760780.875426:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04563800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554777856:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760780.875472:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64376832, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eb40 00010000:00010000:1.0:1789760780.875482:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57f9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.875500:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760780.875506:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.875513:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64376832->64380927] (req 64376832->64380927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.875528:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64376832->64380927] (req 64376832->64380927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.881647:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983026 is committed 00010000:00010000:0.0:1789760780.881729:0:8453:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:1.0:1789760780.881891:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64376832 -> 64380928 00002000:00010000:1.0:1789760780.881896:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125760 -> 125768 00010000:00010000:1.0:1789760780.881908:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64376832->64380927] (req 64376832->64380927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.881921:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57f9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64376832->64380927] (req 64376832->64380927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.881929:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64376832->64380927] (req 64376832->64380927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.881936:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760780.881941:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64376832->64380927] (req 64376832->64380927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.881949:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57f9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64376832->64380927] (req 64376832->64380927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.881969:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57f9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64376832->64380927] (req 64376832->64380927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.881982:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c57f9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64376832->64380927] (req 64376832->64380927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.882078:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04563800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554777856:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6644us (6920us total) trans 4294983026 rc 0/0 00000100:00100000:1.0:1789760780.882092:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15728 00000100:00100000:1.0:1789760780.882101:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724108313800 time 724120993251 state 0x2 ref 0 00000100:00100000:3.0:1789760780.892429:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760780.892494:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554777984 00000100:00100000:1.0:1789760780.892508:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554777984 00000100:00100000:1.0:1789760780.892564:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724108313800 time 724120993251 state 0x0 ref 1 00000100:00100000:1.0:1789760780.892585:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724121253667 time 724120993251 state 0x1 ref 1 00000100:00100000:0.0:1789760780.892701:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724121253667 time 724138307328 state 0x0 ref 1 00000100:00100000:0.0:1789760780.892717:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06492680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554777984:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760780.892766:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64380928, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eb48 00010000:00010000:0.0:1789760780.892775:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5800 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.892792:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760780.892797:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.892803:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5800 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64380928->64385023] (req 64380928->64385023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.892817:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5800 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64380928->64385023] (req 64380928->64385023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.902157:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983027 is committed 00002000:00010000:0.0:1789760780.902276:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64380928 -> 64385024 00002000:00010000:0.0:1789760780.902281:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125768 -> 125776 00010000:00010000:0.0:1789760780.902294:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5800 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64380928->64385023] (req 64380928->64385023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.902306:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5800 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64380928->64385023] (req 64380928->64385023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.902318:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5800 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64380928->64385023] (req 64380928->64385023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.902325:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760780.902329:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5800 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64380928->64385023] (req 64380928->64385023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.902337:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5800 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64380928->64385023] (req 64380928->64385023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.902356:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5800 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64380928->64385023] (req 64380928->64385023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.902363:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5800 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64380928->64385023] (req 64380928->64385023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.902484:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06492680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554777984:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9766us (10158us total) trans 4294983027 rc 0/0 00000100:00100000:0.0:1789760780.902495:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15729 00000100:00100000:0.0:1789760780.902504:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724121253667 time 724138307328 state 0x2 ref 0 00000100:00100000:0.0:1789760780.908425:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760780.908504:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554778112 00000100:00100000:1.0:1789760780.908514:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554778112 00000100:00100000:1.0:1789760780.908568:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724121253667 time 724138307328 state 0x0 ref 1 00000100:00100000:1.0:1789760780.908582:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724138567744 time 724138307328 state 0x1 ref 1 00000100:00100000:1.0:1789760780.908648:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724138567744 time 724154266083 state 0x0 ref 1 00000100:00100000:1.0:1789760780.908670:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554778112:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760780.908724:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64385024, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eb50 00010000:00010000:1.0:1789760780.908733:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5807 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.908748:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760780.908752:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.908758:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5807 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64385024->64389119] (req 64385024->64389119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.908771:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5807 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64385024->64389119] (req 64385024->64389119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.914281:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983028 is committed 00002000:00010000:1.0:1789760780.914825:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64385024 -> 64389120 00002000:00010000:1.0:1789760780.914830:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125776 -> 125784 00010000:00010000:1.0:1789760780.914844:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5807 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64385024->64389119] (req 64385024->64389119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.914857:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5807 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64385024->64389119] (req 64385024->64389119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.914866:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5807 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64385024->64389119] (req 64385024->64389119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.914874:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760780.914878:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5807 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64385024->64389119] (req 64385024->64389119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.914888:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5807 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64385024->64389119] (req 64385024->64389119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.914912:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5807 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64385024->64389119] (req 64385024->64389119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.914921:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5807 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64385024->64389119] (req 64385024->64389119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.915712:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554778112:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7047us (7329us total) trans 4294983028 rc 0/0 00000100:00100000:1.0:1789760780.915724:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15730 00000100:00100000:1.0:1789760780.915741:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724138567744 time 724154266083 state 0x2 ref 0 00000100:00100000:2.0:1789760780.920798:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760780.920948:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554778240 00000100:00100000:1.0:1789760780.920957:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554778240 00000100:00100000:1.0:1789760780.921009:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724138567744 time 724154266083 state 0x0 ref 1 00000100:00100000:1.0:1789760780.921023:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724154526499 time 724154266083 state 0x1 ref 1 00000100:00100000:1.0:1789760780.921061:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724154526499 time 724166672457 state 0x0 ref 1 00000100:00100000:1.0:1789760780.921081:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554778240:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760780.921135:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64389120, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eb58 00010000:00010000:1.0:1789760780.921145:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c580e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.921168:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760780.921172:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.921180:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c580e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64389120->64393215] (req 64389120->64393215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.921194:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c580e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64389120->64393215] (req 64389120->64393215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.926042:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983029 is committed 00002000:00010000:1.0:1789760780.926300:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64389120 -> 64393216 00002000:00010000:1.0:1789760780.926306:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125784 -> 125792 00010000:00010000:1.0:1789760780.926319:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c580e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64389120->64393215] (req 64389120->64393215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.926331:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c580e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64389120->64393215] (req 64389120->64393215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.926340:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c580e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64389120->64393215] (req 64389120->64393215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.926347:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760780.926351:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c580e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64389120->64393215] (req 64389120->64393215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.926360:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c580e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64389120->64393215] (req 64389120->64393215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.927795:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c580e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64389120->64393215] (req 64389120->64393215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.927818:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c580e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64389120->64393215] (req 64389120->64393215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.927926:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554778240:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6840us (7133us total) trans 4294983029 rc 0/0 00000100:00100000:1.0:1789760780.927938:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15731 00000100:00100000:1.0:1789760780.927947:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724154526499 time 724166672457 state 0x2 ref 0 00000100:00100000:3.0:1789760780.935681:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760780.936094:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554778368 00000100:00100000:1.0:1789760780.936104:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554778368 00000100:00100000:1.0:1789760780.936160:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724154526499 time 724166672457 state 0x0 ref 1 00000100:00100000:1.0:1789760780.936188:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724166932873 time 724166672457 state 0x1 ref 1 00000100:00100000:1.0:1789760780.936220:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724166932873 time 724181838451 state 0x0 ref 1 00000100:00100000:1.0:1789760780.936234:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554778368:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760780.936280:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64393216, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eb60 00010000:00010000:1.0:1789760780.936289:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5815 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.936306:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760780.936310:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.936317:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5815 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64393216->64397311] (req 64393216->64397311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.936346:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5815 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64393216->64397311] (req 64393216->64397311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.942103:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983030 is committed 00002000:00010000:3.0:1789760780.942443:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64393216 -> 64397312 00002000:00010000:3.0:1789760780.942448:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125792 -> 125800 00010000:00010000:3.0:1789760780.942462:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5815 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64393216->64397311] (req 64393216->64397311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.942480:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5815 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64393216->64397311] (req 64393216->64397311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.942488:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5815 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64393216->64397311] (req 64393216->64397311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.942495:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760780.942500:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5815 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64393216->64397311] (req 64393216->64397311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.942509:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5815 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64393216->64397311] (req 64393216->64397311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.942530:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5815 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64393216->64397311] (req 64393216->64397311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.942538:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5815 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64393216->64397311] (req 64393216->64397311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.942741:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554778368:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6504us (7063us total) trans 4294983030 rc 0/0 00000100:00100000:3.0:1789760780.942752:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15732 00000100:00100000:3.0:1789760780.942762:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724166932873 time 724181838451 state 0x2 ref 0 00000100:00100000:0.0:1789760780.948346:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760780.948489:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554778496 00000100:00100000:2.0:1789760780.948499:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554778496 00000100:00100000:2.0:1789760780.948553:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724166932873 time 724181838451 state 0x0 ref 1 00000100:00100000:2.0:1789760780.948570:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724182098867 time 724181838451 state 0x1 ref 1 00000100:00100000:2.0:1789760780.948590:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724182098867 time 724194209110 state 0x0 ref 1 00000100:00100000:2.0:1789760780.948603:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554778496:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760780.948649:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64397312, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eb68 00010000:00010000:2.0:1789760780.948658:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c581c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.948674:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760780.948679:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760780.948685:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c581c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64397312->64401407] (req 64397312->64401407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.948711:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c581c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64397312->64401407] (req 64397312->64401407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.956731:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983031 is committed 00002000:00010000:2.0:1789760780.956841:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64397312 -> 64401408 00002000:00010000:2.0:1789760780.956845:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125800 -> 125808 00010000:00010000:2.0:1789760780.956857:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c581c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64397312->64401407] (req 64397312->64401407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.956869:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c581c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64397312->64401407] (req 64397312->64401407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.956878:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c581c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64397312->64401407] (req 64397312->64401407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.956949:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760780.956954:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c581c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64397312->64401407] (req 64397312->64401407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.956964:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c581c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64397312->64401407] (req 64397312->64401407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.956984:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c581c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64397312->64401407] (req 64397312->64401407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760780.956998:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c581c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64397312->64401407] (req 64397312->64401407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760780.957276:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554778496:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8604us (8868us total) trans 4294983031 rc 0/0 00000100:00100000:2.0:1789760780.957287:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15733 00000100:00100000:2.0:1789760780.957297:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724182098867 time 724194209110 state 0x2 ref 0 00000100:00100000:2.0:1789760780.961344:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760780.961520:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554778624 00000100:00100000:0.0:1789760780.961530:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554778624 00000100:00100000:0.0:1789760780.961586:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724182098867 time 724194209110 state 0x0 ref 1 00000100:00100000:0.0:1789760780.961602:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724194469526 time 724194209110 state 0x1 ref 1 00000100:00100000:0.0:1789760780.961742:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724194469526 time 724207358818 state 0x0 ref 1 00000100:00100000:0.0:1789760780.961758:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554778624:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760780.961810:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64401408, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eb70 00010000:00010000:0.0:1789760780.961820:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5823 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.961836:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760780.961841:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.961847:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5823 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64401408->64405503] (req 64401408->64405503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.961860:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5823 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64401408->64405503] (req 64401408->64405503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760780.966357:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983032 is committed 00002000:00010000:0.0:1789760780.966504:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64401408 -> 64405504 00002000:00010000:0.0:1789760780.966509:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125808 -> 125816 00010000:00010000:0.0:1789760780.966521:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5823 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64401408->64405503] (req 64401408->64405503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.966540:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5823 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64401408->64405503] (req 64401408->64405503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.966550:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5823 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64401408->64405503] (req 64401408->64405503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.966557:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760780.966562:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5823 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64401408->64405503] (req 64401408->64405503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.966572:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5823 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64401408->64405503] (req 64401408->64405503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.966592:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5823 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64401408->64405503] (req 64401408->64405503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.966600:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5823 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64401408->64405503] (req 64401408->64405503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760780.966701:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554778624:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4932us (5351us total) trans 4294983032 rc 0/0 00000100:00100000:0.0:1789760780.966712:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15734 00000100:00100000:0.0:1789760780.966722:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724194469526 time 724207358818 state 0x2 ref 0 00000100:00100000:3.0:1789760780.971661:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760780.971800:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554778752 00000100:00100000:1.0:1789760780.971810:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554778752 00000100:00100000:1.0:1789760780.971861:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724194469526 time 724207358818 state 0x0 ref 1 00000100:00100000:1.0:1789760780.971877:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724207619234 time 724207358818 state 0x1 ref 1 00000100:00100000:1.0:1789760780.971901:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724207619234 time 724217519418 state 0x0 ref 1 00000100:00100000:1.0:1789760780.971913:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06491180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554778752:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760780.971958:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64405504, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eb78 00010000:00010000:1.0:1789760780.971967:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c582a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.971984:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760780.971988:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760780.971995:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c582a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64405504->64409599] (req 64405504->64409599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.972009:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c582a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64405504->64409599] (req 64405504->64409599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.976803:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983033 is committed 00002000:00010000:1.0:1789760780.976942:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64405504 -> 64409600 00002000:00010000:1.0:1789760780.976947:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125816 -> 125824 00010000:00010000:1.0:1789760780.976958:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c582a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64405504->64409599] (req 64405504->64409599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.976967:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c582a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64405504->64409599] (req 64405504->64409599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.976973:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c582a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64405504->64409599] (req 64405504->64409599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.976977:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760780.976981:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c582a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64405504->64409599] (req 64405504->64409599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.976987:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c582a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64405504->64409599] (req 64405504->64409599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.977001:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c582a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64405504->64409599] (req 64405504->64409599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760780.977005:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c582a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64405504->64409599] (req 64405504->64409599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760780.977292:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06491180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554778752:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5375us (5636us total) trans 4294983033 rc 0/0 00000100:00100000:1.0:1789760780.977302:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15735 00000100:00100000:1.0:1789760780.977312:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724207619234 time 724217519418 state 0x2 ref 0 00000100:00100000:2.0:1789760780.986934:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760780.987080:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554778880 00000100:00100000:3.0:1789760780.987089:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554778880 00000100:00100000:3.0:1789760780.987158:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724207619234 time 724217519418 state 0x0 ref 1 00000100:00100000:3.0:1789760780.987174:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724217779834 time 724217519418 state 0x1 ref 1 00000100:00100000:3.0:1789760780.987253:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724217779834 time 724232870220 state 0x0 ref 1 00000100:00100000:3.0:1789760780.987267:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a272a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554778880:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760780.987316:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64409600, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eb80 00010000:00010000:3.0:1789760780.987326:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5831 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.987342:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760780.987347:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760780.987353:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5831 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64409600->64413695] (req 64409600->64413695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.987369:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5831 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64409600->64413695] (req 64409600->64413695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760780.992178:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983034 is committed 00002000:00010000:3.0:1789760780.992778:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64409600 -> 64413696 00002000:00010000:3.0:1789760780.992784:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125824 -> 125832 00010000:00010000:3.0:1789760780.992797:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5831 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64409600->64413695] (req 64409600->64413695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.992809:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5831 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64409600->64413695] (req 64409600->64413695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.992818:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5831 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64409600->64413695] (req 64409600->64413695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.992826:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760780.992831:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5831 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64409600->64413695] (req 64409600->64413695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.992841:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5831 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64409600->64413695] (req 64409600->64413695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.992862:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5831 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64409600->64413695] (req 64409600->64413695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760780.992870:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5831 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64409600->64413695] (req 64409600->64413695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760780.992964:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a272a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554778880:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5696us (6036us total) trans 4294983034 rc 0/0 00000100:00100000:3.0:1789760780.992974:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15736 00000100:00100000:3.0:1789760780.992984:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724217779834 time 724232870220 state 0x2 ref 0 00000100:00100000:3.0:1789760780.997219:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760780.997304:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554779008 00000100:00100000:0.0:1789760780.997315:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554779008 00000100:00100000:0.0:1789760780.997372:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724217779834 time 724232870220 state 0x0 ref 1 00000100:00100000:0.0:1789760780.997416:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724233130636 time 724232870220 state 0x1 ref 1 00000100:00100000:0.0:1789760780.997439:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724233130636 time 724243058234 state 0x0 ref 1 00000100:00100000:0.0:1789760780.997455:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554779008:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760780.997501:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64413696, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eb88 00010000:00010000:0.0:1789760780.997510:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5838 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.997524:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760780.997530:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760780.997537:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5838 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64413696->64417791] (req 64413696->64417791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760780.997551:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5838 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64413696->64417791] (req 64413696->64417791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.001299:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983035 is committed 00002000:00010000:0.0:1789760781.003056:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64413696 -> 64417792 00002000:00010000:0.0:1789760781.003065:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125832 -> 125840 00010000:00010000:0.0:1789760781.003078:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5838 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64413696->64417791] (req 64413696->64417791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.003097:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5838 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64413696->64417791] (req 64413696->64417791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.003106:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5838 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64413696->64417791] (req 64413696->64417791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.003114:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760781.003119:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5838 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64413696->64417791] (req 64413696->64417791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.003129:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5838 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64413696->64417791] (req 64413696->64417791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.003149:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5838 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64413696->64417791] (req 64413696->64417791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.003156:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5838 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64413696->64417791] (req 64413696->64417791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.003280:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554779008:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5826us (6065us total) trans 4294983035 rc 0/0 00000100:00100000:0.0:1789760781.003291:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15737 00000100:00100000:0.0:1789760781.003300:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724233130636 time 724243058234 state 0x2 ref 0 00000100:00100000:2.0:1789760781.006892:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760781.006973:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554779136 00000100:00100000:0.0:1789760781.006982:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554779136 00000100:00100000:0.0:1789760781.007052:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724233130636 time 724243058234 state 0x0 ref 1 00000100:00100000:0.0:1789760781.007078:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724243318650 time 724243058234 state 0x1 ref 1 00000100:00080000:3.0:1789760781.007158:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760781.007205:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724243318650 time 724252823169 state 0x0 ref 1 00000100:00100000:0.0:1789760781.007219:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a270380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554779136:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760781.007265:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64417792, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eb90 00010000:00010000:0.0:1789760781.007275:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c583f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.007292:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760781.007296:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.007303:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c583f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64417792->64421887] (req 64417792->64421887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.007317:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c583f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64417792->64421887] (req 64417792->64421887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.014609:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983036 is committed 00002000:00010000:1.0:1789760781.014947:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64417792 -> 64421888 00002000:00010000:1.0:1789760781.014954:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125840 -> 125848 00010000:00010000:1.0:1789760781.014967:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c583f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64417792->64421887] (req 64417792->64421887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.014980:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c583f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64417792->64421887] (req 64417792->64421887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.014989:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c583f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64417792->64421887] (req 64417792->64421887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.014997:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760781.015003:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c583f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64417792->64421887] (req 64417792->64421887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.015013:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c583f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64417792->64421887] (req 64417792->64421887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.015032:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c583f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64417792->64421887] (req 64417792->64421887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.015040:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c583f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64417792->64421887] (req 64417792->64421887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.015242:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a270380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554779136:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8021us (8353us total) trans 4294983036 rc 0/0 00000100:00100000:1.0:1789760781.015257:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15738 00000100:00100000:1.0:1789760781.015267:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724243318650 time 724252823169 state 0x2 ref 0 00000100:00100000:2.0:1789760781.021037:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760781.023942:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554779264 00000100:00100000:1.0:1789760781.023956:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554779264 00000100:00100000:1.0:1789760781.024017:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724243318650 time 724252823169 state 0x0 ref 1 00000100:00100000:1.0:1789760781.024031:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724253083585 time 724252823169 state 0x1 ref 1 00000100:00100000:1.0:1789760781.024053:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724253083585 time 724269672153 state 0x0 ref 1 00000100:00100000:1.0:1789760781.024065:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a270700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554779264:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760781.024111:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64421888, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eb98 00010000:00010000:1.0:1789760781.024120:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5846 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.024135:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760781.024140:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.024146:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5846 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64421888->64425983] (req 64421888->64425983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.024159:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5846 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64421888->64425983] (req 64421888->64425983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.029017:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983037 is committed 00002000:00010000:1.0:1789760781.029189:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64421888 -> 64425984 00002000:00010000:1.0:1789760781.029196:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125848 -> 125856 00010000:00010000:1.0:1789760781.029215:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5846 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64421888->64425983] (req 64421888->64425983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.029230:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5846 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64421888->64425983] (req 64421888->64425983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.029240:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5846 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64421888->64425983] (req 64421888->64425983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.029248:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760781.029253:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5846 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64421888->64425983] (req 64421888->64425983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.029262:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5846 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64421888->64425983] (req 64421888->64425983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.029287:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5846 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64421888->64425983] (req 64421888->64425983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.029295:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5846 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64421888->64425983] (req 64421888->64425983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.029701:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a270700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554779264:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5619us (8656us total) trans 4294983037 rc 0/0 00000100:00100000:1.0:1789760781.029714:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15739 00000100:00100000:1.0:1789760781.029726:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724253083585 time 724269672153 state 0x2 ref 0 00000100:00100000:3.0:1789760781.039775:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760781.040903:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554779392 00000100:00100000:1.0:1789760781.040922:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554779392 00000100:00100000:1.0:1789760781.041002:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724253083585 time 724269672153 state 0x0 ref 1 00000100:00100000:1.0:1789760781.041020:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724269932569 time 724269672153 state 0x1 ref 1 00000100:00100000:1.0:1789760781.041070:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724269932569 time 724286688055 state 0x0 ref 1 00000100:00100000:1.0:1789760781.041086:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06490700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554779392:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760781.041150:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64425984, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eba0 00010000:00010000:1.0:1789760781.041161:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c584d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.041181:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760781.041186:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.041194:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c584d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64425984->64430079] (req 64425984->64430079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.041217:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c584d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64425984->64430079] (req 64425984->64430079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.047684:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983038 is committed 00002000:00010000:3.0:1789760781.047918:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64425984 -> 64430080 00002000:00010000:3.0:1789760781.047925:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125856 -> 125864 00010000:00010000:3.0:1789760781.047943:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c584d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64425984->64430079] (req 64425984->64430079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.047959:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c584d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64425984->64430079] (req 64425984->64430079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.047970:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c584d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64425984->64430079] (req 64425984->64430079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.047979:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760781.047985:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c584d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64425984->64430079] (req 64425984->64430079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.047996:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c584d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64425984->64430079] (req 64425984->64430079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.048021:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c584d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64425984->64430079] (req 64425984->64430079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.048029:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c584d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64425984->64430079] (req 64425984->64430079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.048161:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06490700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554779392:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7072us (8394us total) trans 4294983038 rc 0/0 00000100:00100000:3.0:1789760781.048175:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15740 00000100:00100000:3.0:1789760781.048187:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724269932569 time 724286688055 state 0x2 ref 0 00000100:00100000:3.0:1789760781.050969:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760781.051581:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554779520 00000100:00100000:0.0:1789760781.051596:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554779520 00000100:00100000:0.0:1789760781.051672:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724269932569 time 724286688055 state 0x0 ref 1 00000100:00100000:0.0:1789760781.051702:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724286948471 time 724286688055 state 0x1 ref 1 00000100:00100000:0.0:1789760781.051748:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724286948471 time 724297366569 state 0x0 ref 1 00000100:00100000:0.0:1789760781.051763:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe19850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554779520:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760781.051826:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64430080, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eba8 00010000:00010000:0.0:1789760781.051836:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5854 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.051856:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760781.051861:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.051868:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5854 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64430080->64434175] (req 64430080->64434175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.051885:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5854 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64430080->64434175] (req 64430080->64434175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.060628:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983039 is committed 00002000:00010000:0.0:1789760781.060910:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64430080 -> 64434176 00002000:00010000:0.0:1789760781.060916:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125864 -> 125872 00010000:00010000:0.0:1789760781.060936:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5854 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64430080->64434175] (req 64430080->64434175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.060950:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5854 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64430080->64434175] (req 64430080->64434175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.060960:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5854 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64430080->64434175] (req 64430080->64434175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.060973:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760781.060978:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5854 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64430080->64434175] (req 64430080->64434175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.060989:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5854 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64430080->64434175] (req 64430080->64434175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.061012:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5854 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64430080->64434175] (req 64430080->64434175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.061020:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5854 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64430080->64434175] (req 64430080->64434175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.061259:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe19850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554779520:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9491us (10297us total) trans 4294983039 rc 0/0 00000100:00100000:0.0:1789760781.061272:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15741 00000100:00100000:0.0:1789760781.061284:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724286948471 time 724297366569 state 0x2 ref 0 00000100:00100000:3.0:1789760781.065222:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760781.065293:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554779648 00000100:00100000:0.0:1789760781.065305:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554779648 00000100:00100000:0.0:1789760781.065375:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724286948471 time 724297366569 state 0x0 ref 1 00000100:00100000:0.0:1789760781.065431:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724297626985 time 724297366569 state 0x1 ref 1 00000100:00100000:0.0:1789760781.065467:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724297626985 time 724311085472 state 0x0 ref 1 00000100:00100000:0.0:1789760781.065484:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06493800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554779648:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760781.065547:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64434176, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ebb0 00010000:00010000:0.0:1789760781.065559:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c585b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.065578:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760781.065583:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.065591:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c585b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64434176->64438271] (req 64434176->64438271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.065607:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c585b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64434176->64438271] (req 64434176->64438271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.070795:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983040 is committed 00002000:00010000:0.0:1789760781.071014:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64434176 -> 64438272 00002000:00010000:0.0:1789760781.071020:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125872 -> 125880 00010000:00010000:0.0:1789760781.071038:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c585b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64434176->64438271] (req 64434176->64438271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.071054:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c585b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64434176->64438271] (req 64434176->64438271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.071126:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c585b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64434176->64438271] (req 64434176->64438271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.071137:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760781.071143:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c585b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64434176->64438271] (req 64434176->64438271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.071154:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c585b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64434176->64438271] (req 64434176->64438271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.071178:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c585b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64434176->64438271] (req 64434176->64438271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.071186:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c585b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64434176->64438271] (req 64434176->64438271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.071337:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06493800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554779648:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5847us (6123us total) trans 4294983040 rc 0/0 00000100:00100000:0.0:1789760781.071350:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15742 00000100:00100000:0.0:1789760781.071360:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724297626985 time 724311085472 state 0x2 ref 0 00000100:00100000:2.0:1789760781.075053:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760781.075982:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554779776 00000100:00100000:0.0:1789760781.075997:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554779776 00000100:00100000:0.0:1789760781.076083:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724297626985 time 724311085472 state 0x0 ref 1 00000100:00100000:0.0:1789760781.076103:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724311345888 time 724311085472 state 0x1 ref 1 00000100:00100000:0.0:1789760781.076140:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724311345888 time 724321757799 state 0x0 ref 1 00000100:00100000:0.0:1789760781.076155:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a272a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554779776:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760781.076213:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64438272, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ebb8 00010000:00010000:0.0:1789760781.076225:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5862 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.076244:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760781.076250:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.076258:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5862 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64438272->64442367] (req 64438272->64442367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.076275:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5862 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64438272->64442367] (req 64438272->64442367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.082435:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983041 is committed 00002000:00010000:0.0:1789760781.082638:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64438272 -> 64442368 00002000:00010000:0.0:1789760781.082644:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125880 -> 125888 00010000:00010000:0.0:1789760781.082662:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5862 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64438272->64442367] (req 64438272->64442367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.082678:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5862 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64438272->64442367] (req 64438272->64442367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.082703:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5862 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64438272->64442367] (req 64438272->64442367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.082712:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760781.082718:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5862 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64438272->64442367] (req 64438272->64442367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.082728:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5862 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64438272->64442367] (req 64438272->64442367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.082753:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5862 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64438272->64442367] (req 64438272->64442367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.082761:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5862 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64438272->64442367] (req 64438272->64442367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.082883:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a272a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554779776:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6725us (7836us total) trans 4294983041 rc 0/0 00000100:00100000:0.0:1789760781.082896:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15743 00000100:00100000:0.0:1789760781.082908:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724311345888 time 724321757799 state 0x2 ref 0 00000100:00100000:3.0:1789760781.087353:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760781.087569:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554779904 00000100:00100000:1.0:1789760781.087585:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554779904 00000100:00100000:1.0:1789760781.087649:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724311345888 time 724321757799 state 0x0 ref 1 00000100:00100000:1.0:1789760781.087665:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724322018215 time 724321757799 state 0x1 ref 1 00000100:00100000:1.0:1789760781.087781:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724322018215 time 724333399039 state 0x0 ref 1 00000100:00100000:1.0:1789760781.087796:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06490a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554779904:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760781.087845:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64442368, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ebc0 00010000:00010000:1.0:1789760781.087854:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5869 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.087870:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760781.087874:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.087880:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5869 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64442368->64446463] (req 64442368->64446463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.087894:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5869 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64442368->64446463] (req 64442368->64446463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.097433:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983042 is committed 00002000:00010000:1.0:1789760781.097581:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64442368 -> 64446464 00002000:00010000:1.0:1789760781.097586:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125888 -> 125896 00010000:00010000:1.0:1789760781.097600:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5869 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64442368->64446463] (req 64442368->64446463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.097612:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5869 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64442368->64446463] (req 64442368->64446463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.097621:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5869 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64442368->64446463] (req 64442368->64446463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.097633:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760781.097638:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5869 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64442368->64446463] (req 64442368->64446463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.097647:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5869 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64442368->64446463] (req 64442368->64446463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.097777:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5869 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64442368->64446463] (req 64442368->64446463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.097786:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5869 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64442368->64446463] (req 64442368->64446463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.097892:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06490a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554779904:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10093us (10547us total) trans 4294983042 rc 0/0 00000100:00100000:1.0:1789760781.097903:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15744 00000100:00100000:1.0:1789760781.097914:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724322018215 time 724333399039 state 0x2 ref 0 00000100:00100000:3.0:1789760781.101252:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760781.101426:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554780032 00000100:00100000:2.0:1789760781.101438:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554780032 00000100:00100000:2.0:1789760781.101613:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724322018215 time 724333399039 state 0x0 ref 1 00000100:00100000:2.0:1789760781.101629:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724333659455 time 724333399039 state 0x1 ref 1 00000100:00100000:2.0:1789760781.101728:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724333659455 time 724347345436 state 0x0 ref 1 00000100:00100000:2.0:1789760781.101743:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06491c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554780032:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760781.101790:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64446464, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ebc8 00010000:00010000:2.0:1789760781.101800:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5870 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.101814:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760781.101819:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.101825:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5870 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64446464->64450559] (req 64446464->64450559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.101840:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5870 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64446464->64450559] (req 64446464->64450559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.106684:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983043 is committed 00002000:00010000:3.0:1789760781.106814:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64446464 -> 64450560 00002000:00010000:3.0:1789760781.106819:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125896 -> 125904 00010000:00010000:3.0:1789760781.106834:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5870 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64446464->64450559] (req 64446464->64450559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.106845:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5870 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64446464->64450559] (req 64446464->64450559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.106854:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5870 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64446464->64450559] (req 64446464->64450559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.106861:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760781.106867:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5870 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64446464->64450559] (req 64446464->64450559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.106875:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5870 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64446464->64450559] (req 64446464->64450559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.106894:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5870 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64446464->64450559] (req 64446464->64450559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.106905:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5870 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64446464->64450559] (req 64446464->64450559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.107001:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06491c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554780032:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5256us (5758us total) trans 4294983043 rc 0/0 00000100:00100000:3.0:1789760781.107011:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15745 00000100:00100000:3.0:1789760781.107021:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724333659455 time 724347345436 state 0x2 ref 0 00000100:00100000:2.0:1789760781.110665:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760781.111035:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554780160 00000100:00100000:0.0:1789760781.111044:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554780160 00000100:00100000:0.0:1789760781.111098:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724333659455 time 724347345436 state 0x0 ref 1 00000100:00100000:0.0:1789760781.111114:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724347605852 time 724347345436 state 0x1 ref 1 00000100:00100000:0.0:1789760781.111145:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724347605852 time 724356763549 state 0x0 ref 1 00000100:00100000:0.0:1789760781.111157:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a270380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554780160:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760781.111204:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64450560, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ebd0 00010000:00010000:0.0:1789760781.111213:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5877 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.111980:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760781.111985:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.111992:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5877 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64450560->64454655] (req 64450560->64454655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.112006:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5877 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64450560->64454655] (req 64450560->64454655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.117202:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983044 is committed 00002000:00010000:0.0:1789760781.117431:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64450560 -> 64454656 00002000:00010000:0.0:1789760781.117436:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125904 -> 125912 00010000:00010000:0.0:1789760781.117449:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5877 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64450560->64454655] (req 64450560->64454655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.117464:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5877 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64450560->64454655] (req 64450560->64454655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.117473:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5877 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64450560->64454655] (req 64450560->64454655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.117479:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760781.117485:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5877 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64450560->64454655] (req 64450560->64454655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.117494:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5877 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64450560->64454655] (req 64450560->64454655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.117518:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5877 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64450560->64454655] (req 64450560->64454655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.117525:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5877 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64450560->64454655] (req 64450560->64454655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.117656:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a270380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554780160:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6495us (6994us total) trans 4294983044 rc 0/0 00000100:00100000:0.0:1789760781.117667:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15746 00000100:00100000:0.0:1789760781.117676:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724347605852 time 724356763549 state 0x2 ref 0 00000100:00100000:0.0:1789760781.123929:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760781.124081:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554780288 00000100:00100000:2.0:1789760781.124092:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554780288 00000100:00100000:2.0:1789760781.124143:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724347605852 time 724356763549 state 0x0 ref 1 00000100:00100000:2.0:1789760781.124157:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724357023965 time 724356763549 state 0x1 ref 1 00000100:00100000:2.0:1789760781.124184:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724357023965 time 724369802562 state 0x0 ref 1 00000100:00100000:2.0:1789760781.124195:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554780288:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760781.124257:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64454656, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ebd8 00010000:00010000:2.0:1789760781.124267:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c587e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.124286:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760781.124291:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.124298:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c587e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64454656->64458751] (req 64454656->64458751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.124312:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c587e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64454656->64458751] (req 64454656->64458751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.132287:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983045 is committed 00002000:00010000:2.0:1789760781.132458:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64454656 -> 64458752 00002000:00010000:2.0:1789760781.132462:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125912 -> 125920 00010000:00010000:2.0:1789760781.132476:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c587e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64454656->64458751] (req 64454656->64458751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.132496:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c587e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64454656->64458751] (req 64454656->64458751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.132505:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c587e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64454656->64458751] (req 64454656->64458751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.132512:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760781.132517:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c587e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64454656->64458751] (req 64454656->64458751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.132525:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c587e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64454656->64458751] (req 64454656->64458751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.132549:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c587e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64454656->64458751] (req 64454656->64458751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.132557:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c587e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64454656->64458751] (req 64454656->64458751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.132643:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554780288:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8445us (8720us total) trans 4294983045 rc 0/0 00000100:00100000:2.0:1789760781.132654:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15747 00000100:00100000:2.0:1789760781.132663:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724357023965 time 724369802562 state 0x2 ref 0 00000100:00100000:3.0:1789760781.143454:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760781.144013:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554780416 00000100:00100000:1.0:1789760781.144027:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554780416 00000100:00100000:1.0:1789760781.144193:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724357023965 time 724369802562 state 0x0 ref 1 00000100:00100000:1.0:1789760781.144213:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724370062978 time 724369802562 state 0x1 ref 1 00000100:00100000:1.0:1789760781.145323:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724370062978 time 724390932014 state 0x0 ref 1 00000100:00100000:1.0:1789760781.145359:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06490000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554780416:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760781.145457:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64458752, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ebe0 00010000:00010000:1.0:1789760781.145470:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5885 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.145491:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760781.145497:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.145505:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5885 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64458752->64462847] (req 64458752->64462847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.145522:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5885 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64458752->64462847] (req 64458752->64462847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.153285:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983046 is committed 00002000:00010000:1.0:1789760781.153808:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64458752 -> 64462848 00002000:00010000:1.0:1789760781.153815:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125920 -> 125928 00010000:00010000:1.0:1789760781.153832:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5885 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64458752->64462847] (req 64458752->64462847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.153848:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5885 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64458752->64462847] (req 64458752->64462847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.153859:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5885 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64458752->64462847] (req 64458752->64462847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.153868:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760781.153875:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5885 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64458752->64462847] (req 64458752->64462847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.153897:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5885 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64458752->64462847] (req 64458752->64462847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.153921:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5885 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64458752->64462847] (req 64458752->64462847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.153930:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5885 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64458752->64462847] (req 64458752->64462847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.154216:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06490000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554780416:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8857us (10762us total) trans 4294983046 rc 0/0 00000100:00100000:1.0:1789760781.154231:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15748 00000100:00100000:1.0:1789760781.154243:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724370062978 time 724390932014 state 0x2 ref 0 00000100:00100000:0.0:1789760781.158350:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760781.158636:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554780544 00000100:00100000:3.0:1789760781.158651:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554780544 00000100:00100000:3.0:1789760781.158737:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724370062978 time 724390932014 state 0x0 ref 1 00000100:00100000:3.0:1789760781.158756:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724391192430 time 724390932014 state 0x1 ref 1 00000100:00100000:3.0:1789760781.158797:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724391192430 time 724404415081 state 0x0 ref 1 00000100:00100000:3.0:1789760781.158811:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04562d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554780544:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760781.158874:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64462848, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ebe8 00010000:00010000:3.0:1789760781.158885:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c588c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.158906:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760781.158911:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.158917:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c588c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64462848->64466943] (req 64462848->64466943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.158935:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c588c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64462848->64466943] (req 64462848->64466943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.166744:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983047 is committed 00002000:00010000:3.0:1789760781.166876:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64462848 -> 64466944 00002000:00010000:3.0:1789760781.166883:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125928 -> 125936 00010000:00010000:3.0:1789760781.166899:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c588c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64462848->64466943] (req 64462848->64466943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.166914:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c588c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64462848->64466943] (req 64462848->64466943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.166924:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c588c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64462848->64466943] (req 64462848->64466943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.166932:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760781.166938:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c588c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64462848->64466943] (req 64462848->64466943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.166948:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c588c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64462848->64466943] (req 64462848->64466943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.166970:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c588c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64462848->64466943] (req 64462848->64466943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.166978:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c588c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64462848->64466943] (req 64462848->64466943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.167249:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04562d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554780544:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8432us (9036us total) trans 4294983047 rc 0/0 00000100:00100000:3.0:1789760781.167262:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15749 00000100:00100000:3.0:1789760781.167275:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724391192430 time 724404415081 state 0x2 ref 0 00000100:00100000:0.0:1789760781.177285:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760781.177426:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554780672 00000100:00100000:1.0:1789760781.177439:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554780672 00000100:00100000:1.0:1789760781.177519:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724391192430 time 724404415081 state 0x0 ref 1 00000100:00100000:1.0:1789760781.177538:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724404675497 time 724404415081 state 0x1 ref 1 00000100:00100000:1.0:1789760781.177567:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724404675497 time 724423185312 state 0x0 ref 1 00000100:00100000:1.0:1789760781.177581:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04563800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554780672:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760781.177648:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64466944, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ebf0 00010000:00010000:1.0:1789760781.177661:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5893 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.177680:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760781.177685:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.177704:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5893 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64466944->64471039] (req 64466944->64471039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.177721:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5893 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64466944->64471039] (req 64466944->64471039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.186713:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983048 is committed 00002000:00010000:0.0:1789760781.186845:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64466944 -> 64471040 00002000:00010000:0.0:1789760781.186852:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125936 -> 125944 00010000:00010000:0.0:1789760781.186870:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5893 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64466944->64471039] (req 64466944->64471039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.186885:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5893 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64466944->64471039] (req 64466944->64471039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.186896:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5893 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64466944->64471039] (req 64466944->64471039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.186912:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760781.186918:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5893 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64466944->64471039] (req 64466944->64471039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.186929:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5893 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64466944->64471039] (req 64466944->64471039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.186952:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5893 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64466944->64471039] (req 64466944->64471039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.186960:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5893 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64466944->64471039] (req 64466944->64471039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.187241:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04563800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554780672:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9655us (9962us total) trans 4294983048 rc 0/0 00000100:00100000:0.0:1789760781.187255:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15750 00000100:00100000:0.0:1789760781.187267:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724404675497 time 724423185312 state 0x2 ref 0 00000100:00100000:0.0:1789760781.198259:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760781.198413:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554780800 00000100:00100000:2.0:1789760781.198427:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554780800 00000100:00100000:2.0:1789760781.198727:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724404675497 time 724423185312 state 0x0 ref 1 00000100:00100000:2.0:1789760781.198748:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724423445728 time 724423185312 state 0x1 ref 1 00000100:00100000:2.0:1789760781.198785:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724423445728 time 724444403713 state 0x0 ref 1 00000100:00100000:2.0:1789760781.198800:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554780800:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760781.198864:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64471040, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ebf8 00010000:00010000:2.0:1789760781.198876:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c589a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.198905:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760781.198910:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.198918:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c589a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64471040->64475135] (req 64471040->64475135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.198934:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c589a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64471040->64475135] (req 64471040->64475135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.207507:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983049 is committed 00002000:00010000:2.0:1789760781.207654:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64471040 -> 64475136 00002000:00010000:2.0:1789760781.207660:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125944 -> 125952 00010000:00010000:2.0:1789760781.207678:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c589a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64471040->64475135] (req 64471040->64475135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.207705:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c589a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64471040->64475135] (req 64471040->64475135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.207716:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c589a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64471040->64475135] (req 64471040->64475135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.207724:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760781.207730:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c589a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64471040->64475135] (req 64471040->64475135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.207739:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c589a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64471040->64475135] (req 64471040->64475135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.207768:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c589a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64471040->64475135] (req 64471040->64475135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.207776:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c589a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64471040->64475135] (req 64471040->64475135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.207949:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554780800:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9144us (9699us total) trans 4294983049 rc 0/0 00000100:00100000:2.0:1789760781.207961:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15751 00000100:00100000:2.0:1789760781.207972:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724423445728 time 724444403713 state 0x2 ref 0 00000100:00100000:2.0:1789760781.214808:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760781.214943:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554780928 00000100:00100000:3.0:1789760781.214957:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554780928 00000100:00100000:3.0:1789760781.215039:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724423445728 time 724444403713 state 0x0 ref 1 00000100:00100000:3.0:1789760781.215060:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724444664129 time 724444403713 state 0x1 ref 1 00000100:00100000:3.0:1789760781.215247:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724444664129 time 724460864361 state 0x0 ref 1 00000100:00100000:3.0:1789760781.215262:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a270700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554780928:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760781.215327:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64475136, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ec00 00010000:00010000:3.0:1789760781.215338:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c58a1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.215359:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760781.215364:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.215373:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c58a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64475136->64479231] (req 64475136->64479231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.215425:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c58a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64475136->64479231] (req 64475136->64479231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.222004:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983050 is committed 00002000:00010000:2.0:1789760781.222138:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64475136 -> 64479232 00002000:00010000:2.0:1789760781.222145:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125952 -> 125960 00010000:00010000:2.0:1789760781.222174:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c58a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64475136->64479231] (req 64475136->64479231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.222189:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c58a1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64475136->64479231] (req 64475136->64479231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.222199:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c58a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64475136->64479231] (req 64475136->64479231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.222208:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760781.222213:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c58a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64475136->64479231] (req 64475136->64479231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.222224:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c58a1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64475136->64479231] (req 64475136->64479231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.222249:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c58a1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64475136->64479231] (req 64475136->64479231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.222257:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c58a1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64475136->64479231] (req 64475136->64479231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.222414:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a270700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554780928:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7110us (7574us total) trans 4294983050 rc 0/0 00000100:00100000:2.0:1789760781.222428:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15752 00000100:00100000:2.0:1789760781.222441:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724444664129 time 724460864361 state 0x2 ref 0 00000100:00100000:2.0:1789760781.235869:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760781.236591:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554781056 00000100:00100000:3.0:1789760781.236605:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554781056 00000100:00100000:3.0:1789760781.236916:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724444664129 time 724460864361 state 0x0 ref 1 00000100:00100000:3.0:1789760781.236936:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724461124777 time 724460864361 state 0x1 ref 1 00000100:00100000:3.0:1789760781.237030:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724461124777 time 724482646161 state 0x0 ref 1 00000100:00100000:3.0:1789760781.237047:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554781056:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760781.237130:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64479232, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ec08 00010000:00010000:3.0:1789760781.237142:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c58a8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.237162:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760781.237168:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.237177:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c58a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64479232->64483327] (req 64479232->64483327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.237194:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c58a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64479232->64483327] (req 64479232->64483327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.244873:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983051 is committed 00002000:00010000:2.0:1789760781.245248:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64479232 -> 64483328 00002000:00010000:2.0:1789760781.245255:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125960 -> 125968 00010000:00010000:2.0:1789760781.245273:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c58a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64479232->64483327] (req 64479232->64483327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.245288:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c58a8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64479232->64483327] (req 64479232->64483327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.245299:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c58a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64479232->64483327] (req 64479232->64483327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.245313:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760781.245319:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c58a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64479232->64483327] (req 64479232->64483327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.245330:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c58a8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64479232->64483327] (req 64479232->64483327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.245356:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c58a8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64479232->64483327] (req 64479232->64483327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.245364:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c58a8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64479232->64483327] (req 64479232->64483327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.246996:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554781056:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9938us (11128us total) trans 4294983051 rc 0/0 00000100:00100000:2.0:1789760781.247015:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15753 00000100:00100000:2.0:1789760781.247027:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724461124777 time 724482646161 state 0x2 ref 0 00000100:00100000:0.0:1789760781.252086:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760781.252409:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554781184 00000100:00100000:1.0:1789760781.252425:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554781184 00000100:00100000:1.0:1789760781.252522:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724461124777 time 724482646161 state 0x0 ref 1 00000100:00100000:1.0:1789760781.252544:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724482906577 time 724482646161 state 0x1 ref 1 00000100:00100000:1.0:1789760781.252650:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724482906577 time 724498266293 state 0x0 ref 1 00000100:00100000:1.0:1789760781.252666:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554781184:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760781.252745:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64483328, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ec10 00010000:00010000:1.0:1789760781.252872:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c58af lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.252898:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760781.252903:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.252911:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c58af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64483328->64487423] (req 64483328->64487423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.252928:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c58af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64483328->64487423] (req 64483328->64487423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.264165:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983052 is committed 00002000:00010000:1.0:1789760781.264363:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64483328 -> 64487424 00002000:00010000:1.0:1789760781.264370:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125968 -> 125976 00010000:00010000:1.0:1789760781.264408:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c58af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64483328->64487423] (req 64483328->64487423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.264423:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c58af lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64483328->64487423] (req 64483328->64487423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.264432:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c58af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64483328->64487423] (req 64483328->64487423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.264516:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760781.264522:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c58af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64483328->64487423] (req 64483328->64487423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.264534:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c58af lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64483328->64487423] (req 64483328->64487423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.264565:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c58af lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64483328->64487423] (req 64483328->64487423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.264574:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c58af lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64483328->64487423] (req 64483328->64487423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.264813:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554781184:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12143us (12746us total) trans 4294983052 rc 0/0 00000100:00100000:1.0:1789760781.264827:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15754 00000100:00100000:1.0:1789760781.264839:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724482906577 time 724498266293 state 0x2 ref 0 00000100:00100000:2.0:1789760781.268915:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760781.269045:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554781312 00000100:00100000:0.0:1789760781.269057:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554781312 00000100:00100000:0.0:1789760781.269141:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724482906577 time 724498266293 state 0x0 ref 1 00000100:00100000:0.0:1789760781.269164:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724498526709 time 724498266293 state 0x1 ref 1 00000100:00100000:0.0:1789760781.269192:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724498526709 time 724514810580 state 0x0 ref 1 00000100:00100000:0.0:1789760781.269208:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554781312:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760781.269269:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64487424, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ec18 00010000:00010000:0.0:1789760781.269281:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58b6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.269301:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760781.269306:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.269314:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64487424->64491519] (req 64487424->64491519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.269331:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64487424->64491519] (req 64487424->64491519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.279005:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983053 is committed 00002000:00010000:0.0:1789760781.280743:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64487424 -> 64491520 00002000:00010000:0.0:1789760781.280755:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125976 -> 125984 00010000:00010000:0.0:1789760781.280772:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64487424->64491519] (req 64487424->64491519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.280786:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58b6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64487424->64491519] (req 64487424->64491519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.280796:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64487424->64491519] (req 64487424->64491519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.280805:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760781.280810:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64487424->64491519] (req 64487424->64491519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.280820:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58b6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64487424->64491519] (req 64487424->64491519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.280844:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58b6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64487424->64491519] (req 64487424->64491519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.280853:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58b6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64487424->64491519] (req 64487424->64491519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.280989:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554781312:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11776us (12081us total) trans 4294983053 rc 0/0 00000100:00100000:0.0:1789760781.281002:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15755 00000100:00100000:0.0:1789760781.281013:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724498526709 time 724514810580 state 0x2 ref 0 00000100:00100000:2.0:1789760781.284545:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760781.284837:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554781440 00000100:00100000:1.0:1789760781.284850:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554781440 00000100:00100000:1.0:1789760781.284924:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724498526709 time 724514810580 state 0x0 ref 1 00000100:00100000:1.0:1789760781.284945:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724515070996 time 724514810580 state 0x1 ref 1 00000100:00100000:1.0:1789760781.284973:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724515070996 time 724530591644 state 0x0 ref 1 00000100:00100000:1.0:1789760781.284986:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554781440:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760781.285050:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64491520, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ec20 00010000:00010000:1.0:1789760781.285061:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c58bd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.285096:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760781.285101:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.285109:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c58bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64491520->64495615] (req 64491520->64495615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.285126:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c58bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64491520->64495615] (req 64491520->64495615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.296935:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983054 is committed 00002000:00010000:1.0:1789760781.298329:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64491520 -> 64495616 00002000:00010000:1.0:1789760781.298339:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125984 -> 125992 00010000:00010000:1.0:1789760781.298356:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c58bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64491520->64495615] (req 64491520->64495615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.298371:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c58bd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64491520->64495615] (req 64491520->64495615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.298426:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c58bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64491520->64495615] (req 64491520->64495615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.298435:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760781.298440:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c58bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64491520->64495615] (req 64491520->64495615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.298451:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c58bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64491520->64495615] (req 64491520->64495615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.298480:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c58bd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64491520->64495615] (req 64491520->64495615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.298488:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c58bd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64491520->64495615] (req 64491520->64495615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.299348:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554781440:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14349us (14804us total) trans 4294983054 rc 0/0 00000100:00100000:1.0:1789760781.299364:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15756 00000100:00100000:1.0:1789760781.299777:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724515070996 time 724530591644 state 0x2 ref 0 00000100:00100000:3.0:1789760781.302753:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760781.303604:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554781568 00000100:00100000:2.0:1789760781.303740:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554781568 00000100:00100000:2.0:1789760781.303885:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724515070996 time 724530591644 state 0x0 ref 1 00000100:00100000:2.0:1789760781.303906:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724530852060 time 724530591644 state 0x1 ref 1 00000100:00100000:2.0:1789760781.303946:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724530852060 time 724549563614 state 0x0 ref 1 00000100:00100000:2.0:1789760781.303962:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554781568:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760781.304031:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64495616, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ec28 00010000:00010000:2.0:1789760781.304042:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c58c4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.304060:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760781.304075:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.304083:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c58c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64495616->64499711] (req 64495616->64499711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.304100:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c58c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64495616->64499711] (req 64495616->64499711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.312249:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983055 is committed 00002000:00010000:2.0:1789760781.312416:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64495616 -> 64499712 00002000:00010000:2.0:1789760781.312423:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 125992 -> 126000 00010000:00010000:2.0:1789760781.312440:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c58c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64495616->64499711] (req 64495616->64499711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.312454:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c58c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64495616->64499711] (req 64495616->64499711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.312463:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c58c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64495616->64499711] (req 64495616->64499711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.312477:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760781.312483:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c58c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64495616->64499711] (req 64495616->64499711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.312494:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c58c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64495616->64499711] (req 64495616->64499711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.312517:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c58c4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64495616->64499711] (req 64495616->64499711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.312526:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c58c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64495616->64499711] (req 64495616->64499711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.312729:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554781568:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8763us (9985us total) trans 4294983055 rc 0/0 00000100:00100000:2.0:1789760781.312741:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15757 00000100:00100000:2.0:1789760781.312754:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724530852060 time 724549563614 state 0x2 ref 0 00000100:00100000:2.0:1789760781.316490:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760781.316590:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554781696 00000100:00100000:0.0:1789760781.316604:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554781696 00000100:00100000:0.0:1789760781.316748:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724530852060 time 724549563614 state 0x0 ref 1 00000100:00100000:0.0:1789760781.316778:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724549824030 time 724549563614 state 0x1 ref 1 00000100:00100000:0.0:1789760781.316816:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724549824030 time 724562434797 state 0x0 ref 1 00000100:00100000:0.0:1789760781.316830:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554781696:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760781.316899:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64499712, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ec30 00010000:00010000:0.0:1789760781.316911:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58cb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.316929:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760781.316935:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.316943:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64499712->64503807] (req 64499712->64503807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.316959:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64499712->64503807] (req 64499712->64503807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.329312:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983056 is committed 00002000:00010000:1.0:1789760781.329519:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64499712 -> 64503808 00002000:00010000:1.0:1789760781.329527:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126000 -> 126008 00010000:00010000:1.0:1789760781.329547:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64499712->64503807] (req 64499712->64503807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.329566:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58cb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64499712->64503807] (req 64499712->64503807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.329578:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64499712->64503807] (req 64499712->64503807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.329587:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760781.329593:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64499712->64503807] (req 64499712->64503807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.330131:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58cb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64499712->64503807] (req 64499712->64503807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.330161:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58cb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64499712->64503807] (req 64499712->64503807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.330285:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58cb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64499712->64503807] (req 64499712->64503807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.331796:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554781696:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14952us (15310us total) trans 4294983056 rc 0/0 00000100:00100000:1.0:1789760781.331815:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15758 00000100:00100000:1.0:1789760781.331827:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724549824030 time 724562434797 state 0x2 ref 0 00000100:00100000:3.0:1789760781.338845:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760781.340134:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554781824 00000100:00100000:0.0:1789760781.340151:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554781824 00000100:00100000:0.0:1789760781.340232:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724549824030 time 724562434797 state 0x0 ref 1 00000100:00100000:0.0:1789760781.340252:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724562695213 time 724562434797 state 0x1 ref 1 00000100:00100000:0.0:1789760781.340549:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724562695213 time 724586166147 state 0x0 ref 1 00000100:00100000:0.0:1789760781.340561:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06492680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554781824:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760781.340621:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64503808, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ec38 00010000:00010000:0.0:1789760781.340630:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58d2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.340647:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760781.340651:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.340658:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64503808->64507903] (req 64503808->64507903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.340672:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64503808->64507903] (req 64503808->64507903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.349369:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983057 is committed 00002000:00010000:1.0:1789760781.354049:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64503808 -> 64507904 00002000:00010000:1.0:1789760781.354064:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126008 -> 126016 00010000:00010000:1.0:1789760781.354085:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64503808->64507903] (req 64503808->64507903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.354101:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64503808->64507903] (req 64503808->64507903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.354110:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64503808->64507903] (req 64503808->64507903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.354126:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760781.354133:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64503808->64507903] (req 64503808->64507903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.354142:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64503808->64507903] (req 64503808->64507903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.354167:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58d2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64503808->64507903] (req 64503808->64507903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.354176:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58d2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64503808->64507903] (req 64503808->64507903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.354821:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06492680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554781824:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14250us (15984us total) trans 4294983057 rc 0/0 00000100:00100000:1.0:1789760781.354835:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15759 00000100:00100000:1.0:1789760781.354847:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724562695213 time 724586166147 state 0x2 ref 0 00000100:00100000:0.0:1789760781.358854:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760781.359014:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554781952 00000100:00100000:1.0:1789760781.359030:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554781952 00000100:00100000:1.0:1789760781.359241:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724562695213 time 724586166147 state 0x0 ref 1 00000100:00100000:1.0:1789760781.359259:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724586426563 time 724586166147 state 0x1 ref 1 00000100:00100000:1.0:1789760781.359570:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724586426563 time 724605185751 state 0x0 ref 1 00000100:00100000:1.0:1789760781.359591:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04562a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554781952:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760781.359903:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64507904, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ec40 00010000:00010000:2.0:1789760781.359918:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c58d9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.359940:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760781.359946:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.359956:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c58d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64507904->64511999] (req 64507904->64511999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.359974:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c58d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64507904->64511999] (req 64507904->64511999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.367821:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983058 is committed 00002000:00010000:2.0:1789760781.367977:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64507904 -> 64512000 00002000:00010000:2.0:1789760781.367983:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126016 -> 126024 00010000:00010000:2.0:1789760781.368000:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c58d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64507904->64511999] (req 64507904->64511999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.368015:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c58d9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64507904->64511999] (req 64507904->64511999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.368026:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c58d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64507904->64511999] (req 64507904->64511999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.368034:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760781.368039:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c58d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64507904->64511999] (req 64507904->64511999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.368058:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c58d9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64507904->64511999] (req 64507904->64511999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.368091:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c58d9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64507904->64511999] (req 64507904->64511999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.368104:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c58d9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64507904->64511999] (req 64507904->64511999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.368234:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04562a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554781952:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8640us (9386us total) trans 4294983058 rc 0/0 00000100:00100000:2.0:1789760781.368248:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15760 00000100:00100000:2.0:1789760781.368261:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724586426563 time 724605185751 state 0x2 ref 0 00000100:00100000:2.0:1789760781.377632:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760781.377949:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554782080 00000100:00100000:0.0:1789760781.378141:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554782080 00000100:00100000:0.0:1789760781.379371:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724586426563 time 724605185751 state 0x0 ref 1 00000100:00100000:0.0:1789760781.379410:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724605446167 time 724605185751 state 0x1 ref 1 00000100:00100000:0.0:1789760781.379445:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724605446167 time 724625063764 state 0x0 ref 1 00000100:00100000:0.0:1789760781.379462:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554782080:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760781.379528:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64512000, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ec48 00010000:00010000:0.0:1789760781.379541:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58e0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.379563:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760781.379570:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.379579:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64512000->64516095] (req 64512000->64516095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.379596:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64512000->64516095] (req 64512000->64516095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.393709:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983059 is committed 00002000:00010000:0.0:1789760781.393976:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64512000 -> 64516096 00002000:00010000:0.0:1789760781.393982:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126024 -> 126032 00010000:00010000:0.0:1789760781.394000:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64512000->64516095] (req 64512000->64516095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.394015:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58e0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64512000->64516095] (req 64512000->64516095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.394025:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64512000->64516095] (req 64512000->64516095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.394034:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760781.394040:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64512000->64516095] (req 64512000->64516095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.394051:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64512000->64516095] (req 64512000->64516095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.394091:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58e0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64512000->64516095] (req 64512000->64516095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.394100:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58e0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64512000->64516095] (req 64512000->64516095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.394234:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554782080:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14771us (16610us total) trans 4294983059 rc 0/0 00000100:00100000:0.0:1789760781.394247:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15761 00000100:00100000:0.0:1789760781.394258:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724605446167 time 724625063764 state 0x2 ref 0 00000100:00100000:3.0:1789760781.405444:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760781.405616:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554782208 00000100:00100000:0.0:1789760781.405630:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554782208 00000100:00100000:0.0:1789760781.405714:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724605446167 time 724625063764 state 0x0 ref 1 00000100:00100000:0.0:1789760781.405733:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724625324180 time 724625063764 state 0x1 ref 1 00000100:00100000:0.0:1789760781.405785:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724625324180 time 724651402285 state 0x0 ref 1 00000100:00100000:0.0:1789760781.405799:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554782208:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760781.405877:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64516096, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ec50 00010000:00010000:0.0:1789760781.405890:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58e7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.405916:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760781.405922:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.405931:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64516096->64520191] (req 64516096->64520191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.405948:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64516096->64520191] (req 64516096->64520191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.416490:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983060 is committed 00002000:00010000:0.0:1789760781.416682:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64516096 -> 64520192 00002000:00010000:0.0:1789760781.416698:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126032 -> 126040 00010000:00010000:0.0:1789760781.416712:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64516096->64520191] (req 64516096->64520191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.416725:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58e7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64516096->64520191] (req 64516096->64520191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.416733:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64516096->64520191] (req 64516096->64520191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.416740:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760781.416746:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64516096->64520191] (req 64516096->64520191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.416755:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58e7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64516096->64520191] (req 64516096->64520191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.416773:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58e7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64516096->64520191] (req 64516096->64520191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.416786:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58e7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64516096->64520191] (req 64516096->64520191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.418465:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554782208:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12656us (13024us total) trans 4294983060 rc 0/0 00000100:00100000:0.0:1789760781.418482:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15762 00000100:00100000:0.0:1789760781.418494:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724625324180 time 724651402285 state 0x2 ref 0 00000100:00100000:1.0:1789760781.422040:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760781.422126:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554782336 00000100:00100000:0.0:1789760781.422142:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554782336 00000100:00100000:0.0:1789760781.422200:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724625324180 time 724651402285 state 0x0 ref 1 00000100:00100000:0.0:1789760781.422215:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724651662701 time 724651402285 state 0x1 ref 1 00000100:00100000:0.0:1789760781.422252:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724651662701 time 724667870762 state 0x0 ref 1 00000100:00100000:0.0:1789760781.422265:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554782336:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760781.422312:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64520192, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ec58 00010000:00010000:0.0:1789760781.422320:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58ee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.422336:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760781.422341:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.422348:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64520192->64524287] (req 64520192->64524287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.422369:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64520192->64524287] (req 64520192->64524287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.432299:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983061 is committed 00002000:00010000:2.0:1789760781.432896:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64520192 -> 64524288 00002000:00010000:2.0:1789760781.432904:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126040 -> 126048 00010000:00010000:2.0:1789760781.432922:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64520192->64524287] (req 64520192->64524287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.432937:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64520192->64524287] (req 64520192->64524287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.432947:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64520192->64524287] (req 64520192->64524287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.432955:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760781.432960:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64520192->64524287] (req 64520192->64524287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.432969:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64520192->64524287] (req 64520192->64524287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.432993:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58ee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64520192->64524287] (req 64520192->64524287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.433208:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c58ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64520192->64524287] (req 64520192->64524287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.433329:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554782336:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11059us (11305us total) trans 4294983061 rc 0/0 00000100:00100000:2.0:1789760781.435629:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15763 00000100:00100000:2.0:1789760781.435651:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724651662701 time 724667870762 state 0x2 ref 0 00000100:00100000:2.0:1789760781.448251:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760781.448724:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554782464 00000100:00100000:0.0:1789760781.448738:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554782464 00000100:00100000:0.0:1789760781.448826:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724651662701 time 724667870762 state 0x0 ref 1 00000100:00100000:0.0:1789760781.448846:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724668131178 time 724667870762 state 0x1 ref 1 00000100:00100000:0.0:1789760781.448886:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724668131178 time 724694504168 state 0x0 ref 1 00000100:00100000:0.0:1789760781.448904:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554782464:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760781.448969:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64524288, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ec60 00010000:00010000:0.0:1789760781.448980:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58f5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.448999:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760781.449004:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.449012:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64524288->64528383] (req 64524288->64528383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.449029:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64524288->64528383] (req 64524288->64528383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.457091:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983062 is committed 00002000:00010000:2.0:1789760781.457300:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64524288 -> 64528384 00002000:00010000:2.0:1789760781.457306:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126048 -> 126056 00010000:00010000:2.0:1789760781.457324:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64524288->64528383] (req 64524288->64528383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.457339:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64524288->64528383] (req 64524288->64528383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.457348:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64524288->64528383] (req 64524288->64528383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.457356:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760781.457362:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64524288->64528383] (req 64524288->64528383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.457373:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64524288->64528383] (req 64524288->64528383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.457441:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58f5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64524288->64528383] (req 64524288->64528383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.457450:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c58f5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64524288->64528383] (req 64524288->64528383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.461499:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554782464:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12582us (13246us total) trans 4294983062 rc 0/0 00000100:00100000:1.0:1789760781.461520:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15764 00000100:00100000:1.0:1789760781.461534:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724668131178 time 724694504168 state 0x2 ref 0 00000100:00100000:3.0:1789760781.470622:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760781.470774:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554782592 00000100:00100000:0.0:1789760781.470786:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554782592 00000100:00100000:0.0:1789760781.470860:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724668131178 time 724694504168 state 0x0 ref 1 00000100:00100000:0.0:1789760781.470879:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724694764584 time 724694504168 state 0x1 ref 1 00000100:00100000:0.0:1789760781.470922:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724694764584 time 724716540104 state 0x0 ref 1 00000100:00100000:0.0:1789760781.470936:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06491180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554782592:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760781.470996:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64528384, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ec68 00010000:00010000:0.0:1789760781.471008:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c58fc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.471027:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760781.471033:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.471042:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c58fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64528384->64532479] (req 64528384->64532479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.471059:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c58fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64528384->64532479] (req 64528384->64532479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.479029:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983063 is committed 00002000:00010000:2.0:1789760781.481695:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64528384 -> 64532480 00002000:00010000:2.0:1789760781.481705:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126056 -> 126064 00010000:00010000:2.0:1789760781.481792:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c58fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64528384->64532479] (req 64528384->64532479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.481809:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c58fc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64528384->64532479] (req 64528384->64532479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.481820:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c58fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64528384->64532479] (req 64528384->64532479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.481828:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760781.481834:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c58fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64528384->64532479] (req 64528384->64532479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.481844:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c58fc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64528384->64532479] (req 64528384->64532479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.481870:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c58fc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64528384->64532479] (req 64528384->64532479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.481879:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c58fc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64528384->64532479] (req 64528384->64532479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.483873:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06491180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554782592:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12922us (13248us total) trans 4294983063 rc 0/0 00000100:00100000:2.0:1789760781.483893:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15765 00000100:00100000:2.0:1789760781.483907:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724694764584 time 724716540104 state 0x2 ref 0 00000100:00100000:0.0:1789760781.489222:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760781.489344:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554782720 00000100:00100000:1.0:1789760781.489357:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554782720 00000100:00100000:1.0:1789760781.489467:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724694764584 time 724716540104 state 0x0 ref 1 00000100:00100000:1.0:1789760781.489488:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724716800520 time 724716540104 state 0x1 ref 1 00000100:00100000:1.0:1789760781.489515:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724716800520 time 724735133587 state 0x0 ref 1 00000100:00100000:1.0:1789760781.489530:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04562300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554782720:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760781.489593:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64532480, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ec70 00010000:00010000:1.0:1789760781.489605:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5903 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.489625:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760781.489630:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.489638:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5903 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64532480->64536575] (req 64532480->64536575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.489656:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5903 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64532480->64536575] (req 64532480->64536575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.498638:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983064 is committed 00002000:00010000:1.0:1789760781.498853:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64532480 -> 64536576 00002000:00010000:1.0:1789760781.498860:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126064 -> 126072 00010000:00010000:1.0:1789760781.498878:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5903 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64532480->64536575] (req 64532480->64536575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.498893:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5903 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64532480->64536575] (req 64532480->64536575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.498902:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5903 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64532480->64536575] (req 64532480->64536575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.498909:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760781.498914:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5903 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64532480->64536575] (req 64532480->64536575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.498924:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5903 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64532480->64536575] (req 64532480->64536575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.498950:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5903 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64532480->64536575] (req 64532480->64536575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.498959:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5903 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64532480->64536575] (req 64532480->64536575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.499214:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04562300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554782720:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9678us (9999us total) trans 4294983064 rc 0/0 00000100:00100000:1.0:1789760781.499228:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15766 00000100:00100000:1.0:1789760781.499242:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724716800520 time 724735133587 state 0x2 ref 0 00000100:00100000:0.0:1789760781.509134:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760781.509257:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554782848 00000100:00100000:1.0:1789760781.509270:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554782848 00000100:00100000:1.0:1789760781.509340:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724716800520 time 724735133587 state 0x0 ref 1 00000100:00100000:1.0:1789760781.509358:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724735394003 time 724735133587 state 0x1 ref 1 00000100:00100000:1.0:1789760781.509427:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724735394003 time 724755045238 state 0x0 ref 1 00000100:00100000:1.0:1789760781.509441:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04560380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554782848:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760781.509501:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64536576, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ec78 00010000:00010000:1.0:1789760781.509512:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c590a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.509532:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760781.509538:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.509547:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c590a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64536576->64540671] (req 64536576->64540671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.509562:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c590a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64536576->64540671] (req 64536576->64540671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.514860:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983065 is committed 00002000:00010000:1.0:1789760781.516035:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64536576 -> 64540672 00002000:00010000:1.0:1789760781.516046:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126072 -> 126080 00010000:00010000:1.0:1789760781.516063:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c590a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64536576->64540671] (req 64536576->64540671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.516079:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c590a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64536576->64540671] (req 64536576->64540671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.516089:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c590a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64536576->64540671] (req 64536576->64540671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.516097:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760781.516104:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c590a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64536576->64540671] (req 64536576->64540671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.516181:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c590a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64536576->64540671] (req 64536576->64540671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.516211:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c590a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64536576->64540671] (req 64536576->64540671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.516219:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c590a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64536576->64540671] (req 64536576->64540671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.516338:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04560380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554782848:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6892us (7214us total) trans 4294983065 rc 0/0 00000100:00100000:1.0:1789760781.516555:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15767 00000100:00100000:1.0:1789760781.516568:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724735394003 time 724755045238 state 0x2 ref 0 00000100:00100000:1.0:1789760781.523855:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760781.523974:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554782976 00000100:00100000:0.0:1789760781.523988:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554782976 00000100:00100000:0.0:1789760781.524068:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724735394003 time 724755045238 state 0x0 ref 1 00000100:00100000:0.0:1789760781.524088:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724755305654 time 724755045238 state 0x1 ref 1 00000100:00100000:0.0:1789760781.524166:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724755305654 time 724769783413 state 0x0 ref 1 00000100:00100000:0.0:1789760781.524181:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554782976:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760781.524247:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64540672, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ec80 00010000:00010000:0.0:1789760781.524260:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5911 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.524281:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760781.524287:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.524294:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5911 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64540672->64544767] (req 64540672->64544767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.524310:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5911 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64540672->64544767] (req 64540672->64544767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.537845:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983066 is committed 00002000:00010000:0.0:1789760781.537981:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64540672 -> 64544768 00002000:00010000:0.0:1789760781.537988:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126080 -> 126088 00010000:00010000:0.0:1789760781.538006:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5911 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64540672->64544767] (req 64540672->64544767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.538021:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5911 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64540672->64544767] (req 64540672->64544767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.538031:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5911 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64540672->64544767] (req 64540672->64544767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.538040:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760781.538047:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5911 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64540672->64544767] (req 64540672->64544767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.538057:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5911 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64540672->64544767] (req 64540672->64544767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.538195:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5911 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64540672->64544767] (req 64540672->64544767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.538213:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5911 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64540672->64544767] (req 64540672->64544767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.538338:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554782976:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14153us (14492us total) trans 4294983066 rc 0/0 00000100:00100000:0.0:1789760781.538351:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15768 00000100:00100000:0.0:1789760781.538362:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724755305654 time 724769783413 state 0x2 ref 0 00000100:00100000:0.0:1789760781.541844:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760781.542005:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554783104 00000100:00100000:2.0:1789760781.542021:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554783104 00000100:00100000:2.0:1789760781.542217:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724755305654 time 724769783413 state 0x0 ref 1 00000100:00100000:2.0:1789760781.542238:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724770043829 time 724769783413 state 0x1 ref 1 00000100:00100000:2.0:1789760781.542273:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724770043829 time 724787891089 state 0x0 ref 1 00000100:00100000:2.0:1789760781.542289:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04563b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554783104:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760781.542352:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64544768, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ec88 00010000:00010000:2.0:1789760781.542364:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5918 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.542423:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760781.542429:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.542437:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5918 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64544768->64548863] (req 64544768->64548863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.542454:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5918 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64544768->64548863] (req 64544768->64548863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.550526:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983067 is committed 00002000:00010000:2.0:1789760781.550671:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64544768 -> 64548864 00002000:00010000:2.0:1789760781.550678:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126088 -> 126096 00010000:00010000:2.0:1789760781.550709:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5918 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64544768->64548863] (req 64544768->64548863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.550724:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5918 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64544768->64548863] (req 64544768->64548863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.550735:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5918 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64544768->64548863] (req 64544768->64548863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.550744:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760781.550750:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5918 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64544768->64548863] (req 64544768->64548863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.550760:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5918 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64544768->64548863] (req 64544768->64548863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.550790:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5918 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64544768->64548863] (req 64544768->64548863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.550799:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5918 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64544768->64548863] (req 64544768->64548863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.550925:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04563b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554783104:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8631us (9088us total) trans 4294983067 rc 0/0 00000100:00100000:2.0:1789760781.550938:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15769 00000100:00100000:2.0:1789760781.550951:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724770043829 time 724787891089 state 0x2 ref 0 00000100:00100000:0.0:1789760781.559765:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760781.559928:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554783232 00000100:00100000:1.0:1789760781.559945:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554783232 00000100:00100000:1.0:1789760781.560029:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724770043829 time 724787891089 state 0x0 ref 1 00000100:00100000:1.0:1789760781.560050:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724788151505 time 724787891089 state 0x1 ref 1 00000100:00100000:1.0:1789760781.560301:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724788151505 time 724805916686 state 0x0 ref 1 00000100:00100000:1.0:1789760781.560318:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554783232:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760781.560418:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64548864, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ec90 00010000:00010000:1.0:1789760781.560430:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c591f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.560451:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760781.560457:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.560465:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c591f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64548864->64552959] (req 64548864->64552959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.560482:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c591f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64548864->64552959] (req 64548864->64552959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.568869:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983068 is committed 00002000:00010000:1.0:1789760781.572504:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64548864 -> 64552960 00002000:00010000:1.0:1789760781.572518:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126096 -> 126104 00010000:00010000:1.0:1789760781.572536:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c591f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64548864->64552959] (req 64548864->64552959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.572552:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c591f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64548864->64552959] (req 64548864->64552959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.572562:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c591f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64548864->64552959] (req 64548864->64552959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.572570:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760781.572575:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c591f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64548864->64552959] (req 64548864->64552959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.572586:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c591f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64548864->64552959] (req 64548864->64552959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.572613:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c591f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64548864->64552959] (req 64548864->64552959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.572622:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c591f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64548864->64552959] (req 64548864->64552959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.572924:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554783232:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12601us (13167us total) trans 4294983068 rc 0/0 00000100:00100000:1.0:1789760781.572938:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15770 00000100:00100000:1.0:1789760781.572951:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724788151505 time 724805916686 state 0x2 ref 0 00000100:00100000:1.0:1789760781.584082:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760781.584232:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554783360 00000100:00100000:2.0:1789760781.584246:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554783360 00000100:00100000:2.0:1789760781.584320:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724788151505 time 724805916686 state 0x0 ref 1 00000100:00100000:2.0:1789760781.584339:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724806177102 time 724805916686 state 0x1 ref 1 00000100:00100000:2.0:1789760781.584370:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724806177102 time 724829989839 state 0x0 ref 1 00000100:00100000:2.0:1789760781.584381:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554783360:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760781.584484:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64552960, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ec98 00010000:00010000:2.0:1789760781.584492:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5926 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.584514:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760781.584519:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.584524:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5926 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64552960->64557055] (req 64552960->64557055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.584536:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5926 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64552960->64557055] (req 64552960->64557055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.593051:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983069 is committed 00002000:00010000:3.0:1789760781.593217:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64552960 -> 64557056 00002000:00010000:3.0:1789760781.593223:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126104 -> 126112 00010000:00010000:3.0:1789760781.593241:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5926 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64552960->64557055] (req 64552960->64557055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.593256:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5926 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64552960->64557055] (req 64552960->64557055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.593266:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5926 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64552960->64557055] (req 64552960->64557055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.593274:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760781.593280:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5926 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64552960->64557055] (req 64552960->64557055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.593290:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5926 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64552960->64557055] (req 64552960->64557055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.593316:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5926 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64552960->64557055] (req 64552960->64557055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.593324:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5926 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64552960->64557055] (req 64552960->64557055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.593729:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554783360:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9340us (9663us total) trans 4294983069 rc 0/0 00000100:00100000:3.0:1789760781.593741:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15771 00000100:00100000:3.0:1789760781.593755:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724806177102 time 724829989839 state 0x2 ref 0 00000100:00100000:1.0:1789760781.600447:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760781.600557:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554783488 00000100:00100000:0.0:1789760781.600571:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554783488 00000100:00100000:0.0:1789760781.600780:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724806177102 time 724829989839 state 0x0 ref 1 00000100:00100000:0.0:1789760781.600800:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724830250255 time 724829989839 state 0x1 ref 1 00000100:00100000:0.0:1789760781.600840:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724830250255 time 724846457726 state 0x0 ref 1 00000100:00100000:0.0:1789760781.600856:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554783488:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760781.600918:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64557056, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eca0 00010000:00010000:0.0:1789760781.600929:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c592d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.600948:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760781.600953:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.600961:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c592d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64557056->64561151] (req 64557056->64561151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.600977:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c592d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64557056->64561151] (req 64557056->64561151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.609198:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983070 is committed 00002000:00010000:1.0:1789760781.609417:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64557056 -> 64561152 00002000:00010000:1.0:1789760781.609423:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126112 -> 126120 00010000:00010000:1.0:1789760781.609665:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c592d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64557056->64561151] (req 64557056->64561151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.609682:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c592d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64557056->64561151] (req 64557056->64561151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.609834:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c592d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64557056->64561151] (req 64557056->64561151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.609842:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760781.609847:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c592d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64557056->64561151] (req 64557056->64561151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.609856:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c592d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64557056->64561151] (req 64557056->64561151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.609881:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c592d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64557056->64561151] (req 64557056->64561151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.609889:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c592d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64557056->64561151] (req 64557056->64561151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.611353:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554783488:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9515us (9932us total) trans 4294983070 rc 0/0 00000100:00100000:1.0:1789760781.611373:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15772 00000100:00100000:1.0:1789760781.611408:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724830250255 time 724846457726 state 0x2 ref 0 00000100:00100000:0.0:1789760781.615645:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760781.615746:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554783616 00000100:00100000:2.0:1789760781.615760:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554783616 00000100:00100000:2.0:1789760781.615846:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724830250255 time 724846457726 state 0x0 ref 1 00000100:00100000:2.0:1789760781.615866:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724846718142 time 724846457726 state 0x1 ref 1 00000100:00100000:2.0:1789760781.615902:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724846718142 time 724861520710 state 0x0 ref 1 00000100:00100000:2.0:1789760781.615916:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554783616:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760781.616863:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64561152, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1eca8 00010000:00010000:2.0:1789760781.616881:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5934 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.616902:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760781.616907:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.616914:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5934 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64561152->64565247] (req 64561152->64565247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.616932:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5934 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64561152->64565247] (req 64561152->64565247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.624281:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983071 is committed 00002000:00010000:2.0:1789760781.624620:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64561152 -> 64565248 00002000:00010000:2.0:1789760781.624627:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126120 -> 126128 00010000:00010000:2.0:1789760781.624644:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5934 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64561152->64565247] (req 64561152->64565247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.624659:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5934 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64561152->64565247] (req 64561152->64565247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.624669:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5934 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64561152->64565247] (req 64561152->64565247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.624676:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760781.624682:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5934 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64561152->64565247] (req 64561152->64565247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.624701:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5934 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64561152->64565247] (req 64561152->64565247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.624726:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5934 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64561152->64565247] (req 64561152->64565247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.624734:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5934 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64561152->64565247] (req 64561152->64565247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.624956:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554783616:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9035us (9571us total) trans 4294983071 rc 0/0 00000100:00100000:2.0:1789760781.624970:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15773 00000100:00100000:2.0:1789760781.624995:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724846718142 time 724861520710 state 0x2 ref 0 00000100:00100000:2.0:1789760781.629936:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760781.630256:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554783744 00000100:00100000:1.0:1789760781.630270:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554783744 00000100:00100000:1.0:1789760781.630410:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724846718142 time 724861520710 state 0x0 ref 1 00000100:00100000:1.0:1789760781.630429:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724861781126 time 724861520710 state 0x1 ref 1 00000100:00100000:1.0:1789760781.630505:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724861781126 time 724876122649 state 0x0 ref 1 00000100:00100000:1.0:1789760781.630520:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554783744:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760781.630585:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64565248, mtime: 0x6aad950c, atime: 0x0, ctime: 0x6aad950c, blocks: 0x1ecb0 00010000:00010000:1.0:1789760781.630596:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c593b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.630617:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760781.630622:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.630630:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c593b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64565248->64569343] (req 64565248->64569343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.630647:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c593b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64565248->64569343] (req 64565248->64569343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.637252:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983072 is committed 00002000:00010000:1.0:1789760781.637479:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64565248 -> 64569344 00002000:00010000:1.0:1789760781.637485:0:8441:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760780 -> 1789760781 00002000:00010000:1.0:1789760781.637489:0:8441:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760780 -> 1789760781 00002000:00010000:1.0:1789760781.637492:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126128 -> 126136 00010000:00010000:1.0:1789760781.637511:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c593b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64565248->64569343] (req 64565248->64569343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.637526:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c593b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64565248->64569343] (req 64565248->64569343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.637536:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c593b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64565248->64569343] (req 64565248->64569343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.637544:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760781.637550:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c593b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64565248->64569343] (req 64565248->64569343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.637559:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c593b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64565248->64569343] (req 64565248->64569343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.637585:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c593b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64565248->64569343] (req 64565248->64569343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.637594:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c593b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64565248->64569343] (req 64565248->64569343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.637780:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554783744:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7254us (7848us total) trans 4294983072 rc 0/0 00000100:00100000:1.0:1789760781.637793:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15774 00000100:00100000:1.0:1789760781.637806:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724861781126 time 724876122649 state 0x2 ref 0 00000100:00100000:1.0:1789760781.642906:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760781.643135:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554783872 00000100:00100000:3.0:1789760781.643147:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554783872 00000100:00100000:3.0:1789760781.643223:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724861781126 time 724876122649 state 0x0 ref 1 00000100:00100000:3.0:1789760781.643241:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724876383065 time 724876122649 state 0x1 ref 1 00000100:00100000:3.0:1789760781.643281:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724876383065 time 724888898933 state 0x0 ref 1 00000100:00100000:3.0:1789760781.643295:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554783872:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760781.643357:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64569344, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ecb8 00010000:00010000:3.0:1789760781.643369:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5942 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.644943:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760781.644950:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.644961:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5942 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64569344->64573439] (req 64569344->64573439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.644980:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5942 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64569344->64573439] (req 64569344->64573439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.651971:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983073 is committed 00002000:00010000:3.0:1789760781.652239:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64569344 -> 64573440 00002000:00010000:3.0:1789760781.652246:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126136 -> 126144 00010000:00010000:3.0:1789760781.652264:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5942 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64569344->64573439] (req 64569344->64573439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.652287:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5942 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64569344->64573439] (req 64569344->64573439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.652298:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5942 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64569344->64573439] (req 64569344->64573439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.652307:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760781.652313:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5942 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64569344->64573439] (req 64569344->64573439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.652322:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5942 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64569344->64573439] (req 64569344->64573439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.652347:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5942 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64569344->64573439] (req 64569344->64573439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.652355:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5942 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64569344->64573439] (req 64569344->64573439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.652557:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554783872:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9258us (9660us total) trans 4294983073 rc 0/0 00000100:00100000:3.0:1789760781.652570:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15775 00000100:00100000:3.0:1789760781.652581:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724876383065 time 724888898933 state 0x2 ref 0 00000100:00100000:0.0:1789760781.657259:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760781.657438:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554784000 00000100:00100000:2.0:1789760781.657450:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554784000 00000100:00100000:2.0:1789760781.657525:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724876383065 time 724888898933 state 0x0 ref 1 00000100:00100000:2.0:1789760781.657545:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724889159349 time 724888898933 state 0x1 ref 1 00000100:00100000:2.0:1789760781.657580:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724889159349 time 724903198597 state 0x0 ref 1 00000100:00100000:2.0:1789760781.657594:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554784000:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760781.657675:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64573440, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ecc0 00010000:00010000:2.0:1789760781.657698:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5949 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.657721:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789760781.657726:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.657732:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5949 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64573440->64577535] (req 64573440->64577535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.657747:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5949 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64573440->64577535] (req 64573440->64577535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.665839:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983074 is committed 00002000:00010000:2.0:1789760781.666256:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64573440 -> 64577536 00002000:00010000:2.0:1789760781.666262:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126144 -> 126152 00010000:00010000:2.0:1789760781.666279:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5949 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64573440->64577535] (req 64573440->64577535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.666296:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5949 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64573440->64577535] (req 64573440->64577535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.666307:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5949 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64573440->64577535] (req 64573440->64577535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.666315:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760781.666321:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5949 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64573440->64577535] (req 64573440->64577535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.666332:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5949 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64573440->64577535] (req 64573440->64577535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.666357:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5949 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64573440->64577535] (req 64573440->64577535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.666365:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5949 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64573440->64577535] (req 64573440->64577535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.667892:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554784000:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10282us (10629us total) trans 4294983074 rc 0/0 00000100:00100000:2.0:1789760781.667912:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15776 00000100:00100000:2.0:1789760781.667925:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724889159349 time 724903198597 state 0x2 ref 0 00000100:00100000:2.0:1789760781.672714:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760781.672817:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554784128 00000100:00100000:0.0:1789760781.672831:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554784128 00000100:00100000:0.0:1789760781.672904:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724889159349 time 724903198597 state 0x0 ref 1 00000100:00100000:0.0:1789760781.672924:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724903459013 time 724903198597 state 0x1 ref 1 00000100:00100000:0.0:1789760781.672959:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724903459013 time 724918577714 state 0x0 ref 1 00000100:00100000:0.0:1789760781.672974:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554784128:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760781.673040:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64577536, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ecc8 00010000:00010000:0.0:1789760781.673051:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5950 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.673078:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760781.673084:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.673092:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5950 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64577536->64581631] (req 64577536->64581631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.673110:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5950 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64577536->64581631] (req 64577536->64581631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.679418:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983075 is committed 00002000:00010000:0.0:1789760781.679646:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64577536 -> 64581632 00002000:00010000:0.0:1789760781.679652:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126152 -> 126160 00010000:00010000:0.0:1789760781.679669:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5950 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64577536->64581631] (req 64577536->64581631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.679696:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5950 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64577536->64581631] (req 64577536->64581631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.679714:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5950 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64577536->64581631] (req 64577536->64581631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.679788:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760781.679794:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5950 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64577536->64581631] (req 64577536->64581631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.679805:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5950 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64577536->64581631] (req 64577536->64581631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.679831:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5950 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64577536->64581631] (req 64577536->64581631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.679838:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5950 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64577536->64581631] (req 64577536->64581631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.680115:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554784128:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7136us (7407us total) trans 4294983075 rc 0/0 00000100:00100000:0.0:1789760781.680128:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15777 00000100:00100000:0.0:1789760781.680139:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724903459013 time 724918577714 state 0x2 ref 0 00000100:00100000:1.0:1789760781.684580:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760781.684683:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554784256 00000100:00100000:0.0:1789760781.684708:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554784256 00000100:00100000:0.0:1789760781.684958:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724903459013 time 724918577714 state 0x0 ref 1 00000100:00100000:0.0:1789760781.684977:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724918838130 time 724918577714 state 0x1 ref 1 00000100:00100000:0.0:1789760781.685017:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724918838130 time 724930634360 state 0x0 ref 1 00000100:00100000:0.0:1789760781.685032:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554784256:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760781.685105:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64581632, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ecd0 00010000:00010000:0.0:1789760781.685115:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5957 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.685133:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760781.685138:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.685146:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5957 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64581632->64585727] (req 64581632->64585727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.685162:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5957 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64581632->64585727] (req 64581632->64585727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.690955:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983076 is committed 00002000:00010000:0.0:1789760781.691121:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64581632 -> 64585728 00002000:00010000:0.0:1789760781.691126:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126160 -> 126168 00010000:00010000:0.0:1789760781.691141:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5957 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64581632->64585727] (req 64581632->64585727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.691154:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5957 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64581632->64585727] (req 64581632->64585727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.691165:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5957 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64581632->64585727] (req 64581632->64585727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.691179:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760781.691184:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5957 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64581632->64585727] (req 64581632->64585727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.691194:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5957 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64581632->64585727] (req 64581632->64585727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.691215:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5957 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64581632->64585727] (req 64581632->64585727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.691223:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5957 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64581632->64585727] (req 64581632->64585727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.691356:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554784256:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6323us (6786us total) trans 4294983076 rc 0/0 00000100:00100000:0.0:1789760781.691367:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15778 00000100:00100000:0.0:1789760781.691428:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724918838130 time 724930634360 state 0x2 ref 0 00000100:00100000:2.0:1789760781.700002:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760781.700113:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554784384 00000100:00100000:3.0:1789760781.700123:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554784384 00000100:00100000:3.0:1789760781.700181:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724918838130 time 724930634360 state 0x0 ref 1 00000100:00100000:3.0:1789760781.700197:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724930894776 time 724930634360 state 0x1 ref 1 00000100:00100000:3.0:1789760781.700276:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724930894776 time 724945893509 state 0x0 ref 1 00000100:00100000:3.0:1789760781.700289:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554784384:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760781.700337:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64585728, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ecd8 00010000:00010000:3.0:1789760781.700347:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c595e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.700367:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760781.700372:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.700406:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c595e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64585728->64589823] (req 64585728->64589823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.700420:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c595e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64585728->64589823] (req 64585728->64589823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760781.705372:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983077 is committed 00002000:00010000:3.0:1789760781.705746:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64585728 -> 64589824 00002000:00010000:3.0:1789760781.705752:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126168 -> 126176 00010000:00010000:3.0:1789760781.705766:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c595e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64585728->64589823] (req 64585728->64589823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.705778:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c595e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64585728->64589823] (req 64585728->64589823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.705792:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c595e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64585728->64589823] (req 64585728->64589823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.705799:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760781.705804:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c595e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64585728->64589823] (req 64585728->64589823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.705812:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c595e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64585728->64589823] (req 64585728->64589823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.705831:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c595e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64585728->64589823] (req 64585728->64589823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.705838:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c595e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64585728->64589823] (req 64585728->64589823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.705986:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554784384:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5694us (5986us total) trans 4294983077 rc 0/0 00000100:00100000:3.0:1789760781.705996:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15779 00000100:00100000:3.0:1789760781.706005:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724930894776 time 724945893509 state 0x2 ref 0 00000100:00100000:2.0:1789760781.715400:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760781.715598:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554784512 00000100:00100000:1.0:1789760781.715607:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554784512 00000100:00100000:1.0:1789760781.715659:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724930894776 time 724945893509 state 0x0 ref 1 00000100:00100000:1.0:1789760781.715675:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724946153925 time 724945893509 state 0x1 ref 1 00000100:00100000:1.0:1789760781.715754:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724946153925 time 724961371926 state 0x0 ref 1 00000100:00100000:1.0:1789760781.715767:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554784512:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760781.715834:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64589824, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ece0 00010000:00010000:1.0:1789760781.715843:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5965 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.715858:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760781.715862:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.715868:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5965 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64589824->64593919] (req 64589824->64593919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.715881:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5965 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64589824->64593919] (req 64589824->64593919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.722183:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983078 is committed 00002000:00010000:2.0:1789760781.722532:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64589824 -> 64593920 00002000:00010000:2.0:1789760781.722538:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126176 -> 126184 00010000:00010000:2.0:1789760781.722550:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5965 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64589824->64593919] (req 64589824->64593919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.722564:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5965 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64589824->64593919] (req 64589824->64593919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.722575:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5965 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64589824->64593919] (req 64589824->64593919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.722583:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760781.722588:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5965 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64589824->64593919] (req 64589824->64593919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.722597:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5965 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64589824->64593919] (req 64589824->64593919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.722624:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5965 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64589824->64593919] (req 64589824->64593919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.722632:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5965 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64589824->64593919] (req 64589824->64593919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.722765:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554784512:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6995us (7386us total) trans 4294983078 rc 0/0 00000100:00100000:2.0:1789760781.722775:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15780 00000100:00100000:2.0:1789760781.722793:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724946153925 time 724961371926 state 0x2 ref 0 00000100:00100000:1.0:1789760781.729198:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760781.729316:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554784640 00000100:00100000:0.0:1789760781.729325:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554784640 00000100:00100000:0.0:1789760781.729374:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724946153925 time 724961371926 state 0x0 ref 1 00000100:00100000:0.0:1789760781.729499:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724961632342 time 724961371926 state 0x1 ref 1 00000100:00100000:0.0:1789760781.729570:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724961632342 time 724975188065 state 0x0 ref 1 00000100:00100000:0.0:1789760781.729584:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554784640:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760781.729630:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64593920, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ece8 00010000:00010000:0.0:1789760781.729638:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c596c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.729653:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:0.0:1789760781.729658:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.729664:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c596c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64593920->64598015] (req 64593920->64598015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.729678:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c596c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64593920->64598015] (req 64593920->64598015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.736142:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983079 is committed 00002000:00010000:0.0:1789760781.736260:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64593920 -> 64598016 00002000:00010000:0.0:1789760781.736265:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126184 -> 126192 00010000:00010000:0.0:1789760781.736280:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c596c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64593920->64598015] (req 64593920->64598015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.736291:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c596c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64593920->64598015] (req 64593920->64598015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.736300:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c596c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64593920->64598015] (req 64593920->64598015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.736308:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760781.736313:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c596c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64593920->64598015] (req 64593920->64598015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.736322:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c596c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64593920->64598015] (req 64593920->64598015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.736340:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c596c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64593920->64598015] (req 64593920->64598015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.736349:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c596c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64593920->64598015] (req 64593920->64598015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.736587:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554784640:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7000us (7394us total) trans 4294983079 rc 0/0 00000100:00100000:0.0:1789760781.736598:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15781 00000100:00100000:0.0:1789760781.736607:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724961632342 time 724975188065 state 0x2 ref 0 00000100:00100000:0.0:1789760781.740028:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760781.740163:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554784768 00000100:00100000:1.0:1789760781.740173:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554784768 00000100:00100000:1.0:1789760781.740221:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724961632342 time 724975188065 state 0x0 ref 1 00000100:00100000:1.0:1789760781.740236:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724975448481 time 724975188065 state 0x1 ref 1 00000100:00100000:1.0:1789760781.740330:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724975448481 time 724985947838 state 0x0 ref 1 00000100:00100000:1.0:1789760781.740343:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554784768:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760781.740558:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64598016, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ecf0 00010000:00010000:1.0:1789760781.740567:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5973 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.740582:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:1.0:1789760781.740587:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.740592:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5973 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64598016->64602111] (req 64598016->64602111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.740605:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5973 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64598016->64602111] (req 64598016->64602111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.746698:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983080 is committed 00002000:00010000:1.0:1789760781.750218:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64598016 -> 64602112 00002000:00010000:1.0:1789760781.750228:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126192 -> 126200 00010000:00010000:1.0:1789760781.750241:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5973 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64598016->64602111] (req 64598016->64602111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.750259:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5973 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64598016->64602111] (req 64598016->64602111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.750267:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5973 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64598016->64602111] (req 64598016->64602111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.750273:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760781.750278:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5973 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64598016->64602111] (req 64598016->64602111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.750287:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5973 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64598016->64602111] (req 64598016->64602111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.750307:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5973 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64598016->64602111] (req 64598016->64602111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.750315:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5973 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64598016->64602111] (req 64598016->64602111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.750431:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554784768:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10085us (10408us total) trans 4294983080 rc 0/0 00000100:00100000:1.0:1789760781.750443:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15782 00000100:00100000:1.0:1789760781.750453:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724975448481 time 724985947838 state 0x2 ref 0 00000100:00100000:2.0:1789760781.754770:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760781.754957:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554784896 00000100:00100000:3.0:1789760781.754966:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554784896 00000100:00100000:3.0:1789760781.755017:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724975448481 time 724985947838 state 0x0 ref 1 00000100:00100000:3.0:1789760781.755028:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724986208254 time 724985947838 state 0x1 ref 1 00000100:00100000:3.0:1789760781.755129:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724986208254 time 725000747215 state 0x0 ref 1 00000100:00100000:3.0:1789760781.755142:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554784896:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760781.755192:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64602112, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ecf8 00010000:00010000:3.0:1789760781.755201:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c597a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.755215:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789760781.755219:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.755225:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c597a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64602112->64606207] (req 64602112->64606207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.755238:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c597a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64602112->64606207] (req 64602112->64606207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.760226:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983081 is committed 00002000:00010000:3.0:1789760781.760693:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64602112 -> 64606208 00002000:00010000:3.0:1789760781.760699:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126200 -> 126208 00010000:00010000:3.0:1789760781.760711:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c597a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64602112->64606207] (req 64602112->64606207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.760724:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c597a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64602112->64606207] (req 64602112->64606207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.760732:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c597a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64602112->64606207] (req 64602112->64606207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.760744:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760781.760749:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c597a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64602112->64606207] (req 64602112->64606207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.760759:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c597a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64602112->64606207] (req 64602112->64606207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.760848:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c597a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64602112->64606207] (req 64602112->64606207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.760857:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c597a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64602112->64606207] (req 64602112->64606207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.760948:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554784896:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5805us (6183us total) trans 4294983081 rc 0/0 00000100:00100000:3.0:1789760781.760959:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15783 00000100:00100000:3.0:1789760781.760968:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724986208254 time 725000747215 state 0x2 ref 0 00000100:00100000:2.0:1789760781.765672:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760781.765846:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554785024 00000100:00100000:1.0:1789760781.765856:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554785024 00000100:00100000:1.0:1789760781.765908:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 724986208254 time 725000747215 state 0x0 ref 1 00000100:00100000:1.0:1789760781.765923:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725001007631 time 725000747215 state 0x1 ref 1 00000100:00100000:1.0:1789760781.765986:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725001007631 time 725011604629 state 0x0 ref 1 00000100:00100000:1.0:1789760781.765999:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554785024:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760781.766040:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64606208, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ed00 00010000:00010000:1.0:1789760781.766047:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5981 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.766060:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760781.766063:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.766067:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5981 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64606208->64610303] (req 64606208->64610303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.766076:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5981 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64606208->64610303] (req 64606208->64610303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.771302:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983082 is committed 00002000:00010000:1.0:1789760781.771560:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64606208 -> 64610304 00002000:00010000:1.0:1789760781.771566:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126208 -> 126216 00010000:00010000:1.0:1789760781.771579:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5981 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64606208->64610303] (req 64606208->64610303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.771591:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5981 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64606208->64610303] (req 64606208->64610303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.771605:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5981 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64606208->64610303] (req 64606208->64610303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.771612:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760781.771618:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5981 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64606208->64610303] (req 64606208->64610303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.771627:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5981 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64606208->64610303] (req 64606208->64610303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.771646:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5981 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64606208->64610303] (req 64606208->64610303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.771653:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5981 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64606208->64610303] (req 64606208->64610303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.771799:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554785024:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5789us (6122us total) trans 4294983082 rc 0/0 00000100:00100000:1.0:1789760781.771810:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15784 00000100:00100000:1.0:1789760781.771819:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725001007631 time 725011604629 state 0x2 ref 0 00000100:00100000:0.0:1789760781.775875:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760781.776032:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554785152 00000100:00100000:2.0:1789760781.776041:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554785152 00000100:00100000:2.0:1789760781.776095:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725001007631 time 725011604629 state 0x0 ref 1 00000100:00100000:2.0:1789760781.776109:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725011865045 time 725011604629 state 0x1 ref 1 00000100:00100000:2.0:1789760781.776134:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725011865045 time 725021752377 state 0x0 ref 1 00000100:00100000:2.0:1789760781.776145:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554785152:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760781.776193:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64610304, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ed08 00010000:00010000:2.0:1789760781.776212:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5988 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.776226:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760781.776230:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.776236:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5988 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64610304->64614399] (req 64610304->64614399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.776250:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5988 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64610304->64614399] (req 64610304->64614399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.781707:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983083 is committed 00002000:00010000:0.0:1789760781.782135:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64610304 -> 64614400 00002000:00010000:0.0:1789760781.782203:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126216 -> 126224 00010000:00010000:0.0:1789760781.782217:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5988 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64610304->64614399] (req 64610304->64614399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.782229:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5988 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64610304->64614399] (req 64610304->64614399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.782243:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5988 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64610304->64614399] (req 64610304->64614399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.782359:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760781.782364:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5988 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64610304->64614399] (req 64610304->64614399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.782372:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5988 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64610304->64614399] (req 64610304->64614399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.782676:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5988 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64610304->64614399] (req 64610304->64614399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.782698:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5988 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64610304->64614399] (req 64610304->64614399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.783056:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554785152:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6905us (7183us total) trans 4294983083 rc 0/0 00000100:00100000:0.0:1789760781.783077:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15785 00000100:00100000:0.0:1789760781.783087:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725011865045 time 725021752377 state 0x2 ref 0 00000100:00100000:2.0:1789760781.792975:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760781.793322:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554785280 00000100:00100000:0.0:1789760781.793342:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554785280 00000100:00100000:0.0:1789760781.793432:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725011865045 time 725021752377 state 0x0 ref 1 00000100:00100000:0.0:1789760781.793447:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725022012793 time 725021752377 state 0x1 ref 1 00000100:00100000:0.0:1789760781.793468:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725022012793 time 725039086871 state 0x0 ref 1 00000100:00100000:0.0:1789760781.793480:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554785280:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760781.793524:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64614400, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ed10 00010000:00010000:0.0:1789760781.793535:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c598f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.793549:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760781.793553:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.793560:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c598f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64614400->64618495] (req 64614400->64618495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.793578:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c598f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64614400->64618495] (req 64614400->64618495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.800913:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983084 is committed 00002000:00010000:0.0:1789760781.801012:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64614400 -> 64618496 00002000:00010000:0.0:1789760781.801018:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126224 -> 126232 00010000:00010000:0.0:1789760781.801031:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c598f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64614400->64618495] (req 64614400->64618495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.801092:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c598f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64614400->64618495] (req 64614400->64618495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.801102:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c598f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64614400->64618495] (req 64614400->64618495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.801116:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760781.801120:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c598f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64614400->64618495] (req 64614400->64618495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.801130:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c598f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64614400->64618495] (req 64614400->64618495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.801315:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c598f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64614400->64618495] (req 64614400->64618495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.801325:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c598f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64614400->64618495] (req 64614400->64618495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.801531:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554785280:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8048us (8559us total) trans 4294983084 rc 0/0 00000100:00100000:0.0:1789760781.801542:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15786 00000100:00100000:0.0:1789760781.801552:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725022012793 time 725039086871 state 0x2 ref 0 00000100:00100000:2.0:1789760781.804930:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760781.805406:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554785408 00000100:00100000:3.0:1789760781.805416:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554785408 00000100:00100000:3.0:1789760781.805516:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725022012793 time 725039086871 state 0x0 ref 1 00000100:00100000:3.0:1789760781.805531:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725039347287 time 725039086871 state 0x1 ref 1 00000100:00100000:3.0:1789760781.805599:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725039347287 time 725051216521 state 0x0 ref 1 00000100:00100000:3.0:1789760781.805613:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554785408:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760781.805656:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64618496, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ed18 00010000:00010000:3.0:1789760781.805665:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5996 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.805682:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760781.805696:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.805702:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5996 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64618496->64622591] (req 64618496->64622591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.805715:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5996 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64618496->64622591] (req 64618496->64622591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.810122:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983085 is committed 00002000:00010000:3.0:1789760781.810281:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64618496 -> 64622592 00002000:00010000:3.0:1789760781.810286:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126232 -> 126240 00010000:00010000:3.0:1789760781.810301:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5996 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64618496->64622591] (req 64618496->64622591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.810313:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5996 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64618496->64622591] (req 64618496->64622591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.810321:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5996 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64618496->64622591] (req 64618496->64622591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.810327:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760781.810332:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5996 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64618496->64622591] (req 64618496->64622591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.810340:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5996 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64618496->64622591] (req 64618496->64622591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.810358:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5996 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64618496->64622591] (req 64618496->64622591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.810366:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5996 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64618496->64622591] (req 64618496->64622591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.810491:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554785408:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4875us (5564us total) trans 4294983085 rc 0/0 00000100:00100000:3.0:1789760781.810503:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15787 00000100:00100000:3.0:1789760781.810512:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725039347287 time 725051216521 state 0x2 ref 0 00000100:00100000:2.0:1789760781.814321:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760781.814600:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554785536 00000100:00100000:1.0:1789760781.814610:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554785536 00000100:00100000:1.0:1789760781.814669:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725039347287 time 725051216521 state 0x0 ref 1 00000100:00100000:1.0:1789760781.814693:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725051476937 time 725051216521 state 0x1 ref 1 00000100:00100000:1.0:1789760781.814714:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725051476937 time 725060333332 state 0x0 ref 1 00000100:00100000:1.0:1789760781.814727:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554785536:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760781.814776:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64622592, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ed20 00010000:00010000:1.0:1789760781.814785:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c599d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.814801:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760781.814805:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.814811:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c599d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64622592->64626687] (req 64622592->64626687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.814823:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c599d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64622592->64626687] (req 64622592->64626687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.820213:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983086 is committed 00002000:00010000:1.0:1789760781.820332:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64622592 -> 64626688 00002000:00010000:1.0:1789760781.820338:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126240 -> 126248 00010000:00010000:1.0:1789760781.820351:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c599d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64622592->64626687] (req 64622592->64626687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.820364:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c599d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64622592->64626687] (req 64622592->64626687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.820372:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c599d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64622592->64626687] (req 64622592->64626687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.820421:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760781.820427:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c599d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64622592->64626687] (req 64622592->64626687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.820436:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c599d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64622592->64626687] (req 64622592->64626687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.820465:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c599d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64622592->64626687] (req 64622592->64626687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.820473:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c599d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64622592->64626687] (req 64622592->64626687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.820560:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554785536:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5832us (6245us total) trans 4294983086 rc 0/0 00000100:00100000:1.0:1789760781.820573:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15788 00000100:00100000:1.0:1789760781.820582:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725051476937 time 725060333332 state 0x2 ref 0 00000100:00100000:2.0:1789760781.823759:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760781.823905:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554785664 00000100:00100000:0.0:1789760781.823914:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554785664 00000100:00100000:0.0:1789760781.824034:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725051476937 time 725060333332 state 0x0 ref 1 00000100:00100000:0.0:1789760781.824049:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725060593748 time 725060333332 state 0x1 ref 1 00000100:00100000:0.0:1789760781.824079:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725060593748 time 725069697807 state 0x0 ref 1 00000100:00100000:0.0:1789760781.824103:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554785664:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760781.824148:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64626688, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ed28 00010000:00010000:0.0:1789760781.824158:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c59a4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.824173:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760781.824178:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.824186:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c59a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64626688->64630783] (req 64626688->64630783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.824198:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c59a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64626688->64630783] (req 64626688->64630783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.828369:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983087 is committed 00002000:00010000:0.0:1789760781.828505:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64626688 -> 64630784 00002000:00010000:0.0:1789760781.828510:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126248 -> 126256 00010000:00010000:0.0:1789760781.828523:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c59a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64626688->64630783] (req 64626688->64630783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.828536:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c59a4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64626688->64630783] (req 64626688->64630783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.828545:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c59a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64626688->64630783] (req 64626688->64630783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.828553:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760781.828559:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c59a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64626688->64630783] (req 64626688->64630783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.828569:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c59a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64626688->64630783] (req 64626688->64630783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.828589:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c59a4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64626688->64630783] (req 64626688->64630783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.828596:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c59a4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64626688->64630783] (req 64626688->64630783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.828677:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554785664:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4572us (4921us total) trans 4294983087 rc 0/0 00000100:00100000:0.0:1789760781.828697:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15789 00000100:00100000:0.0:1789760781.828706:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725060593748 time 725069697807 state 0x2 ref 0 00000100:00100000:2.0:1789760781.833613:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760781.833674:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554785792 00000100:00100000:0.0:1789760781.833744:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554785792 00000100:00100000:0.0:1789760781.833918:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725060593748 time 725069697807 state 0x0 ref 1 00000100:00100000:0.0:1789760781.833932:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725069958223 time 725069697807 state 0x1 ref 1 00000100:00100000:0.0:1789760781.833995:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725069958223 time 725079612371 state 0x0 ref 1 00000100:00100000:0.0:1789760781.834008:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554785792:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760781.834086:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64630784, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ed30 00010000:00010000:0.0:1789760781.834097:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c59ab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.834111:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760781.834116:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.834123:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c59ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64630784->64634879] (req 64630784->64634879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.834137:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c59ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64630784->64634879] (req 64630784->64634879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.842723:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983088 is committed 00002000:00010000:1.0:1789760781.842831:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64630784 -> 64634880 00002000:00010000:1.0:1789760781.842836:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126256 -> 126264 00010000:00010000:1.0:1789760781.842849:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c59ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64630784->64634879] (req 64630784->64634879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.842861:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c59ab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64630784->64634879] (req 64630784->64634879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.842876:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c59ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64630784->64634879] (req 64630784->64634879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.842883:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760781.842888:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c59ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64630784->64634879] (req 64630784->64634879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.842898:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c59ab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64630784->64634879] (req 64630784->64634879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.842921:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c59ab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64630784->64634879] (req 64630784->64634879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.842929:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c59ab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64630784->64634879] (req 64630784->64634879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.843104:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554785792:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9094us (9495us total) trans 4294983088 rc 0/0 00000100:00100000:1.0:1789760781.843115:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15790 00000100:00100000:1.0:1789760781.843123:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725069958223 time 725079612371 state 0x2 ref 0 00000100:00100000:2.0:1789760781.850424:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760781.851780:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554785920 00000100:00100000:3.0:1789760781.851793:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554785920 00000100:00100000:3.0:1789760781.851864:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725069958223 time 725079612371 state 0x0 ref 1 00000100:00100000:3.0:1789760781.851880:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725079872787 time 725079612371 state 0x1 ref 1 00000100:00100000:3.0:1789760781.851923:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725079872787 time 725097541508 state 0x0 ref 1 00000100:00100000:3.0:1789760781.851937:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554785920:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760781.851987:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64634880, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ed38 00010000:00010000:3.0:1789760781.851997:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c59b2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.852014:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760781.852018:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.852025:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c59b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64634880->64638975] (req 64634880->64638975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.852040:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c59b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64634880->64638975] (req 64634880->64638975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.858418:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983089 is committed 00002000:00010000:3.0:1789760781.859579:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64634880 -> 64638976 00002000:00010000:3.0:1789760781.859586:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126264 -> 126272 00010000:00010000:3.0:1789760781.859600:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c59b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64634880->64638975] (req 64634880->64638975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.859613:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c59b2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64634880->64638975] (req 64634880->64638975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.859622:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c59b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64634880->64638975] (req 64634880->64638975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.859630:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760781.859634:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c59b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64634880->64638975] (req 64634880->64638975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.859649:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c59b2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64634880->64638975] (req 64634880->64638975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.859669:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c59b2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64634880->64638975] (req 64634880->64638975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.859677:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c59b2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64634880->64638975] (req 64634880->64638975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760781.861213:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554785920:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9267us (10789us total) trans 4294983089 rc 0/0 00000100:00100000:3.0:1789760781.861228:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15791 00000100:00100000:3.0:1789760781.861238:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725079872787 time 725097541508 state 0x2 ref 0 00000100:00100000:2.0:1789760781.865946:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760781.866038:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554786048 00000100:00100000:1.0:1789760781.866048:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554786048 00000100:00100000:1.0:1789760781.866107:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725079872787 time 725097541508 state 0x0 ref 1 00000100:00100000:1.0:1789760781.866126:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725097801924 time 725097541508 state 0x1 ref 1 00000100:00100000:1.0:1789760781.866222:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725097801924 time 725111839184 state 0x0 ref 1 00000100:00100000:1.0:1789760781.866236:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554786048:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760781.866284:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64638976, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ed40 00010000:00010000:1.0:1789760781.866293:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59b9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.866310:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760781.866315:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.866321:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64638976->64643071] (req 64638976->64643071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.866336:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64638976->64643071] (req 64638976->64643071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760781.871013:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983090 is committed 00002000:00010000:1.0:1789760781.871175:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64638976 -> 64643072 00002000:00010000:1.0:1789760781.871180:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126272 -> 126280 00010000:00010000:1.0:1789760781.871194:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64638976->64643071] (req 64638976->64643071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.871206:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64638976->64643071] (req 64638976->64643071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.871216:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64638976->64643071] (req 64638976->64643071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.871225:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760781.871230:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64638976->64643071] (req 64638976->64643071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.871239:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59b9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64638976->64643071] (req 64638976->64643071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.871259:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59b9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64638976->64643071] (req 64638976->64643071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.871266:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59b9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64638976->64643071] (req 64638976->64643071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.872375:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554786048:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6132us (6500us total) trans 4294983090 rc 0/0 00000100:00100000:1.0:1789760781.872406:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15792 00000100:00100000:1.0:1789760781.872415:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725097801924 time 725111839184 state 0x2 ref 0 00000100:00100000:2.0:1789760781.875373:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760781.875547:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554786176 00000100:00100000:0.0:1789760781.875557:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554786176 00000100:00100000:0.0:1789760781.875611:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725097801924 time 725111839184 state 0x0 ref 1 00000100:00100000:0.0:1789760781.875627:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725112099600 time 725111839184 state 0x1 ref 1 00000100:00100000:0.0:1789760781.875659:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725112099600 time 725121277458 state 0x0 ref 1 00000100:00100000:0.0:1789760781.875673:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554786176:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760781.875730:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64643072, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ed48 00010000:00010000:0.0:1789760781.875739:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c59c0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.875756:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760781.875761:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760781.875778:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c59c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64643072->64647167] (req 64643072->64647167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.875794:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c59c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64643072->64647167] (req 64643072->64647167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.883617:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983091 is committed 00002000:00010000:0.0:1789760781.883740:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64643072 -> 64647168 00002000:00010000:0.0:1789760781.883745:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126280 -> 126288 00010000:00010000:0.0:1789760781.883758:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c59c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64643072->64647167] (req 64643072->64647167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.883781:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c59c0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64643072->64647167] (req 64643072->64647167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.883791:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c59c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64643072->64647167] (req 64643072->64647167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.883798:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760781.883802:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c59c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64643072->64647167] (req 64643072->64647167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.883812:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c59c0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64643072->64647167] (req 64643072->64647167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.883831:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c59c0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64643072->64647167] (req 64643072->64647167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.883840:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c59c0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64643072->64647167] (req 64643072->64647167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.883928:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554786176:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8253us (8558us total) trans 4294983091 rc 0/0 00000100:00100000:0.0:1789760781.883939:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15793 00000100:00100000:0.0:1789760781.883948:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725112099600 time 725121277458 state 0x2 ref 0 00000100:00100000:2.0:1789760781.891269:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760781.891715:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554786304 00000100:00100000:1.0:1789760781.891726:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554786304 00000100:00100000:1.0:1789760781.891780:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725112099600 time 725121277458 state 0x0 ref 1 00000100:00100000:1.0:1789760781.891795:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725121537874 time 725121277458 state 0x1 ref 1 00000100:00100000:1.0:1789760781.891924:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725121537874 time 725137541285 state 0x0 ref 1 00000100:00100000:1.0:1789760781.891940:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554786304:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760781.891987:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64647168, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ed50 00010000:00010000:1.0:1789760781.891996:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59c7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.892011:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760781.892015:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.892021:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64647168->64651263] (req 64647168->64651263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.892035:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64647168->64651263] (req 64647168->64651263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.899007:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983092 is committed 00002000:00010000:1.0:1789760781.901650:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64647168 -> 64651264 00002000:00010000:1.0:1789760781.901660:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126288 -> 126296 00010000:00010000:1.0:1789760781.901674:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64647168->64651263] (req 64647168->64651263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.901889:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64647168->64651263] (req 64647168->64651263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.901898:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64647168->64651263] (req 64647168->64651263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.901904:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760781.901910:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64647168->64651263] (req 64647168->64651263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.901919:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64647168->64651263] (req 64647168->64651263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.901939:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59c7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64647168->64651263] (req 64647168->64651263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.901947:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59c7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64647168->64651263] (req 64647168->64651263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760781.902051:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554786304:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10109us (10789us total) trans 4294983092 rc 0/0 00000100:00100000:1.0:1789760781.902069:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15794 00000100:00100000:1.0:1789760781.902080:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725121537874 time 725137541285 state 0x2 ref 0 00010000:00010000:0.0:1789760781.905576:0:6509:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789760781.908023:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760781.908242:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554786432 00000100:00100000:3.0:1789760781.908252:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554786432 00000100:00100000:3.0:1789760781.908308:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725121537874 time 725137541285 state 0x0 ref 1 00000100:00100000:3.0:1789760781.908325:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725137801701 time 725137541285 state 0x1 ref 1 00000100:00100000:3.0:1789760781.908358:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725137801701 time 725153976796 state 0x0 ref 1 00000100:00100000:3.0:1789760781.908370:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554786432:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760781.908594:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64651264, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ed58 00010000:00010000:3.0:1789760781.908603:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c59ce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.908619:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760781.908624:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760781.908630:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c59ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64651264->64655359] (req 64651264->64655359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760781.908642:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c59ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64651264->64655359] (req 64651264->64655359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.919497:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983093 is committed 00002000:00010000:2.0:1789760781.919731:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64651264 -> 64655360 00002000:00010000:2.0:1789760781.919737:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126296 -> 126304 00010000:00010000:2.0:1789760781.919750:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c59ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64651264->64655359] (req 64651264->64655359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.919763:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c59ce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64651264->64655359] (req 64651264->64655359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.919771:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c59ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64651264->64655359] (req 64651264->64655359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.919778:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760781.919782:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c59ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64651264->64655359] (req 64651264->64655359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.919791:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c59ce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64651264->64655359] (req 64651264->64655359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.919811:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c59ce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64651264->64655359] (req 64651264->64655359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.919818:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c59ce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64651264->64655359] (req 64651264->64655359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.919959:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554786432:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11585us (11939us total) trans 4294983093 rc 0/0 00000100:00100000:2.0:1789760781.919970:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15795 00000100:00100000:2.0:1789760781.919979:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725137801701 time 725153976796 state 0x2 ref 0 00000100:00100000:2.0:1789760781.925177:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760781.925255:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554786560 00000100:00100000:1.0:1789760781.925266:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554786560 00000100:00100000:1.0:1789760781.925319:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725137801701 time 725153976796 state 0x0 ref 1 00000100:00100000:1.0:1789760781.925336:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725154237212 time 725153976796 state 0x1 ref 1 00000100:00100000:1.0:1789760781.925371:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725154237212 time 725170989136 state 0x0 ref 1 00000100:00100000:1.0:1789760781.925415:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f84000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554786560:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760781.925469:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64655360, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ed60 00010000:00010000:1.0:1789760781.925478:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59d5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.925495:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760781.925500:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760781.925506:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64655360->64659455] (req 64655360->64659455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760781.925520:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64655360->64659455] (req 64655360->64659455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.930700:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983094 is committed 00002000:00010000:0.0:1789760781.933676:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64655360 -> 64659456 00002000:00010000:0.0:1789760781.933695:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126304 -> 126312 00010000:00010000:0.0:1789760781.933709:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64655360->64659455] (req 64655360->64659455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.933722:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59d5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64655360->64659455] (req 64655360->64659455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.933732:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64655360->64659455] (req 64655360->64659455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.933804:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760781.933811:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64655360->64659455] (req 64655360->64659455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.933820:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59d5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64655360->64659455] (req 64655360->64659455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.933977:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59d5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64655360->64659455] (req 64655360->64659455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760781.933988:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c59d5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64655360->64659455] (req 64655360->64659455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760781.934242:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f84000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554786560:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8826us (9069us total) trans 4294983094 rc 0/0 00000100:00100000:0.0:1789760781.934254:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15796 00000100:00100000:0.0:1789760781.934264:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725154237212 time 725170989136 state 0x2 ref 0 00000100:00100000:0.0:1789760781.941607:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760781.942331:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554786688 00000100:00100000:2.0:1789760781.942342:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554786688 00000100:00100000:2.0:1789760781.942436:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725154237212 time 725170989136 state 0x0 ref 1 00000100:00100000:2.0:1789760781.942526:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725171249552 time 725170989136 state 0x1 ref 1 00000100:00100000:2.0:1789760781.942617:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725171249552 time 725188233586 state 0x0 ref 1 00000100:00100000:2.0:1789760781.942631:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554786688:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760781.942694:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64659456, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ed68 00010000:00010000:2.0:1789760781.942703:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59dc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.942718:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760781.942723:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.942729:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64659456->64663551] (req 64659456->64663551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.942742:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64659456->64663551] (req 64659456->64663551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.949271:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983095 is committed 00002000:00010000:2.0:1789760781.949464:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64659456 -> 64663552 00002000:00010000:2.0:1789760781.949469:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126312 -> 126320 00010000:00010000:2.0:1789760781.949483:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64659456->64663551] (req 64659456->64663551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.949500:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59dc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64659456->64663551] (req 64659456->64663551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.949508:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64659456->64663551] (req 64659456->64663551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.949515:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760781.949519:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64659456->64663551] (req 64659456->64663551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.949527:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64659456->64663551] (req 64659456->64663551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.949546:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59dc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64659456->64663551] (req 64659456->64663551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.949552:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59dc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64659456->64663551] (req 64659456->64663551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.949655:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554786688:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7021us (8053us total) trans 4294983095 rc 0/0 00000100:00100000:2.0:1789760781.949666:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15797 00000100:00100000:2.0:1789760781.949693:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725171249552 time 725188233586 state 0x2 ref 0 00000100:00100000:1.0:1789760781.954969:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760781.955050:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554786816 00000100:00100000:2.0:1789760781.955060:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554786816 00000100:00100000:2.0:1789760781.955117:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725171249552 time 725188233586 state 0x0 ref 1 00000100:00100000:2.0:1789760781.955132:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725188494002 time 725188233586 state 0x1 ref 1 00000100:00100000:2.0:1789760781.955165:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725188494002 time 725200783018 state 0x0 ref 1 00000100:00100000:2.0:1789760781.955177:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe19450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554786816:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760781.955223:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64663552, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ed70 00010000:00010000:2.0:1789760781.955233:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c59e3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.955250:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760781.955256:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.955263:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c59e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64663552->64667647] (req 64663552->64667647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.955281:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c59e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64663552->64667647] (req 64663552->64667647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760781.962584:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983096 is committed 00002000:00010000:2.0:1789760781.962723:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64663552 -> 64667648 00002000:00010000:2.0:1789760781.962727:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126320 -> 126328 00010000:00010000:2.0:1789760781.962740:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c59e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64663552->64667647] (req 64663552->64667647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.962752:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c59e3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64663552->64667647] (req 64663552->64667647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.962766:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c59e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64663552->64667647] (req 64663552->64667647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.962774:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760781.962778:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c59e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64663552->64667647] (req 64663552->64667647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.962785:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c59e3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64663552->64667647] (req 64663552->64667647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.962803:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c59e3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64663552->64667647] (req 64663552->64667647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.962810:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c59e3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64663552->64667647] (req 64663552->64667647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.962902:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe19450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554786816:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7722us (7938us total) trans 4294983096 rc 0/0 00000100:00100000:2.0:1789760781.962913:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15798 00000100:00100000:2.0:1789760781.962923:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725188494002 time 725200783018 state 0x2 ref 0 00000100:00100000:1.0:1789760781.967917:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760781.968193:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554786944 00000100:00100000:2.0:1789760781.968203:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554786944 00000100:00100000:2.0:1789760781.968256:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725188494002 time 725200783018 state 0x0 ref 1 00000100:00100000:2.0:1789760781.968271:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725201043434 time 725200783018 state 0x1 ref 1 00000100:00100000:2.0:1789760781.968294:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725201043434 time 725213913048 state 0x0 ref 1 00000100:00100000:2.0:1789760781.968307:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554786944:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760781.968354:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64667648, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ed78 00010000:00010000:2.0:1789760781.968362:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59ea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.968404:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760781.968409:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.968416:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64667648->64671743] (req 64667648->64671743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.968430:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64667648->64671743] (req 64667648->64671743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.975987:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983097 is committed 00002000:00010000:2.0:1789760781.979720:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64667648 -> 64671744 00002000:00010000:2.0:1789760781.979730:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126328 -> 126336 00010000:00010000:2.0:1789760781.979744:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64667648->64671743] (req 64667648->64671743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.979757:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64667648->64671743] (req 64667648->64671743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.979765:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64667648->64671743] (req 64667648->64671743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.979777:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760781.979782:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64667648->64671743] (req 64667648->64671743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.979791:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59ea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64667648->64671743] (req 64667648->64671743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.979811:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59ea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64667648->64671743] (req 64667648->64671743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.979819:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59ea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64667648->64671743] (req 64667648->64671743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.980104:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554786944:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11721us (12120us total) trans 4294983097 rc 0/0 00000100:00100000:2.0:1789760781.980117:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15799 00000100:00100000:2.0:1789760781.980126:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725201043434 time 725213913048 state 0x2 ref 0 00000100:00100000:0.0:1789760781.985845:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760781.986028:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554787072 00000100:00100000:2.0:1789760781.986047:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554787072 00000100:00100000:2.0:1789760781.986103:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725201043434 time 725213913048 state 0x0 ref 1 00000100:00100000:2.0:1789760781.986118:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725214173464 time 725213913048 state 0x1 ref 1 00000100:00100000:2.0:1789760781.986151:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725214173464 time 725231769744 state 0x0 ref 1 00000100:00100000:2.0:1789760781.986168:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554787072:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760781.986215:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64671744, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ed80 00010000:00010000:2.0:1789760781.986223:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59f1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.986237:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760781.986241:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760781.986247:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64671744->64675839] (req 64671744->64675839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.986260:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64671744->64675839] (req 64671744->64675839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760781.991483:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983098 is committed 00002000:00010000:2.0:1789760781.992947:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64671744 -> 64675840 00002000:00010000:2.0:1789760781.992954:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126336 -> 126344 00010000:00010000:2.0:1789760781.992968:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64671744->64675839] (req 64671744->64675839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.992981:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59f1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64671744->64675839] (req 64671744->64675839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.992990:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64671744->64675839] (req 64671744->64675839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.992998:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760781.993004:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64671744->64675839] (req 64671744->64675839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.993013:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64671744->64675839] (req 64671744->64675839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.993096:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59f1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64671744->64675839] (req 64671744->64675839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760781.993106:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c59f1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64671744->64675839] (req 64671744->64675839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760781.993276:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554787072:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7105us (7431us total) trans 4294983098 rc 0/0 00000100:00100000:2.0:1789760781.993289:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15800 00000100:00100000:2.0:1789760781.993300:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725214173464 time 725231769744 state 0x2 ref 0 00000100:00100000:1.0:1789760781.999639:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760781.999812:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554787200 00000100:00100000:0.0:1789760781.999822:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554787200 00000100:00100000:0.0:1789760781.999874:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725214173464 time 725231769744 state 0x0 ref 1 00000100:00100000:0.0:1789760781.999888:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725232030160 time 725231769744 state 0x1 ref 1 00000100:00100000:0.0:1789760781.999954:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725232030160 time 725245572360 state 0x0 ref 1 00000100:00100000:0.0:1789760781.999967:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554787200:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760782.000017:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64675840, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ed88 00010000:00010000:0.0:1789760782.000026:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c59f8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.000170:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760782.000177:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.000185:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c59f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64675840->64679935] (req 64675840->64679935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.000201:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c59f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64675840->64679935] (req 64675840->64679935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.009514:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983099 is committed 00002000:00010000:0.0:1789760782.009795:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64675840 -> 64679936 00002000:00010000:0.0:1789760782.009800:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126344 -> 126352 00010000:00010000:0.0:1789760782.009876:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c59f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64675840->64679935] (req 64675840->64679935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.009890:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c59f8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64675840->64679935] (req 64675840->64679935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.009956:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c59f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64675840->64679935] (req 64675840->64679935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.009965:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760782.009970:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c59f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64675840->64679935] (req 64675840->64679935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.009979:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c59f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64675840->64679935] (req 64675840->64679935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.010001:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c59f8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64675840->64679935] (req 64675840->64679935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.010014:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c59f8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64675840->64679935] (req 64675840->64679935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.010175:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554787200:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10203us (10540us total) trans 4294983099 rc 0/0 00000100:00100000:0.0:1789760782.010187:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15801 00000100:00100000:0.0:1789760782.010198:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725232030160 time 725245572360 state 0x2 ref 0 00000100:00100000:1.0:1789760782.017528:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760782.017707:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554787328 00000100:00100000:2.0:1789760782.017717:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554787328 00000100:00100000:2.0:1789760782.017768:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725232030160 time 725245572360 state 0x0 ref 1 00000100:00100000:2.0:1789760782.017784:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725245832776 time 725245572360 state 0x1 ref 1 00000100:00100000:2.0:1789760782.017872:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725245832776 time 725263427387 state 0x0 ref 1 00000100:00100000:2.0:1789760782.017888:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554787328:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760782.018046:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64679936, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ed90 00010000:00010000:2.0:1789760782.018056:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c59ff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.018071:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760782.018075:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.018081:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c59ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64679936->64684031] (req 64679936->64684031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.018093:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c59ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64679936->64684031] (req 64679936->64684031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.029783:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983100 is committed 00002000:00010000:2.0:1789760782.029956:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64679936 -> 64684032 00002000:00010000:2.0:1789760782.029961:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126352 -> 126360 00010000:00010000:2.0:1789760782.029974:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c59ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64679936->64684031] (req 64679936->64684031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.029987:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c59ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64679936->64684031] (req 64679936->64684031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.029996:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c59ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64679936->64684031] (req 64679936->64684031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.030003:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760782.030008:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c59ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64679936->64684031] (req 64679936->64684031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.030017:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c59ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64679936->64684031] (req 64679936->64684031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.030037:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c59ff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64679936->64684031] (req 64679936->64684031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.030044:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c59ff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64679936->64684031] (req 64679936->64684031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.030181:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554787328:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12291us (12658us total) trans 4294983100 rc 0/0 00000100:00100000:2.0:1789760782.030191:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15802 00000100:00100000:2.0:1789760782.030200:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725245832776 time 725263427387 state 0x2 ref 0 00000100:00100000:2.0:1789760782.041618:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760782.041823:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554787456 00000100:00100000:1.0:1789760782.041834:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554787456 00000100:00100000:1.0:1789760782.041888:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725245832776 time 725263427387 state 0x0 ref 1 00000100:00100000:1.0:1789760782.041904:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725263687803 time 725263427387 state 0x1 ref 1 00000100:00100000:1.0:1789760782.041939:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725263687803 time 725287557070 state 0x0 ref 1 00000100:00100000:1.0:1789760782.041953:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554787456:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760782.042001:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64684032, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ed98 00010000:00010000:1.0:1789760782.042011:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5a06 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.042026:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760782.042031:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.042036:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5a06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64684032->64688127] (req 64684032->64688127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.042054:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5a06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64684032->64688127] (req 64684032->64688127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.052880:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983101 is committed 00002000:00010000:0.0:1789760782.053003:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64684032 -> 64688128 00002000:00010000:0.0:1789760782.053008:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126360 -> 126368 00010000:00010000:0.0:1789760782.053022:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5a06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64684032->64688127] (req 64684032->64688127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.053034:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5a06 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64684032->64688127] (req 64684032->64688127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.053042:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5a06 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64684032->64688127] (req 64684032->64688127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.053048:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760782.053053:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5a06 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64684032->64688127] (req 64684032->64688127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.053074:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5a06 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64684032->64688127] (req 64684032->64688127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.053097:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5a06 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64684032->64688127] (req 64684032->64688127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.053105:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5a06 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64684032->64688127] (req 64684032->64688127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.053669:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554787456:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11709us (12052us total) trans 4294983101 rc 0/0 00000100:00100000:0.0:1789760782.053689:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15803 00000100:00100000:0.0:1789760782.053700:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725263687803 time 725287557070 state 0x2 ref 0 00000100:00100000:0.0:1789760782.070859:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760782.070969:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554787584 00000100:00100000:2.0:1789760782.070979:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554787584 00000100:00100000:2.0:1789760782.071034:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725263687803 time 725287557070 state 0x0 ref 1 00000100:00100000:2.0:1789760782.071050:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725287817486 time 725287557070 state 0x1 ref 1 00000100:00100000:2.0:1789760782.071082:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725287817486 time 725316700801 state 0x0 ref 1 00000100:00100000:2.0:1789760782.071093:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554787584:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760782.071144:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64688128, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1eda0 00010000:00010000:2.0:1789760782.071153:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5a0d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.071174:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760782.071179:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.071185:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5a0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64688128->64692223] (req 64688128->64692223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.071197:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5a0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64688128->64692223] (req 64688128->64692223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.079069:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983102 is committed 00002000:00010000:2.0:1789760782.079291:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64688128 -> 64692224 00002000:00010000:2.0:1789760782.079297:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126368 -> 126376 00010000:00010000:2.0:1789760782.079311:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5a0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64688128->64692223] (req 64688128->64692223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.079700:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5a0d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64688128->64692223] (req 64688128->64692223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.079712:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5a0d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64688128->64692223] (req 64688128->64692223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.079720:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760782.079724:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5a0d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64688128->64692223] (req 64688128->64692223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.079733:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5a0d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64688128->64692223] (req 64688128->64692223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.079758:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5a0d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64688128->64692223] (req 64688128->64692223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.079766:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5a0d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64688128->64692223] (req 64688128->64692223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.079899:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554787584:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8801us (9044us total) trans 4294983102 rc 0/0 00000100:00100000:2.0:1789760782.079910:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15804 00000100:00100000:2.0:1789760782.079919:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725287817486 time 725316700801 state 0x2 ref 0 00000100:00100000:1.0:1789760782.088571:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760782.088662:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554787712 00000100:00100000:0.0:1789760782.088674:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554787712 00000100:00100000:0.0:1789760782.088738:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725287817486 time 725316700801 state 0x0 ref 1 00000100:00100000:0.0:1789760782.088813:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725316961217 time 725316700801 state 0x1 ref 1 00000100:00100000:0.0:1789760782.088843:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725316961217 time 725334461551 state 0x0 ref 1 00000100:00100000:0.0:1789760782.088857:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554787712:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760782.088961:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64692224, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1eda8 00010000:00010000:0.0:1789760782.088972:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a14 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.088992:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760782.088996:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.089002:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64692224->64696319] (req 64692224->64696319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.089015:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64692224->64696319] (req 64692224->64696319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.095916:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983103 is committed 00002000:00010000:0.0:1789760782.096114:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64692224 -> 64696320 00002000:00010000:0.0:1789760782.096119:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126376 -> 126384 00010000:00010000:0.0:1789760782.096132:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64692224->64696319] (req 64692224->64696319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.096144:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a14 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64692224->64696319] (req 64692224->64696319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.096152:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a14 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64692224->64696319] (req 64692224->64696319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.096160:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760782.096165:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a14 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64692224->64696319] (req 64692224->64696319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.096173:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a14 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64692224->64696319] (req 64692224->64696319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.096198:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a14 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64692224->64696319] (req 64692224->64696319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.096206:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a14 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64692224->64696319] (req 64692224->64696319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.096308:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554787712:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7449us (7743us total) trans 4294983103 rc 0/0 00000100:00100000:0.0:1789760782.096319:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15805 00000100:00100000:0.0:1789760782.096329:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725316961217 time 725334461551 state 0x2 ref 0 00000100:00100000:0.0:1789760782.102553:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760782.102932:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554787840 00000100:00100000:1.0:1789760782.102942:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554787840 00000100:00100000:1.0:1789760782.102996:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725316961217 time 725334461551 state 0x0 ref 1 00000100:00100000:1.0:1789760782.103012:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725334721967 time 725334461551 state 0x1 ref 1 00000100:00100000:1.0:1789760782.103041:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725334721967 time 725348659943 state 0x0 ref 1 00000100:00100000:1.0:1789760782.103054:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554787840:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760782.103117:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64696320, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1edb0 00010000:00010000:1.0:1789760782.103127:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5a1b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.103142:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760782.103146:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.103152:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5a1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64696320->64700415] (req 64696320->64700415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.103165:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5a1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64696320->64700415] (req 64696320->64700415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.109851:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983104 is committed 00002000:00010000:1.0:1789760782.110000:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64696320 -> 64700416 00002000:00010000:1.0:1789760782.110005:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126384 -> 126392 00010000:00010000:1.0:1789760782.110017:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5a1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64696320->64700415] (req 64696320->64700415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.110030:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5a1b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64696320->64700415] (req 64696320->64700415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.110040:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5a1b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64696320->64700415] (req 64696320->64700415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.110048:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760782.110053:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5a1b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64696320->64700415] (req 64696320->64700415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.110074:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5a1b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64696320->64700415] (req 64696320->64700415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.110095:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5a1b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64696320->64700415] (req 64696320->64700415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.110103:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5a1b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64696320->64700415] (req 64696320->64700415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.110199:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554787840:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7142us (7650us total) trans 4294983104 rc 0/0 00000100:00100000:1.0:1789760782.110211:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15806 00000100:00100000:1.0:1789760782.110220:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725334721967 time 725348659943 state 0x2 ref 0 00000100:00100000:2.0:1789760782.116016:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760782.116370:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554787968 00000100:00100000:0.0:1789760782.116411:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554787968 00000100:00100000:0.0:1789760782.116475:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725334721967 time 725348659943 state 0x0 ref 1 00000100:00100000:0.0:1789760782.116490:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725348920359 time 725348659943 state 0x1 ref 1 00000100:00100000:0.0:1789760782.116521:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725348920359 time 725362139810 state 0x0 ref 1 00000100:00100000:0.0:1789760782.116535:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554787968:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760782.116667:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64700416, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1edb8 00010000:00010000:0.0:1789760782.116677:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a22 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.116708:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760782.116712:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.116719:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64700416->64704511] (req 64700416->64704511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.116731:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64700416->64704511] (req 64700416->64704511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.125960:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983105 is committed 00002000:00010000:0.0:1789760782.137798:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64700416 -> 64704512 00002000:00010000:0.0:1789760782.137808:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126392 -> 126400 00010000:00010000:0.0:1789760782.137823:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64700416->64704511] (req 64700416->64704511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.137836:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a22 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64700416->64704511] (req 64700416->64704511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.137845:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a22 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64700416->64704511] (req 64700416->64704511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.137853:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760782.137858:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a22 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64700416->64704511] (req 64700416->64704511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.137866:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a22 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64700416->64704511] (req 64700416->64704511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.137892:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a22 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64700416->64704511] (req 64700416->64704511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.137901:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a22 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64700416->64704511] (req 64700416->64704511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.138024:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554787968:12345-192.168.203.22@tcp:4:dd.0 Request processed in 21485us (22010us total) trans 4294983105 rc 0/0 00000100:00100000:0.0:1789760782.138037:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15807 00000100:00100000:0.0:1789760782.138046:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725348920359 time 725362139810 state 0x2 ref 0 00000100:00100000:2.0:1789760782.147027:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760782.147330:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554788096 00000100:00100000:0.0:1789760782.147340:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554788096 00000100:00100000:0.0:1789760782.147432:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725348920359 time 725362139810 state 0x0 ref 1 00000100:00100000:0.0:1789760782.147447:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725362400226 time 725362139810 state 0x1 ref 1 00000100:00100000:0.0:1789760782.147469:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725362400226 time 725393087391 state 0x0 ref 1 00000100:00100000:0.0:1789760782.147480:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554788096:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760782.147527:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64704512, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1edc0 00010000:00010000:0.0:1789760782.147536:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a29 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.147552:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760782.147558:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.147564:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64704512->64708607] (req 64704512->64708607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.147821:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64704512->64708607] (req 64704512->64708607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.154526:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983106 is committed 00002000:00010000:0.0:1789760782.154623:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64704512 -> 64708608 00002000:00010000:0.0:1789760782.154628:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126400 -> 126408 00010000:00010000:0.0:1789760782.154640:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64704512->64708607] (req 64704512->64708607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.154654:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a29 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64704512->64708607] (req 64704512->64708607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.154664:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a29 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64704512->64708607] (req 64704512->64708607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.154672:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760782.154677:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a29 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64704512->64708607] (req 64704512->64708607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.154697:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a29 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64704512->64708607] (req 64704512->64708607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.154717:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a29 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64704512->64708607] (req 64704512->64708607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.154730:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a29 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64704512->64708607] (req 64704512->64708607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.154889:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554788096:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7404us (7865us total) trans 4294983106 rc 0/0 00000100:00100000:0.0:1789760782.154900:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15808 00000100:00100000:0.0:1789760782.154910:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725362400226 time 725393087391 state 0x2 ref 0 00000100:00100000:2.0:1789760782.161237:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760782.161326:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554788224 00000100:00100000:0.0:1789760782.161336:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554788224 00000100:00100000:0.0:1789760782.161435:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725362400226 time 725393087391 state 0x0 ref 1 00000100:00100000:0.0:1789760782.161450:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725393347807 time 725393087391 state 0x1 ref 1 00000100:00100000:0.0:1789760782.161481:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725393347807 time 725407099956 state 0x0 ref 1 00000100:00100000:0.0:1789760782.161494:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554788224:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760782.161537:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64708608, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1edc8 00010000:00010000:0.0:1789760782.161545:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a30 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.161560:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760782.161564:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.161570:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64708608->64712703] (req 64708608->64712703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.161593:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64708608->64712703] (req 64708608->64712703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.166198:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983107 is committed 00002000:00010000:2.0:1789760782.171027:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64708608 -> 64712704 00002000:00010000:2.0:1789760782.171036:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126408 -> 126416 00010000:00010000:2.0:1789760782.171049:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64708608->64712703] (req 64708608->64712703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.171125:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a30 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64708608->64712703] (req 64708608->64712703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.171134:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64708608->64712703] (req 64708608->64712703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.171147:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760782.171152:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64708608->64712703] (req 64708608->64712703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.171161:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a30 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64708608->64712703] (req 64708608->64712703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.171181:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a30 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64708608->64712703] (req 64708608->64712703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.171188:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5a30 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64708608->64712703] (req 64708608->64712703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.171476:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554788224:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9977us (10242us total) trans 4294983107 rc 0/0 00000100:00100000:2.0:1789760782.171489:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15809 00000100:00100000:2.0:1789760782.171508:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725393347807 time 725407099956 state 0x2 ref 0 00000100:00100000:2.0:1789760782.178158:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760782.178322:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554788352 00000100:00100000:1.0:1789760782.178333:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554788352 00000100:00100000:1.0:1789760782.178421:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725393347807 time 725407099956 state 0x0 ref 1 00000100:00100000:1.0:1789760782.178437:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725407360372 time 725407099956 state 0x1 ref 1 00000100:00100000:1.0:1789760782.178464:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725407360372 time 725424082516 state 0x0 ref 1 00000100:00100000:1.0:1789760782.178477:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554788352:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760782.178530:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64712704, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1edd0 00010000:00010000:1.0:1789760782.178540:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c5a37 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.178556:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760782.178561:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.178568:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c5a37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64712704->64716799] (req 64712704->64716799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.178582:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c5a37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64712704->64716799] (req 64712704->64716799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.187838:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983108 is committed 00002000:00010000:1.0:1789760782.188188:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64712704 -> 64716800 00002000:00010000:1.0:1789760782.188194:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126416 -> 126424 00010000:00010000:1.0:1789760782.188206:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c5a37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64712704->64716799] (req 64712704->64716799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.188219:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c5a37 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64712704->64716799] (req 64712704->64716799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.188229:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c5a37 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64712704->64716799] (req 64712704->64716799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.188237:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655de00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760782.188242:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c5a37 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64712704->64716799] (req 64712704->64716799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.188251:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c5a37 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64712704->64716799] (req 64712704->64716799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.188271:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c5a37 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64712704->64716799] (req 64712704->64716799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.188285:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c5a37 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64712704->64716799] (req 64712704->64716799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.188620:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554788352:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9900us (10226us total) trans 4294983108 rc 0/0 00000100:00100000:1.0:1789760782.188631:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15810 00000100:00100000:1.0:1789760782.188640:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725407360372 time 725424082516 state 0x2 ref 0 00000100:00100000:1.0:1789760782.194262:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760782.194906:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554788480 00000100:00100000:2.0:1789760782.194918:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554788480 00000100:00100000:2.0:1789760782.194973:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725407360372 time 725424082516 state 0x0 ref 1 00000100:00100000:2.0:1789760782.194989:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725424342932 time 725424082516 state 0x1 ref 1 00000100:00100000:2.0:1789760782.195027:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725424342932 time 725440644483 state 0x0 ref 1 00000100:00100000:2.0:1789760782.195041:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554788480:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760782.195088:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64716800, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1edd8 00010000:00010000:2.0:1789760782.195097:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a3e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.195113:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760782.195117:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.195123:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64716800->64720895] (req 64716800->64720895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.195137:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64716800->64720895] (req 64716800->64720895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.203545:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983109 is committed 00002000:00010000:2.0:1789760782.204007:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64716800 -> 64720896 00002000:00010000:2.0:1789760782.204013:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126424 -> 126432 00010000:00010000:2.0:1789760782.204027:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64716800->64720895] (req 64716800->64720895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.204040:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a3e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64716800->64720895] (req 64716800->64720895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.204049:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64716800->64720895] (req 64716800->64720895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.204113:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760782.204119:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64716800->64720895] (req 64716800->64720895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.204128:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a3e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64716800->64720895] (req 64716800->64720895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.204148:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a3e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64716800->64720895] (req 64716800->64720895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.204155:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a3e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64716800->64720895] (req 64716800->64720895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.204299:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554788480:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9255us (10042us total) trans 4294983109 rc 0/0 00000100:00100000:2.0:1789760782.204311:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15811 00000100:00100000:2.0:1789760782.204321:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725424342932 time 725440644483 state 0x2 ref 0 00000100:00100000:2.0:1789760782.220643:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760782.220735:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554788608 00000100:00100000:0.0:1789760782.220744:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554788608 00000100:00100000:0.0:1789760782.220796:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725424342932 time 725440644483 state 0x0 ref 1 00000100:00100000:0.0:1789760782.220810:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725440904899 time 725440644483 state 0x1 ref 1 00000100:00100000:0.0:1789760782.220845:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725440904899 time 725466463211 state 0x0 ref 1 00000100:00100000:0.0:1789760782.220857:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554788608:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760782.220912:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64720896, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ede0 00010000:00010000:0.0:1789760782.220921:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a45 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.220937:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760782.220941:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.220947:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64720896->64724991] (req 64720896->64724991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.220961:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64720896->64724991] (req 64720896->64724991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.225653:0:3649:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03f86d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3649:1876699573916672:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789760782.225731:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760782.226046:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb054c8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3648:1876699573917056:0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.226089:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760782.226111:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb054c8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3648:1876699573917568:0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.226134:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760782.226158:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb054c9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3648:1876699573918080:0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.226179:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760782.226199:0:3648:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:0.0:1789760782.226205:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb054c9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3648:1876699573917184:0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.226225:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760782.226241:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb054cad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3648:1876699573917696:0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.226261:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760782.226278:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb04f84a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3648:1876699573918208:0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.226297:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760782.226318:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb054c8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5711db52-8702-4d6b-b42c-8543e9bf1cb2:3646:1876699573916800:0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.226347:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760782.226394:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb054caa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573917312:0@lo:400:kworker.0 00000100:00100000:3.0:1789760782.226403:0:11526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573916672 00000100:00100000:3.0:1789760782.226411:0:11526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573916672 00000100:00100000:0.0:1789760782.226420:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760782.226467:0:11526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 362 00000100:00100000:3.0:1789760782.226477:0:11526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3649:x1876699573916672:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760782.226513:0:11526:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:3.0:1789760782.226560:0:11526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3649:x1876699573916672:12345-0@lo:13:osp-pre-1-0.0 Request processed in 84us (829us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760782.226567:0:11526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 362 00000100:00080000:3.0:1789760782.226588:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789760782.227141:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb054c8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573917824:0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.227179:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760782.227184:0:6520:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573917568 00000100:00100000:1.0:1789760782.227190:0:6520:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573917568 00000100:00100000:0.0:1789760782.227211:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb098df100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573918336:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1789760782.227233:0:6520:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573918080 00000100:00100000:0.0:1789760782.227239:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760782.227239:0:6520:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573918080 00000100:00100000:1.0:1789760782.227268:0:6520:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573917184 00000100:00100000:0.0:1789760782.227269:0:3646:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:1.0:1789760782.227273:0:6520:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573917184 00000100:00100000:0.0:1789760782.227278:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb054c9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3646:1876699573916928:0@lo:400:kworker.0 00000100:00100000:1.0:1789760782.227294:0:6520:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 879 00000100:00100000:1.0:1789760782.227304:0:6520:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3648:x1876699573917568:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.227311:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760782.227327:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb054cb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3646:1876699573917440:0@lo:400:kworker.0 00000100:00100000:3.0:1789760782.227419:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573917696 00000100:00100000:3.0:1789760782.227423:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573917696 00000100:00100000:3.0:1789760782.227610:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573918208 00000100:00100000:3.0:1789760782.227615:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573918208 00000100:00100000:3.0:1789760782.227637:0:8435:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573917824 00000100:00100000:3.0:1789760782.227641:0:8435:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573917824 00000100:00100000:3.0:1789760782.227658:0:8435:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 296 00000100:00100000:3.0:1789760782.227664:0:8435:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3648:x1876699573917696:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.227850:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760782.227868:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb054c9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3646:1876699573917952:0@lo:400:kworker.0 00000100:00100000:1.0:1789760782.227874:0:6520:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3648:x1876699573917568:12345-0@lo:400:kworker.0 Request processed in 569us (1735us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760782.227883:0:6520:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 879 00000100:00100000:0.0:1789760782.227897:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760782.227902:0:6520:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573916928 00000100:00100000:1.0:1789760782.227907:0:6520:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573916928 00000100:00100000:1.0:1789760782.227930:0:6520:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573917440 00000100:00100000:0.0:1789760782.227932:0:9992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 297 00000100:00100000:1.0:1789760782.227934:0:6520:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573917440 00000100:00100000:0.0:1789760782.227944:0:9992:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3648:x1876699573918208:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760782.227951:0:6520:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573917952 00000100:00100000:1.0:1789760782.227955:0:6520:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573917952 00000100:00100000:1.0:1789760782.227976:0:6520:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 880 00000100:00100000:1.0:1789760782.227985:0:6520:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3648:x1876699573918080:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.227990:0:9992:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3648:x1876699573918208:12345-0@lo:400:kworker.0 Request processed in 48us (1689us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760782.227998:0:9992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 297 00000100:00100000:0.0:1789760782.228015:0:9992:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 298 00000100:00100000:0.0:1789760782.228023:0:9992:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573917824:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760782.228030:0:6520:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3648:x1876699573918080:12345-0@lo:400:kworker.0 Request processed in 47us (1850us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760782.228038:0:6520:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 880 00000100:00100000:1.0:1789760782.228055:0:6520:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 881 00000100:00100000:0.0:1789760782.228067:0:9992:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573917824:12345-0@lo:400:kworker.0 Request processed in 45us (888us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760782.228074:0:9992:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 298 00000100:00100000:0.0:1789760782.228097:0:6515:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573916800 00000100:00100000:0.0:1789760782.228102:0:6515:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573916800 00000100:00100000:0.0:1789760782.228129:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 471 00000100:00100000:0.0:1789760782.228138:0:6515:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc320b6450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5711db52-8702-4d6b-b42c-8543e9bf1cb2+24:3646:x1876699573916800:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760782.228150:0:8435:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3648:x1876699573917696:12345-0@lo:400:kworker.0 Request processed in 484us (1886us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760782.228158:0:8435:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 296 00000100:00100000:0.0:1789760782.228177:0:6515:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc320b6450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5711db52-8702-4d6b-b42c-8543e9bf1cb2+24:3646:x1876699573916800:12345-0@lo:400:kworker.0 Request processed in 40us (1828us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760782.228184:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 471 00000100:00100000:0.0:1789760782.228220:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573917056 00000100:00100000:0.0:1789760782.228224:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573917056 00000100:00100000:0.0:1789760782.228241:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573917312 00000100:00100000:0.0:1789760782.228244:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573917312 00000100:00100000:0.0:1789760782.228254:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573918336 00000100:00100000:0.0:1789760782.228257:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573918336 00000100:00100000:0.0:1789760782.228267:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 386 00000100:00100000:0.0:1789760782.228273:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3648:x1876699573917056:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.228304:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3648:x1876699573917056:12345-0@lo:400:kworker.0 Request processed in 31us (2215us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760782.228311:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 386 00000100:00100000:0.0:1789760782.228323:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 387 00000100:00100000:0.0:1789760782.228329:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f85880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573917312:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.228358:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f85880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573917312:12345-0@lo:400:kworker.0 Request processed in 29us (1938us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760782.228364:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 387 00000100:00100000:0.0:1789760782.228375:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 388 00000100:00100000:0.0:1789760782.228403:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3646:x1876699573918336:12345-0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1789760782.228649:0:6520:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3648:x1876699573917184:12345-0@lo:400:kworker.0 00000004:00000024:0.0:1789760782.228847:0:6525:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446227968 avail 1299988480 00000001:00080000:3.0:1789760782.229326:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983110 is committed 00000100:00100000:0.0:1789760782.229540:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3646:x1876699573918336:12345-0@lo:41:osp-pre-0-1.0 Request processed in 1137us (2300us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760782.229548:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 388 00000100:00100000:0.0:1789760782.229797:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb054c9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3648:1876699573917184:0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.229815:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb054cad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3648:1876699573917696:0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.229825:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb04f84a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3648:1876699573918208:0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.229836:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb054c8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3648:1876699573917056:0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.229851:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb054c8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3648:1876699573917568:0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.229861:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb054c9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3648:1876699573918080:0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.229907:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb098df100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573918336:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789760782.229919:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb054c8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:5711db52-8702-4d6b-b42c-8543e9bf1cb2:3646:1876699573916800:0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.229929:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb054caa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573917312:0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.229939:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb054c8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573917824:0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.229952:0:3646:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760782.229999:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03f85180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573918464:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789760782.230035:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00010000:0.0:1789760782.230110:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64720896 -> 64724992 00002000:00010000:0.0:1789760782.230115:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126432 -> 126440 00000100:00100000:2.0:1789760782.230125:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573918464 00010000:00010000:0.0:1789760782.230127:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64720896->64724991] (req 64720896->64724991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.230134:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573918464 00010000:00010000:0.0:1789760782.230143:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a45 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64720896->64724991] (req 64720896->64724991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.230152:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a45 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64720896->64724991] (req 64720896->64724991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.230159:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760782.230163:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a45 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64720896->64724991] (req 64720896->64724991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.230172:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a45 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64720896->64724991] (req 64720896->64724991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.230192:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 363 00010000:00010000:0.0:1789760782.230193:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a45 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64720896->64724991] (req 64720896->64724991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.230204:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3646:x1876699573918464:12345-0@lo:13:osp-pre-1-1.0 00010000:00010000:0.0:1789760782.230206:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a45 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64720896->64724991] (req 64720896->64724991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00000024:2.0:1789760782.230228:0:8440:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:2.0:1789760782.230290:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3646:x1876699573918464:12345-0@lo:13:osp-pre-1-1.0 Request processed in 89us (254us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760782.230296:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 363 00000100:00100000:3.0:1789760782.230347:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03f85180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573918464:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789760782.230459:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554788608:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9601us (9823us total) trans 4294983110 rc 0/0 00000100:00100000:0.0:1789760782.230469:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15812 00000100:00100000:0.0:1789760782.230478:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725440904899 time 725466463211 state 0x2 ref 0 00000100:00100000:0.0:1789760782.231042:0:3649:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03f86d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3649:1876699573916672:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760782.231075:0:11089:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 882 00000100:00100000:0.0:1789760782.231084:0:11089:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f87b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3646:x1876699573916928:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.231140:0:11089:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f87b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3646:x1876699573916928:12345-0@lo:400:kworker.0 Request processed in 58us (3832us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760782.231147:0:11089:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 882 00000100:00100000:0.0:1789760782.231162:0:11089:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 883 00000100:00100000:0.0:1789760782.231169:0:11089:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3646:x1876699573917440:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.231201:0:11089:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3646:x1876699573917440:12345-0@lo:400:kworker.0 Request processed in 32us (3350us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760782.231208:0:11089:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 883 00000100:00100000:3.0:1789760782.231221:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb054c9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3646:1876699573916928:0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.231224:0:11089:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 884 00000100:00100000:3.0:1789760782.231240:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb054cb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3646:1876699573917440:0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.231246:0:11089:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f84700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3646:x1876699573917952:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760782.231294:0:11089:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3646:x1876699573917952:12345-0@lo:400:kworker.0 Request processed in 60us (3397us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760782.231303:0:11089:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 884 00000100:00080000:0.0:1789760782.231335:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789760782.231341:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789760782.231343:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789760782.231346:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:3.0:1789760782.231403:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb054c9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3646:1876699573917952:0@lo:400:kworker.0 00000100:00100000:1.0:1789760782.231748:0:6520:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f84000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3648:x1876699573917184:12345-0@lo:400:kworker.0 Request processed in 3683us (5518us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760782.231757:0:6520:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 881 00000100:00100000:1.0:1789760782.236591:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760782.237674:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554788736 00000100:00100000:0.0:1789760782.237696:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554788736 00000100:00100000:0.0:1789760782.237754:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725440904899 time 725466463211 state 0x0 ref 1 00000100:00100000:0.0:1789760782.237769:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725466723627 time 725466463211 state 0x1 ref 1 00000100:00100000:0.0:1789760782.237805:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725466723627 time 725483423226 state 0x0 ref 1 00000100:00100000:0.0:1789760782.237818:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554788736:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760782.237868:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64724992, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ede8 00010000:00010000:0.0:1789760782.237888:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a4c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.237904:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760782.237910:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.237916:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64724992->64729087] (req 64724992->64729087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.237930:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64724992->64729087] (req 64724992->64729087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.245028:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983111 is committed 00002000:00010000:0.0:1789760782.245257:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64724992 -> 64729088 00002000:00010000:0.0:1789760782.245263:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126440 -> 126448 00010000:00010000:0.0:1789760782.245276:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64724992->64729087] (req 64724992->64729087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.245289:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a4c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64724992->64729087] (req 64724992->64729087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.245298:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64724992->64729087] (req 64724992->64729087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.245306:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760782.245311:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64724992->64729087] (req 64724992->64729087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.245321:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a4c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64724992->64729087] (req 64724992->64729087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.245341:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a4c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64724992->64729087] (req 64724992->64729087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.245348:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a4c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64724992->64729087] (req 64724992->64729087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.245853:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554788736:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8030us (9265us total) trans 4294983111 rc 0/0 00000100:00100000:1.0:1789760782.245865:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15813 00000100:00100000:1.0:1789760782.245878:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725466723627 time 725483423226 state 0x2 ref 0 00000100:00100000:2.0:1789760782.252724:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760782.252900:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554788864 00000100:00100000:1.0:1789760782.252910:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554788864 00000100:00100000:1.0:1789760782.252965:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725466723627 time 725483423226 state 0x0 ref 1 00000100:00100000:1.0:1789760782.252983:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725483683642 time 725483423226 state 0x1 ref 1 00000100:00100000:1.0:1789760782.253016:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725483683642 time 725498634554 state 0x0 ref 1 00000100:00100000:1.0:1789760782.253030:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554788864:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760782.253077:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64729088, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1edf0 00010000:00010000:1.0:1789760782.253088:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a53 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.253105:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760782.253109:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.253116:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64729088->64733183] (req 64729088->64733183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.253131:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64729088->64733183] (req 64729088->64733183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.260098:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983112 is committed 00002000:00010000:1.0:1789760782.260198:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64729088 -> 64733184 00002000:00010000:1.0:1789760782.260204:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126448 -> 126456 00010000:00010000:1.0:1789760782.260217:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64729088->64733183] (req 64729088->64733183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.260231:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a53 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64729088->64733183] (req 64729088->64733183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.260241:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a53 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64729088->64733183] (req 64729088->64733183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.260249:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760782.260254:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a53 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64729088->64733183] (req 64729088->64733183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.260277:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a53 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64729088->64733183] (req 64729088->64733183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.260296:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a53 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64729088->64733183] (req 64729088->64733183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.260303:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a53 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64729088->64733183] (req 64729088->64733183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.260569:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554788864:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7534us (7845us total) trans 4294983112 rc 0/0 00000100:00100000:1.0:1789760782.260583:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15814 00000100:00100000:1.0:1789760782.260594:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725483683642 time 725498634554 state 0x2 ref 0 00000100:00100000:2.0:1789760782.273179:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760782.273311:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554788992 00000100:00100000:0.0:1789760782.273323:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554788992 00000100:00100000:0.0:1789760782.273483:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725483683642 time 725498634554 state 0x0 ref 1 00000100:00100000:0.0:1789760782.273501:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725498894970 time 725498634554 state 0x1 ref 1 00000100:00100000:0.0:1789760782.273569:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725498894970 time 725519186433 state 0x0 ref 1 00000100:00100000:0.0:1789760782.273583:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554788992:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760782.273631:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64733184, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1edf8 00010000:00010000:0.0:1789760782.273641:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a5a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.273657:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760782.273662:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.273668:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64733184->64737279] (req 64733184->64737279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.273691:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64733184->64737279] (req 64733184->64737279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.278751:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983113 is committed 00002000:00010000:0.0:1789760782.278886:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64733184 -> 64737280 00002000:00010000:0.0:1789760782.278891:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126456 -> 126464 00010000:00010000:0.0:1789760782.278905:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64733184->64737279] (req 64733184->64737279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.278919:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a5a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64733184->64737279] (req 64733184->64737279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.278929:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a5a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64733184->64737279] (req 64733184->64737279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.278943:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760782.278948:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a5a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64733184->64737279] (req 64733184->64737279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.278957:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a5a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64733184->64737279] (req 64733184->64737279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.278978:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a5a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64733184->64737279] (req 64733184->64737279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.278986:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a5a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64733184->64737279] (req 64733184->64737279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.279135:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554788992:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5550us (5961us total) trans 4294983113 rc 0/0 00000100:00100000:0.0:1789760782.279146:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15815 00000100:00100000:0.0:1789760782.279155:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725498894970 time 725519186433 state 0x2 ref 0 00000100:00100000:2.0:1789760782.287266:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760782.287368:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554789120 00000100:00100000:0.0:1789760782.287461:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554789120 00000100:00100000:0.0:1789760782.287516:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725498894970 time 725519186433 state 0x0 ref 1 00000100:00100000:0.0:1789760782.287533:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725519446849 time 725519186433 state 0x1 ref 1 00000100:00100000:0.0:1789760782.287600:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725519446849 time 725533218212 state 0x0 ref 1 00000100:00100000:0.0:1789760782.287617:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554789120:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760782.287666:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64737280, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ee00 00010000:00010000:0.0:1789760782.287676:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a61 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.287701:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760782.287705:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.287722:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64737280->64741375] (req 64737280->64741375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.287736:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64737280->64741375] (req 64737280->64741375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.293474:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983114 is committed 00002000:00010000:2.0:1789760782.293950:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64737280 -> 64741376 00002000:00010000:2.0:1789760782.293956:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126464 -> 126472 00010000:00010000:2.0:1789760782.293971:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64737280->64741375] (req 64737280->64741375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.293984:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a61 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64737280->64741375] (req 64737280->64741375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.293994:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64737280->64741375] (req 64737280->64741375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.294002:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760782.294008:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64737280->64741375] (req 64737280->64741375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.294018:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a61 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64737280->64741375] (req 64737280->64741375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.294041:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a61 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64737280->64741375] (req 64737280->64741375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.294050:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5a61 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64737280->64741375] (req 64737280->64741375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.294174:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554789120:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6553us (6916us total) trans 4294983114 rc 0/0 00000100:00100000:2.0:1789760782.294186:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15816 00000100:00100000:2.0:1789760782.294196:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725519446849 time 725533218212 state 0x2 ref 0 00000100:00100000:2.0:1789760782.307978:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760782.308483:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554789248 00000100:00100000:1.0:1789760782.308493:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554789248 00000100:00100000:1.0:1789760782.308550:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725519446849 time 725533218212 state 0x0 ref 1 00000100:00100000:1.0:1789760782.308565:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725533478628 time 725533218212 state 0x1 ref 1 00000100:00100000:1.0:1789760782.308602:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725533478628 time 725554220071 state 0x0 ref 1 00000100:00100000:1.0:1789760782.308614:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554789248:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760782.308851:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64741376, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ee08 00010000:00010000:1.0:1789760782.308861:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a68 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.308877:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760782.308881:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.308887:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64741376->64745471] (req 64741376->64745471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.308903:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64741376->64745471] (req 64741376->64745471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.316049:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983115 is committed 00002000:00010000:1.0:1789760782.320821:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64741376 -> 64745472 00002000:00010000:1.0:1789760782.320832:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126472 -> 126480 00010000:00010000:1.0:1789760782.320846:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64741376->64745471] (req 64741376->64745471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.320859:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a68 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64741376->64745471] (req 64741376->64745471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.320868:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a68 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64741376->64745471] (req 64741376->64745471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.320875:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760782.320880:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a68 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64741376->64745471] (req 64741376->64745471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.320889:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a68 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64741376->64745471] (req 64741376->64745471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.320913:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a68 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64741376->64745471] (req 64741376->64745471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.321584:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a68 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64741376->64745471] (req 64741376->64745471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.321829:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554789248:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13211us (13855us total) trans 4294983115 rc 0/0 00000100:00100000:1.0:1789760782.321843:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15817 00000100:00100000:1.0:1789760782.321852:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725533478628 time 725554220071 state 0x2 ref 0 00000100:00100000:0.0:1789760782.331191:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760782.331344:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554789376 00000100:00100000:1.0:1789760782.331353:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554789376 00000100:00100000:1.0:1789760782.331449:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725533478628 time 725554220071 state 0x0 ref 1 00000100:00100000:1.0:1789760782.331466:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725554480487 time 725554220071 state 0x1 ref 1 00000100:00100000:1.0:1789760782.331497:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725554480487 time 725577115466 state 0x0 ref 1 00000100:00100000:1.0:1789760782.331509:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554789376:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760782.331559:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64745472, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ee10 00010000:00010000:1.0:1789760782.331569:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a6f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.331585:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760782.331590:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.331598:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64745472->64749567] (req 64745472->64749567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.331612:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64745472->64749567] (req 64745472->64749567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.336365:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983116 is committed 00002000:00010000:1.0:1789760782.336525:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64745472 -> 64749568 00002000:00010000:1.0:1789760782.336530:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126480 -> 126488 00010000:00010000:1.0:1789760782.336544:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64745472->64749567] (req 64745472->64749567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.336557:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a6f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64745472->64749567] (req 64745472->64749567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.336567:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64745472->64749567] (req 64745472->64749567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.336574:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760782.336579:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64745472->64749567] (req 64745472->64749567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.336587:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a6f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64745472->64749567] (req 64745472->64749567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.336606:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a6f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64745472->64749567] (req 64745472->64749567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.336618:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c5a6f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64745472->64749567] (req 64745472->64749567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.336727:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554789376:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5216us (5542us total) trans 4294983116 rc 0/0 00000100:00100000:1.0:1789760782.336737:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15818 00000100:00100000:1.0:1789760782.336747:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725554480487 time 725577115466 state 0x2 ref 0 00000100:00100000:2.0:1789760782.342883:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760782.343089:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554789504 00000100:00100000:0.0:1789760782.343100:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554789504 00000100:00100000:0.0:1789760782.343157:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725554480487 time 725577115466 state 0x0 ref 1 00000100:00100000:0.0:1789760782.343173:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725577375882 time 725577115466 state 0x1 ref 1 00000100:00100000:0.0:1789760782.343208:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725577375882 time 725588826390 state 0x0 ref 1 00000100:00100000:0.0:1789760782.343221:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f84000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554789504:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760782.343268:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64749568, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ee18 00010000:00010000:0.0:1789760782.343279:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5a76 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.343296:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760782.343300:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.343306:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5a76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64749568->64753663] (req 64749568->64753663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.343321:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5a76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64749568->64753663] (req 64749568->64753663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.349011:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983117 is committed 00002000:00010000:0.0:1789760782.349117:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64749568 -> 64753664 00002000:00010000:0.0:1789760782.349122:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126488 -> 126496 00010000:00010000:0.0:1789760782.349135:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5a76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64749568->64753663] (req 64749568->64753663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.349149:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5a76 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64749568->64753663] (req 64749568->64753663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.349158:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5a76 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64749568->64753663] (req 64749568->64753663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.349166:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760782.349171:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5a76 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64749568->64753663] (req 64749568->64753663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.349180:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5a76 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64749568->64753663] (req 64749568->64753663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.349198:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5a76 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64749568->64753663] (req 64749568->64753663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.349207:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5a76 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64749568->64753663] (req 64749568->64753663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.349303:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f84000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554789504:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6080us (6426us total) trans 4294983117 rc 0/0 00000100:00100000:0.0:1789760782.349314:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15819 00000100:00100000:0.0:1789760782.349324:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725577375882 time 725588826390 state 0x2 ref 0 00000100:00100000:2.0:1789760782.354273:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760782.354631:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554789632 00000100:00100000:0.0:1789760782.354641:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554789632 00000100:00100000:0.0:1789760782.354718:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725577375882 time 725588826390 state 0x0 ref 1 00000100:00100000:0.0:1789760782.354733:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725589086806 time 725588826390 state 0x1 ref 1 00000100:00100000:0.0:1789760782.354811:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725589086806 time 725600428289 state 0x0 ref 1 00000100:00100000:0.0:1789760782.354825:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554789632:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760782.354873:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64753664, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ee20 00010000:00010000:0.0:1789760782.354888:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5a7d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.354906:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760782.354912:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.354919:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5a7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64753664->64757759] (req 64753664->64757759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.354933:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5a7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64753664->64757759] (req 64753664->64757759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.360251:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983118 is committed 00002000:00010000:2.0:1789760782.360600:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64753664 -> 64757760 00002000:00010000:2.0:1789760782.360605:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126496 -> 126504 00010000:00010000:2.0:1789760782.360618:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5a7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64753664->64757759] (req 64753664->64757759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.360630:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5a7d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64753664->64757759] (req 64753664->64757759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.360640:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5a7d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64753664->64757759] (req 64753664->64757759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.360725:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760782.360730:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5a7d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64753664->64757759] (req 64753664->64757759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.360740:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5a7d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64753664->64757759] (req 64753664->64757759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.360759:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5a7d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64753664->64757759] (req 64753664->64757759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.360771:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5a7d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64753664->64757759] (req 64753664->64757759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.360892:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554789632:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6065us (6624us total) trans 4294983118 rc 0/0 00000100:00100000:2.0:1789760782.360902:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15820 00000100:00100000:2.0:1789760782.360911:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725589086806 time 725600428289 state 0x2 ref 0 00000100:00100000:2.0:1789760782.365908:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760782.366123:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554789760 00000100:00100000:3.0:1789760782.366136:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554789760 00000100:00100000:3.0:1789760782.366197:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725589086806 time 725600428289 state 0x0 ref 1 00000100:00100000:3.0:1789760782.366212:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725600688705 time 725600428289 state 0x1 ref 1 00000100:00100000:3.0:1789760782.366284:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725600688705 time 725611901767 state 0x0 ref 1 00000100:00100000:3.0:1789760782.366297:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554789760:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760782.366371:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64757760, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ee28 00010000:00010000:3.0:1789760782.366407:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5a84 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.366423:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760782.366428:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.366443:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5a84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64757760->64761855] (req 64757760->64761855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.366458:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5a84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64757760->64761855] (req 64757760->64761855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.371183:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983119 is committed 00002000:00010000:3.0:1789760782.371299:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64757760 -> 64761856 00002000:00010000:3.0:1789760782.371305:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126504 -> 126512 00010000:00010000:3.0:1789760782.371327:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5a84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64757760->64761855] (req 64757760->64761855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.371341:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5a84 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64757760->64761855] (req 64757760->64761855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.371355:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5a84 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64757760->64761855] (req 64757760->64761855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.371371:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760782.371895:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5a84 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64757760->64761855] (req 64757760->64761855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.371910:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5a84 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64757760->64761855] (req 64757760->64761855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.371930:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5a84 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64757760->64761855] (req 64757760->64761855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.371938:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5a84 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64757760->64761855] (req 64757760->64761855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.373183:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554789760:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6880us (7276us total) trans 4294983119 rc 0/0 00000100:00100000:0.0:1789760782.373196:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15821 00000100:00100000:0.0:1789760782.373208:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725600688705 time 725611901767 state 0x2 ref 0 00000100:00100000:1.0:1789760782.378002:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760782.378148:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554789888 00000100:00100000:2.0:1789760782.378158:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554789888 00000100:00100000:2.0:1789760782.378205:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725600688705 time 725611901767 state 0x0 ref 1 00000100:00100000:2.0:1789760782.378220:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725612162183 time 725611901767 state 0x1 ref 1 00000100:00100000:2.0:1789760782.378282:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725612162183 time 725623900582 state 0x0 ref 1 00000100:00100000:2.0:1789760782.378294:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554789888:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760782.378551:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64761856, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ee30 00010000:00010000:2.0:1789760782.378561:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a8b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.378576:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760782.378581:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.378588:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64761856->64765951] (req 64761856->64765951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.378601:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64761856->64765951] (req 64761856->64765951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.383598:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983120 is committed 00002000:00010000:2.0:1789760782.383867:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64761856 -> 64765952 00002000:00010000:2.0:1789760782.383871:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126512 -> 126520 00010000:00010000:2.0:1789760782.383885:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64761856->64765951] (req 64761856->64765951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.383898:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a8b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64761856->64765951] (req 64761856->64765951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.383906:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64761856->64765951] (req 64761856->64765951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.383914:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760782.383919:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64761856->64765951] (req 64761856->64765951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.383927:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a8b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64761856->64765951] (req 64761856->64765951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.383947:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a8b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64761856->64765951] (req 64761856->64765951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.383961:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a8b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64761856->64765951] (req 64761856->64765951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.384044:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554789888:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5747us (6048us total) trans 4294983120 rc 0/0 00000100:00100000:2.0:1789760782.384054:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15822 00000100:00100000:2.0:1789760782.384808:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725612162183 time 725623900582 state 0x2 ref 0 00000100:00100000:2.0:1789760782.389030:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760782.391883:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554790016 00000100:00100000:0.0:1789760782.391897:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554790016 00000100:00100000:0.0:1789760782.391954:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725612162183 time 725623900582 state 0x0 ref 1 00000100:00100000:0.0:1789760782.391969:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725624160998 time 725623900582 state 0x1 ref 1 00000100:00100000:0.0:1789760782.392000:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725624160998 time 725637618827 state 0x0 ref 1 00000100:00100000:0.0:1789760782.392012:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554790016:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760782.392053:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64765952, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ee38 00010000:00010000:0.0:1789760782.394534:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5a92 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.394551:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760782.394557:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.394562:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5a92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64765952->64770047] (req 64765952->64770047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.394579:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5a92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64765952->64770047] (req 64765952->64770047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.399552:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983121 is committed 00002000:00010000:2.0:1789760782.405723:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64765952 -> 64770048 00002000:00010000:2.0:1789760782.405733:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126520 -> 126528 00010000:00010000:2.0:1789760782.405746:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5a92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64765952->64770047] (req 64765952->64770047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.405963:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5a92 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64765952->64770047] (req 64765952->64770047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.405972:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5a92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64765952->64770047] (req 64765952->64770047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.405979:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760782.405984:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5a92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64765952->64770047] (req 64765952->64770047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.405992:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5a92 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64765952->64770047] (req 64765952->64770047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.406012:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5a92 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64765952->64770047] (req 64765952->64770047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.406018:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c5a92 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64765952->64770047] (req 64765952->64770047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.406174:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554790016:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14158us (17146us total) trans 4294983121 rc 0/0 00000100:00100000:2.0:1789760782.406185:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15823 00000100:00100000:2.0:1789760782.406194:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725624160998 time 725637618827 state 0x2 ref 0 00000100:00100000:0.0:1789760782.412249:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760782.413324:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554790144 00000100:00100000:2.0:1789760782.413415:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554790144 00000100:00100000:2.0:1789760782.413546:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725624160998 time 725637618827 state 0x0 ref 1 00000100:00100000:2.0:1789760782.413562:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725637879243 time 725637618827 state 0x1 ref 1 00000100:00100000:2.0:1789760782.413691:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725637879243 time 725659298065 state 0x0 ref 1 00000100:00100000:2.0:1789760782.413705:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554790144:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760782.413751:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64770048, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ee40 00010000:00010000:2.0:1789760782.413770:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a99 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.413785:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760782.413790:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.413796:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64770048->64774143] (req 64770048->64774143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.413810:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64770048->64774143] (req 64770048->64774143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.419797:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983122 is committed 00002000:00010000:0.0:1789760782.419954:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64770048 -> 64774144 00002000:00010000:0.0:1789760782.419960:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126528 -> 126536 00010000:00010000:0.0:1789760782.419974:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64770048->64774143] (req 64770048->64774143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.419985:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a99 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64770048->64774143] (req 64770048->64774143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.419994:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64770048->64774143] (req 64770048->64774143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.420001:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760782.420006:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64770048->64774143] (req 64770048->64774143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.420015:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a99 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64770048->64774143] (req 64770048->64774143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.420034:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a99 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64770048->64774143] (req 64770048->64774143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.420042:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5a99 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64770048->64774143] (req 64770048->64774143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.420184:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554790144:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6478us (7940us total) trans 4294983122 rc 0/0 00000100:00100000:0.0:1789760782.420192:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15824 00000100:00100000:0.0:1789760782.420199:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725637879243 time 725659298065 state 0x2 ref 0 00000100:00100000:1.0:1789760782.424169:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760782.424279:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554790272 00000100:00100000:0.0:1789760782.424290:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554790272 00000100:00100000:0.0:1789760782.424344:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725637879243 time 725659298065 state 0x0 ref 1 00000100:00100000:0.0:1789760782.424360:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725659558481 time 725659298065 state 0x1 ref 1 00000100:00100000:2.0:1789760782.424480:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725659558481 time 725670096324 state 0x0 ref 1 00000100:00100000:2.0:1789760782.424495:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554790272:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760782.424545:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64774144, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ee48 00010000:00010000:2.0:1789760782.424554:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5aa0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.424571:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760782.424575:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.424582:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5aa0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64774144->64778239] (req 64774144->64778239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.424596:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5aa0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64774144->64778239] (req 64774144->64778239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.431631:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983123 is committed 00002000:00010000:3.0:1789760782.432079:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64774144 -> 64778240 00002000:00010000:3.0:1789760782.432084:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126536 -> 126544 00010000:00010000:3.0:1789760782.432097:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5aa0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64774144->64778239] (req 64774144->64778239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.432108:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5aa0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64774144->64778239] (req 64774144->64778239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.432115:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5aa0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64774144->64778239] (req 64774144->64778239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.432120:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760782.432123:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5aa0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64774144->64778239] (req 64774144->64778239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.432129:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5aa0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64774144->64778239] (req 64774144->64778239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.432143:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5aa0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64774144->64778239] (req 64774144->64778239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.432148:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5aa0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64774144->64778239] (req 64774144->64778239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.432225:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554790272:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7729us (8060us total) trans 4294983123 rc 0/0 00000100:00100000:3.0:1789760782.432236:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15825 00000100:00100000:3.0:1789760782.432245:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725659558481 time 725670096324 state 0x2 ref 0 00000100:00100000:2.0:1789760782.435612:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760782.435712:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554790400 00000100:00100000:1.0:1789760782.435722:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554790400 00000100:00100000:1.0:1789760782.435783:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725659558481 time 725670096324 state 0x0 ref 1 00000100:00100000:1.0:1789760782.435795:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725670356740 time 725670096324 state 0x1 ref 1 00000100:00100000:1.0:1789760782.435819:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725670356740 time 725681437864 state 0x0 ref 1 00000100:00100000:1.0:1789760782.435840:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a272a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554790400:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760782.435883:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64778240, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ee50 00010000:00010000:1.0:1789760782.435892:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aa7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.435913:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760782.435917:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.435923:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aa7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64778240->64782335] (req 64778240->64782335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.435937:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aa7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64778240->64782335] (req 64778240->64782335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.441590:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983124 is committed 00002000:00010000:1.0:1789760782.441723:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64778240 -> 64782336 00002000:00010000:1.0:1789760782.441728:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126544 -> 126552 00010000:00010000:1.0:1789760782.441741:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aa7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64778240->64782335] (req 64778240->64782335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.441753:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aa7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64778240->64782335] (req 64778240->64782335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.441762:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aa7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64778240->64782335] (req 64778240->64782335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.441769:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760782.441774:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aa7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64778240->64782335] (req 64778240->64782335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.441783:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aa7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64778240->64782335] (req 64778240->64782335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.441806:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aa7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64778240->64782335] (req 64778240->64782335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.441814:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aa7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64778240->64782335] (req 64778240->64782335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.442006:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a272a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554790400:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6171us (6399us total) trans 4294983124 rc 0/0 00000100:00100000:1.0:1789760782.442017:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15826 00000100:00100000:1.0:1789760782.442026:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725670356740 time 725681437864 state 0x2 ref 0 00000100:00100000:0.0:1789760782.455508:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760782.455621:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554790528 00000100:00100000:1.0:1789760782.455631:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554790528 00000100:00100000:1.0:1789760782.455675:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725670356740 time 725681437864 state 0x0 ref 1 00000100:00100000:1.0:1789760782.455704:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725681698280 time 725681437864 state 0x1 ref 1 00000100:00100000:1.0:1789760782.455766:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725681698280 time 725701384929 state 0x0 ref 1 00000100:00100000:1.0:1789760782.455776:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554790528:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760782.455825:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64782336, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ee58 00010000:00010000:1.0:1789760782.455833:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.455848:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760782.455853:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.455860:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64782336->64786431] (req 64782336->64786431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.455873:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64782336->64786431] (req 64782336->64786431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.461923:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983125 is committed 00002000:00010000:1.0:1789760782.462290:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64782336 -> 64786432 00002000:00010000:1.0:1789760782.462295:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126552 -> 126560 00010000:00010000:1.0:1789760782.462308:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64782336->64786431] (req 64782336->64786431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.462321:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64782336->64786431] (req 64782336->64786431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.462331:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64782336->64786431] (req 64782336->64786431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.462338:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760782.462343:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64782336->64786431] (req 64782336->64786431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.462353:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64782336->64786431] (req 64782336->64786431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.462372:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64782336->64786431] (req 64782336->64786431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.462407:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64782336->64786431] (req 64782336->64786431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.462509:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554790528:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6729us (7064us total) trans 4294983125 rc 0/0 00000100:00100000:1.0:1789760782.462520:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15827 00000100:00100000:1.0:1789760782.462530:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725681698280 time 725701384929 state 0x2 ref 0 00000100:00100000:2.0:1789760782.467586:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760782.467847:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554790656 00000100:00100000:0.0:1789760782.467857:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554790656 00000100:00100000:0.0:1789760782.467921:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725681698280 time 725701384929 state 0x0 ref 1 00000100:00100000:0.0:1789760782.467936:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725701645345 time 725701384929 state 0x1 ref 1 00000100:00100000:0.0:1789760782.467965:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725701645345 time 725713583312 state 0x0 ref 1 00000100:00100000:0.0:1789760782.467977:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a270380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554790656:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760782.468023:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64786432, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ee60 00010000:00010000:0.0:1789760782.468040:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5ab5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.468056:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760782.468061:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.468068:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5ab5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64786432->64790527] (req 64786432->64790527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.468083:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5ab5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64786432->64790527] (req 64786432->64790527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.475918:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983126 is committed 00002000:00010000:0.0:1789760782.476153:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64786432 -> 64790528 00002000:00010000:0.0:1789760782.476159:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126560 -> 126568 00010000:00010000:0.0:1789760782.476172:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5ab5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64786432->64790527] (req 64786432->64790527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.476184:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5ab5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64786432->64790527] (req 64786432->64790527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.476193:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5ab5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64786432->64790527] (req 64786432->64790527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.476200:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760782.476204:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5ab5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64786432->64790527] (req 64786432->64790527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.476212:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5ab5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64786432->64790527] (req 64786432->64790527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.476235:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5ab5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64786432->64790527] (req 64786432->64790527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.476243:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5ab5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64786432->64790527] (req 64786432->64790527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.476338:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a270380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554790656:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8358us (9278us total) trans 4294983126 rc 0/0 00000100:00100000:0.0:1789760782.476348:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15828 00000100:00100000:0.0:1789760782.476357:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725701645345 time 725713583312 state 0x2 ref 0 00000100:00100000:2.0:1789760782.485023:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760782.485352:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554790784 00000100:00100000:3.0:1789760782.485362:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554790784 00000100:00100000:3.0:1789760782.486271:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725701645345 time 725713583312 state 0x0 ref 1 00000100:00100000:3.0:1789760782.486289:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725713843728 time 725713583312 state 0x1 ref 1 00000100:00100000:3.0:1789760782.486333:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725713843728 time 725731951332 state 0x0 ref 1 00000100:00100000:3.0:1789760782.486349:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a270700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554790784:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760782.486417:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64790528, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ee68 00010000:00010000:3.0:1789760782.486427:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5abc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.486444:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760782.486449:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.486455:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5abc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64790528->64794623] (req 64790528->64794623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.486470:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5abc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64790528->64794623] (req 64790528->64794623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.493174:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983127 is committed 00002000:00010000:3.0:1789760782.495140:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64790528 -> 64794624 00002000:00010000:3.0:1789760782.495150:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126568 -> 126576 00010000:00010000:3.0:1789760782.495163:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5abc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64790528->64794623] (req 64790528->64794623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.495177:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5abc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64790528->64794623] (req 64790528->64794623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.495186:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5abc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64790528->64794623] (req 64790528->64794623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.495200:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760782.495205:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5abc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64790528->64794623] (req 64790528->64794623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.495214:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5abc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64790528->64794623] (req 64790528->64794623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.495233:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5abc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64790528->64794623] (req 64790528->64794623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.495373:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5abc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64790528->64794623] (req 64790528->64794623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.495573:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a270700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554790784:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9224us (10554us total) trans 4294983127 rc 0/0 00000100:00100000:3.0:1789760782.495584:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15829 00000100:00100000:3.0:1789760782.495594:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725713843728 time 725731951332 state 0x2 ref 0 00000100:00100000:2.0:1789760782.502046:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760782.506013:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554790912 00000100:00100000:1.0:1789760782.506028:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554790912 00000100:00100000:1.0:1789760782.506092:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725713843728 time 725731951332 state 0x0 ref 1 00000100:00100000:1.0:1789760782.506108:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725732211748 time 725731951332 state 0x1 ref 1 00000100:00100000:1.0:1789760782.506132:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725732211748 time 725751751620 state 0x0 ref 1 00000100:00100000:1.0:1789760782.506145:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554790912:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760782.506193:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64794624, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ee70 00010000:00010000:1.0:1789760782.506202:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ac3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.506219:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760782.506223:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.506229:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ac3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64794624->64798719] (req 64794624->64798719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.506243:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ac3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64794624->64798719] (req 64794624->64798719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.516696:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983128 is committed 00002000:00010000:1.0:1789760782.516938:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64794624 -> 64798720 00002000:00010000:1.0:1789760782.516943:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126576 -> 126584 00010000:00010000:1.0:1789760782.516956:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ac3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64794624->64798719] (req 64794624->64798719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.516967:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ac3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64794624->64798719] (req 64794624->64798719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.516976:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ac3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64794624->64798719] (req 64794624->64798719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.516984:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760782.516988:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ac3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64794624->64798719] (req 64794624->64798719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.516996:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ac3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64794624->64798719] (req 64794624->64798719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.517015:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ac3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64794624->64798719] (req 64794624->64798719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.517024:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ac3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64794624->64798719] (req 64794624->64798719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.517129:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554790912:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10981us (15091us total) trans 4294983128 rc 0/0 00000100:00100000:1.0:1789760782.517140:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15830 00000100:00100000:1.0:1789760782.517149:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725732211748 time 725751751620 state 0x2 ref 0 00000100:00100000:2.0:1789760782.524937:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760782.525213:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554791040 00000100:00100000:1.0:1789760782.525222:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554791040 00000100:00100000:1.0:1789760782.525274:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725732211748 time 725751751620 state 0x0 ref 1 00000100:00100000:1.0:1789760782.525290:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725752012036 time 725751751620 state 0x1 ref 1 00000100:00100000:1.0:1789760782.525316:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725752012036 time 725770933991 state 0x0 ref 1 00000100:00100000:1.0:1789760782.525329:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04562d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554791040:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760782.525420:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64798720, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ee78 00010000:00010000:1.0:1789760782.525430:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.525447:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760782.525452:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.525458:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64798720->64802815] (req 64798720->64802815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.525472:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64798720->64802815] (req 64798720->64802815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.531008:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983129 is committed 00002000:00010000:1.0:1789760782.531177:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64798720 -> 64802816 00002000:00010000:1.0:1789760782.531182:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126584 -> 126592 00010000:00010000:1.0:1789760782.531196:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64798720->64802815] (req 64798720->64802815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.531209:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64798720->64802815] (req 64798720->64802815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.531218:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64798720->64802815] (req 64798720->64802815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.531225:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760782.531230:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64798720->64802815] (req 64798720->64802815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.531239:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64798720->64802815] (req 64798720->64802815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.531259:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64798720->64802815] (req 64798720->64802815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.531266:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5aca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64798720->64802815] (req 64798720->64802815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.531358:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04562d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554791040:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6027us (6426us total) trans 4294983129 rc 0/0 00000100:00100000:1.0:1789760782.531369:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15831 00000100:00100000:1.0:1789760782.531453:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725752012036 time 725770933991 state 0x2 ref 0 00000100:00100000:2.0:1789760782.539772:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760782.539874:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554791168 00000100:00100000:0.0:1789760782.539884:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554791168 00000100:00100000:0.0:1789760782.540138:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725752012036 time 725770933991 state 0x0 ref 1 00000100:00100000:0.0:1789760782.540155:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725771194407 time 725770933991 state 0x1 ref 1 00000100:00100000:0.0:1789760782.540193:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725771194407 time 725785811182 state 0x0 ref 1 00000100:00100000:0.0:1789760782.540208:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04563800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554791168:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760782.540270:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64802816, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ee80 00010000:00010000:0.0:1789760782.540280:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5ad1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.540297:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760782.540303:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.540310:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5ad1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64802816->64806911] (req 64802816->64806911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.540325:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5ad1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64802816->64806911] (req 64802816->64806911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.545550:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983130 is committed 00002000:00010000:0.0:1789760782.546165:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64802816 -> 64806912 00002000:00010000:0.0:1789760782.546172:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126592 -> 126600 00010000:00010000:0.0:1789760782.546186:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5ad1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64802816->64806911] (req 64802816->64806911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.546200:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5ad1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64802816->64806911] (req 64802816->64806911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.546210:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5ad1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64802816->64806911] (req 64802816->64806911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.546217:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760782.546221:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5ad1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64802816->64806911] (req 64802816->64806911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.546229:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5ad1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64802816->64806911] (req 64802816->64806911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.546248:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5ad1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64802816->64806911] (req 64802816->64806911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.546260:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5ad1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64802816->64806911] (req 64802816->64806911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.546358:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04563800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554791168:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6149us (7311us total) trans 4294983130 rc 0/0 00000100:00100000:0.0:1789760782.546369:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15832 00000100:00100000:0.0:1789760782.546429:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725771194407 time 725785811182 state 0x2 ref 0 00000100:00100000:2.0:1789760782.550260:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760782.550343:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554791296 00000100:00100000:3.0:1789760782.550353:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554791296 00000100:00100000:3.0:1789760782.550449:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725771194407 time 725785811182 state 0x0 ref 1 00000100:00100000:3.0:1789760782.550463:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725786071598 time 725785811182 state 0x1 ref 1 00000100:00100000:3.0:1789760782.550485:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725786071598 time 725796103089 state 0x0 ref 1 00000100:00100000:3.0:1789760782.550497:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554791296:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760782.550546:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64806912, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ee88 00010000:00010000:3.0:1789760782.550560:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5ad8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.550579:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760782.550584:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.550591:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5ad8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64806912->64811007] (req 64806912->64811007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.550605:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5ad8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64806912->64811007] (req 64806912->64811007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.558354:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983131 is committed 00002000:00010000:3.0:1789760782.558499:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64806912 -> 64811008 00002000:00010000:3.0:1789760782.558505:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126600 -> 126608 00010000:00010000:3.0:1789760782.558517:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5ad8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64806912->64811007] (req 64806912->64811007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.558530:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5ad8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64806912->64811007] (req 64806912->64811007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.558539:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5ad8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64806912->64811007] (req 64806912->64811007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.558546:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760782.558551:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5ad8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64806912->64811007] (req 64806912->64811007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.558560:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5ad8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64806912->64811007] (req 64806912->64811007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.558580:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5ad8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64806912->64811007] (req 64806912->64811007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.558588:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5ad8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64806912->64811007] (req 64806912->64811007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.558698:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554791296:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8197us (8442us total) trans 4294983131 rc 0/0 00000100:00100000:3.0:1789760782.558708:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15833 00000100:00100000:3.0:1789760782.558717:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725786071598 time 725796103089 state 0x2 ref 0 00000100:00100000:2.0:1789760782.564055:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760782.564143:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554791424 00000100:00100000:1.0:1789760782.564154:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554791424 00000100:00100000:1.0:1789760782.564210:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725786071598 time 725796103089 state 0x0 ref 1 00000100:00100000:1.0:1789760782.564226:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725796363505 time 725796103089 state 0x1 ref 1 00000100:00100000:1.0:1789760782.564258:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725796363505 time 725809876447 state 0x0 ref 1 00000100:00100000:1.0:1789760782.564271:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554791424:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760782.564322:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64811008, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ee90 00010000:00010000:1.0:1789760782.564331:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5adf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.564348:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760782.564353:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.564361:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5adf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64811008->64815103] (req 64811008->64815103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.564375:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5adf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64811008->64815103] (req 64811008->64815103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.570720:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983132 is committed 00002000:00010000:2.0:1789760782.570864:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64811008 -> 64815104 00002000:00010000:2.0:1789760782.570869:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126608 -> 126616 00010000:00010000:2.0:1789760782.570881:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5adf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64811008->64815103] (req 64811008->64815103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.570895:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5adf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64811008->64815103] (req 64811008->64815103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.570904:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5adf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64811008->64815103] (req 64811008->64815103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.570912:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760782.570916:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5adf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64811008->64815103] (req 64811008->64815103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.570926:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5adf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64811008->64815103] (req 64811008->64815103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.570945:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5adf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64811008->64815103] (req 64811008->64815103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.570954:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5adf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64811008->64815103] (req 64811008->64815103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.571088:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554791424:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6814us (7038us total) trans 4294983132 rc 0/0 00000100:00100000:2.0:1789760782.571098:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15834 00000100:00100000:2.0:1789760782.571108:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725796363505 time 725809876447 state 0x2 ref 0 00000100:00100000:2.0:1789760782.579661:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760782.579772:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554791552 00000100:00100000:1.0:1789760782.579783:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554791552 00000100:00100000:1.0:1789760782.579831:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725796363505 time 725809876447 state 0x0 ref 1 00000100:00100000:1.0:1789760782.579846:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725810136863 time 725809876447 state 0x1 ref 1 00000100:00100000:1.0:1789760782.579876:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725810136863 time 725825494981 state 0x0 ref 1 00000100:00100000:1.0:1789760782.579888:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04562a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554791552:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760782.579932:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64815104, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1ee98 00010000:00010000:1.0:1789760782.579941:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5ae6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.579955:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760782.579959:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.579966:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5ae6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64815104->64819199] (req 64815104->64819199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.579979:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5ae6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64815104->64819199] (req 64815104->64819199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.585530:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983133 is committed 00002000:00010000:0.0:1789760782.585706:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64815104 -> 64819200 00002000:00010000:0.0:1789760782.585712:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126616 -> 126624 00010000:00010000:0.0:1789760782.585726:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5ae6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64815104->64819199] (req 64815104->64819199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.585738:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5ae6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64815104->64819199] (req 64815104->64819199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.585747:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5ae6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64815104->64819199] (req 64815104->64819199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.585756:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760782.585761:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5ae6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64815104->64819199] (req 64815104->64819199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.585769:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5ae6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64815104->64819199] (req 64815104->64819199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.585792:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5ae6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64815104->64819199] (req 64815104->64819199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.585800:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5ae6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64815104->64819199] (req 64815104->64819199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.585896:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04562a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554791552:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6004us (6239us total) trans 4294983133 rc 0/0 00000100:00100000:0.0:1789760782.585906:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15835 00000100:00100000:0.0:1789760782.585915:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725810136863 time 725825494981 state 0x2 ref 0 00000100:00100000:0.0:1789760782.588778:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760782.588901:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554791680 00000100:00100000:2.0:1789760782.588910:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554791680 00000100:00100000:2.0:1789760782.588962:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725810136863 time 725825494981 state 0x0 ref 1 00000100:00100000:2.0:1789760782.588977:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725825755397 time 725825494981 state 0x1 ref 1 00000100:00100000:2.0:1789760782.589004:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725825755397 time 725834622892 state 0x0 ref 1 00000100:00100000:2.0:1789760782.589025:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554791680:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760782.589072:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64819200, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1eea0 00010000:00010000:2.0:1789760782.589081:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5aed lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.589097:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760782.589101:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.589108:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5aed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64819200->64823295] (req 64819200->64823295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.589121:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5aed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64819200->64823295] (req 64819200->64823295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.593577:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983134 is committed 00002000:00010000:2.0:1789760782.593705:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64819200 -> 64823296 00002000:00010000:2.0:1789760782.593710:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126624 -> 126632 00010000:00010000:2.0:1789760782.593725:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5aed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64819200->64823295] (req 64819200->64823295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.593738:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5aed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64819200->64823295] (req 64819200->64823295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.593747:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5aed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64819200->64823295] (req 64819200->64823295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.593756:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760782.593762:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5aed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64819200->64823295] (req 64819200->64823295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.593781:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5aed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64819200->64823295] (req 64819200->64823295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.593801:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5aed lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64819200->64823295] (req 64819200->64823295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.593808:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5aed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64819200->64823295] (req 64819200->64823295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.593898:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554791680:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4871us (5130us total) trans 4294983134 rc 0/0 00000100:00100000:2.0:1789760782.594547:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15836 00000100:00100000:2.0:1789760782.594557:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725825755397 time 725834622892 state 0x2 ref 0 00000100:00100000:0.0:1789760782.599635:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760782.599743:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554791808 00000100:00100000:3.0:1789760782.599753:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554791808 00000100:00100000:3.0:1789760782.599803:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725825755397 time 725834622892 state 0x0 ref 1 00000100:00100000:3.0:1789760782.599817:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725834883308 time 725834622892 state 0x1 ref 1 00000100:00100000:3.0:1789760782.599878:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725834883308 time 725845496406 state 0x0 ref 1 00000100:00100000:3.0:1789760782.599891:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554791808:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760782.599948:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64823296, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1eea8 00010000:00010000:3.0:1789760782.599957:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5af4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.599972:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760782.599977:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.599983:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5af4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64823296->64827391] (req 64823296->64827391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.599996:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5af4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64823296->64827391] (req 64823296->64827391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.603759:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983135 is committed 00002000:00010000:3.0:1789760782.603897:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64823296 -> 64827392 00002000:00010000:3.0:1789760782.603902:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126632 -> 126640 00010000:00010000:3.0:1789760782.603914:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5af4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64823296->64827391] (req 64823296->64827391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.603926:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5af4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64823296->64827391] (req 64823296->64827391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.603936:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5af4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64823296->64827391] (req 64823296->64827391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.603943:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760782.603949:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5af4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64823296->64827391] (req 64823296->64827391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.603958:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5af4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64823296->64827391] (req 64823296->64827391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.603978:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5af4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64823296->64827391] (req 64823296->64827391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.603986:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5af4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64823296->64827391] (req 64823296->64827391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.604169:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554791808:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4274us (4537us total) trans 4294983135 rc 0/0 00000100:00100000:3.0:1789760782.604187:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15837 00000100:00100000:3.0:1789760782.604196:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725834883308 time 725845496406 state 0x2 ref 0 00000100:00100000:1.0:1789760782.612333:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760782.612441:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554791936 00000100:00100000:2.0:1789760782.612449:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554791936 00000100:00100000:2.0:1789760782.612504:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725834883308 time 725845496406 state 0x0 ref 1 00000100:00100000:2.0:1789760782.612521:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725845756822 time 725845496406 state 0x1 ref 1 00000100:00100000:2.0:1789760782.612558:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725845756822 time 725858176088 state 0x0 ref 1 00000100:00100000:2.0:1789760782.612569:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554791936:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760782.612621:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64827392, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1eeb0 00010000:00010000:2.0:1789760782.612631:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5afb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.612647:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760782.612651:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.612658:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5afb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64827392->64831487] (req 64827392->64831487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.612671:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5afb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64827392->64831487] (req 64827392->64831487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.616942:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983136 is committed 00002000:00010000:2.0:1789760782.617065:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64827392 -> 64831488 00002000:00010000:2.0:1789760782.617070:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126640 -> 126648 00010000:00010000:2.0:1789760782.617083:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5afb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64827392->64831487] (req 64827392->64831487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.617097:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5afb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64827392->64831487] (req 64827392->64831487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.617106:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5afb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64827392->64831487] (req 64827392->64831487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.617113:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760782.617118:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5afb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64827392->64831487] (req 64827392->64831487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.617128:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5afb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64827392->64831487] (req 64827392->64831487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.617147:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5afb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64827392->64831487] (req 64827392->64831487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.617156:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5afb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64827392->64831487] (req 64827392->64831487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.617246:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554791936:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4674us (4919us total) trans 4294983136 rc 0/0 00000100:00100000:2.0:1789760782.617258:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15838 00000100:00100000:2.0:1789760782.617266:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725845756822 time 725858176088 state 0x2 ref 0 00000100:00100000:0.0:1789760782.622078:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760782.622253:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554792064 00000100:00100000:1.0:1789760782.622263:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554792064 00000100:00100000:1.0:1789760782.622323:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725845756822 time 725858176088 state 0x0 ref 1 00000100:00100000:1.0:1789760782.622339:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725858436504 time 725858176088 state 0x1 ref 1 00000100:00100000:1.0:1789760782.622368:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725858436504 time 725867986416 state 0x0 ref 1 00000100:00100000:1.0:1789760782.622404:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554792064:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760782.622613:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64831488, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1eeb8 00010000:00010000:1.0:1789760782.622623:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5b02 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.622646:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760782.622651:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.622658:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5b02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64831488->64835583] (req 64831488->64835583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.622671:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5b02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64831488->64835583] (req 64831488->64835583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.627187:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983137 is committed 00002000:00010000:1.0:1789760782.627420:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64831488 -> 64835584 00002000:00010000:1.0:1789760782.627426:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126648 -> 126656 00010000:00010000:1.0:1789760782.627448:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5b02 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64831488->64835583] (req 64831488->64835583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.627463:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5b02 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64831488->64835583] (req 64831488->64835583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.627472:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5b02 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64831488->64835583] (req 64831488->64835583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.627480:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760782.627485:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5b02 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64831488->64835583] (req 64831488->64835583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.627493:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5b02 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64831488->64835583] (req 64831488->64835583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.627512:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5b02 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64831488->64835583] (req 64831488->64835583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.627520:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5b02 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64831488->64835583] (req 64831488->64835583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.627620:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554792064:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5236us (5545us total) trans 4294983137 rc 0/0 00000100:00100000:1.0:1789760782.627630:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15839 00000100:00100000:1.0:1789760782.627649:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725858436504 time 725867986416 state 0x2 ref 0 00000100:00100000:0.0:1789760782.633421:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760782.633650:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554792192 00000100:00100000:2.0:1789760782.633659:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554792192 00000100:00100000:2.0:1789760782.633759:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725858436504 time 725867986416 state 0x0 ref 1 00000100:00100000:2.0:1789760782.633775:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725868246832 time 725867986416 state 0x1 ref 1 00000100:00100000:2.0:1789760782.633844:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725868246832 time 725879461046 state 0x0 ref 1 00000100:00100000:2.0:1789760782.633858:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554792192:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760782.633903:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64835584, mtime: 0x6aad950d, atime: 0x0, ctime: 0x6aad950d, blocks: 0x1eec0 00010000:00010000:2.0:1789760782.633912:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5b09 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.633928:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760782.633932:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.633939:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5b09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64835584->64839679] (req 64835584->64839679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.633954:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5b09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64835584->64839679] (req 64835584->64839679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.640161:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983138 is committed 00002000:00010000:2.0:1789760782.640279:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64835584 -> 64839680 00002000:00010000:2.0:1789760782.640283:0:15101:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760781 -> 1789760782 00002000:00010000:2.0:1789760782.640286:0:15101:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760781 -> 1789760782 00002000:00010000:2.0:1789760782.640288:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126656 -> 126664 00010000:00010000:2.0:1789760782.640301:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5b09 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64835584->64839679] (req 64835584->64839679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.640313:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5b09 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64835584->64839679] (req 64835584->64839679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.640323:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5b09 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64835584->64839679] (req 64835584->64839679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.640329:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760782.640334:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5b09 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64835584->64839679] (req 64835584->64839679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.640343:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5b09 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64835584->64839679] (req 64835584->64839679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.640361:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5b09 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64835584->64839679] (req 64835584->64839679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.640368:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5b09 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64835584->64839679] (req 64835584->64839679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.641800:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554792192:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7937us (8416us total) trans 4294983138 rc 0/0 00000100:00100000:2.0:1789760782.641827:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15840 00000100:00100000:2.0:1789760782.641837:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725868246832 time 725879461046 state 0x2 ref 0 00000100:00100000:3.0:1789760782.645985:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760782.646062:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554792320 00000100:00100000:0.0:1789760782.646073:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554792320 00000100:00100000:0.0:1789760782.646127:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725868246832 time 725879461046 state 0x0 ref 1 00000100:00100000:0.0:1789760782.646143:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725879721462 time 725879461046 state 0x1 ref 1 00000100:00100000:0.0:1789760782.646189:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725879721462 time 725891807071 state 0x0 ref 1 00000100:00100000:0.0:1789760782.646202:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554792320:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760782.646250:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64839680, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1eec8 00010000:00010000:0.0:1789760782.646261:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5b10 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.646276:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760782.646280:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.646287:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5b10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64839680->64843775] (req 64839680->64843775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.646300:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5b10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64839680->64843775] (req 64839680->64843775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.651018:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983139 is committed 00002000:00010000:0.0:1789760782.652773:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64839680 -> 64843776 00002000:00010000:0.0:1789760782.652782:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126664 -> 126672 00010000:00010000:0.0:1789760782.652796:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5b10 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64839680->64843775] (req 64839680->64843775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.652862:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5b10 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64839680->64843775] (req 64839680->64843775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.652873:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5b10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64839680->64843775] (req 64839680->64843775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.652881:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760782.652885:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5b10 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64839680->64843775] (req 64839680->64843775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.652894:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5b10 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64839680->64843775] (req 64839680->64843775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.652914:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5b10 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64839680->64843775] (req 64839680->64843775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.652923:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5b10 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64839680->64843775] (req 64839680->64843775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.653338:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554792320:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7133us (7357us total) trans 4294983139 rc 0/0 00000100:00100000:0.0:1789760782.653349:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15841 00000100:00100000:0.0:1789760782.653367:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725879721462 time 725891807071 state 0x2 ref 0 00000100:00100000:0.0:1789760782.657121:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760782.657180:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554792448 00000100:00100000:1.0:1789760782.657191:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554792448 00000100:00100000:1.0:1789760782.657244:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725879721462 time 725891807071 state 0x0 ref 1 00000100:00100000:1.0:1789760782.657260:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725892067487 time 725891807071 state 0x1 ref 1 00000100:00100000:1.0:1789760782.657340:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725892067487 time 725902957976 state 0x0 ref 1 00000100:00100000:1.0:1789760782.657353:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554792448:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760782.657432:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64843776, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1eed0 00010000:00010000:1.0:1789760782.657442:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b17 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.657458:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760782.657463:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.657469:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64843776->64847871] (req 64843776->64847871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.657483:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64843776->64847871] (req 64843776->64847871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.662455:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983140 is committed 00002000:00010000:2.0:1789760782.662587:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64843776 -> 64847872 00002000:00010000:2.0:1789760782.662592:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126672 -> 126680 00010000:00010000:2.0:1789760782.662607:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b17 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64843776->64847871] (req 64843776->64847871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.662620:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b17 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64843776->64847871] (req 64843776->64847871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.662630:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64843776->64847871] (req 64843776->64847871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.662638:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760782.662653:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b17 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64843776->64847871] (req 64843776->64847871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.662664:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b17 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64843776->64847871] (req 64843776->64847871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.662695:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b17 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64843776->64847871] (req 64843776->64847871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.662702:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b17 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64843776->64847871] (req 64843776->64847871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.662851:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554792448:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5496us (5734us total) trans 4294983140 rc 0/0 00000100:00100000:2.0:1789760782.662863:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15842 00000100:00100000:2.0:1789760782.662872:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725892067487 time 725902957976 state 0x2 ref 0 00000100:00100000:1.0:1789760782.666775:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760782.666896:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554792576 00000100:00100000:0.0:1789760782.666906:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554792576 00000100:00100000:0.0:1789760782.666961:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725892067487 time 725902957976 state 0x0 ref 1 00000100:00100000:0.0:1789760782.666976:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725903218392 time 725902957976 state 0x1 ref 1 00000100:00100000:0.0:1789760782.667002:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725903218392 time 725912620874 state 0x0 ref 1 00000100:00100000:0.0:1789760782.667016:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554792576:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760782.667556:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64847872, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1eed8 00010000:00010000:0.0:1789760782.667567:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5b1e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.667587:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760782.667592:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.667599:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5b1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64847872->64851967] (req 64847872->64851967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.667613:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5b1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64847872->64851967] (req 64847872->64851967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.672351:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983141 is committed 00002000:00010000:0.0:1789760782.672541:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64847872 -> 64851968 00002000:00010000:0.0:1789760782.672545:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126680 -> 126688 00010000:00010000:0.0:1789760782.672559:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5b1e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64847872->64851967] (req 64847872->64851967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.672571:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5b1e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64847872->64851967] (req 64847872->64851967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.672582:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5b1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64847872->64851967] (req 64847872->64851967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.672588:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760782.672593:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5b1e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64847872->64851967] (req 64847872->64851967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.672602:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5b1e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64847872->64851967] (req 64847872->64851967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.672625:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5b1e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64847872->64851967] (req 64847872->64851967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.672633:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5b1e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64847872->64851967] (req 64847872->64851967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.672749:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554792576:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5731us (5979us total) trans 4294983141 rc 0/0 00000100:00100000:0.0:1789760782.672759:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15843 00000100:00100000:0.0:1789760782.672768:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725903218392 time 725912620874 state 0x2 ref 0 00000100:00100000:3.0:1789760782.676415:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760782.676570:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554792704 00000100:00100000:2.0:1789760782.676579:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554792704 00000100:00100000:2.0:1789760782.676632:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725903218392 time 725912620874 state 0x0 ref 1 00000100:00100000:2.0:1789760782.676660:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725912881290 time 725912620874 state 0x1 ref 1 00000100:00100000:2.0:1789760782.676713:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725912881290 time 725922331261 state 0x0 ref 1 00000100:00100000:2.0:1789760782.676726:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06490700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554792704:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760782.676771:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64851968, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1eee0 00010000:00010000:2.0:1789760782.676779:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5b25 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.676794:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:2.0:1789760782.676798:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.676804:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5b25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64851968->64856063] (req 64851968->64856063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.676830:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5b25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64851968->64856063] (req 64851968->64856063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.683786:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983142 is committed 00002000:00010000:3.0:1789760782.684244:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64851968 -> 64856064 00002000:00010000:3.0:1789760782.684250:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126688 -> 126696 00010000:00010000:3.0:1789760782.684263:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5b25 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64851968->64856063] (req 64851968->64856063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.684276:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5b25 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64851968->64856063] (req 64851968->64856063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.684285:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5b25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64851968->64856063] (req 64851968->64856063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.684293:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760782.684298:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5b25 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64851968->64856063] (req 64851968->64856063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.684308:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5b25 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64851968->64856063] (req 64851968->64856063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.684326:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5b25 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64851968->64856063] (req 64851968->64856063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.684334:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5b25 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64851968->64856063] (req 64851968->64856063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.684603:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06490700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554792704:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7873us (8222us total) trans 4294983142 rc 0/0 00000100:00100000:3.0:1789760782.684614:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15844 00000100:00100000:3.0:1789760782.684625:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725912881290 time 725922331261 state 0x2 ref 0 00000100:00100000:3.0:1789760782.690258:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760782.690355:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554792832 00000100:00100000:0.0:1789760782.690364:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554792832 00000100:00100000:0.0:1789760782.692142:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725912881290 time 725922331261 state 0x0 ref 1 00000100:00100000:0.0:1789760782.692250:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725922591677 time 725922331261 state 0x1 ref 1 00000100:00100000:0.0:1789760782.692339:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725922591677 time 725937956828 state 0x0 ref 1 00000100:00100000:0.0:1789760782.692356:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06493800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554792832:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760782.692429:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64856064, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1eee8 00010000:00010000:0.0:1789760782.692439:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5b2c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.692461:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760782.692465:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.692472:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5b2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64856064->64860159] (req 64856064->64860159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.692485:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5b2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64856064->64860159] (req 64856064->64860159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.696304:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983143 is committed 00002000:00010000:0.0:1789760782.696441:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64856064 -> 64860160 00002000:00010000:0.0:1789760782.696446:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126696 -> 126704 00010000:00010000:0.0:1789760782.696458:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5b2c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64856064->64860159] (req 64856064->64860159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.696470:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5b2c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64856064->64860159] (req 64856064->64860159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.696480:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5b2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64856064->64860159] (req 64856064->64860159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.696488:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760782.696493:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5b2c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64856064->64860159] (req 64856064->64860159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.696501:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5b2c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64856064->64860159] (req 64856064->64860159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.696526:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5b2c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64856064->64860159] (req 64856064->64860159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.696534:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5b2c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64856064->64860159] (req 64856064->64860159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.696623:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06493800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554792832:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4269us (6370us total) trans 4294983143 rc 0/0 00000100:00100000:0.0:1789760782.696633:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15845 00000100:00100000:0.0:1789760782.696641:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725922591677 time 725937956828 state 0x2 ref 0 00000100:00100000:0.0:1789760782.699742:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760782.699895:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554792960 00000100:00100000:1.0:1789760782.699904:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554792960 00000100:00100000:1.0:1789760782.699954:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725922591677 time 725937956828 state 0x0 ref 1 00000100:00100000:1.0:1789760782.699969:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725938217244 time 725937956828 state 0x1 ref 1 00000100:00100000:1.0:1789760782.699997:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725938217244 time 725945615714 state 0x0 ref 1 00000100:00100000:1.0:1789760782.700009:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554792960:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760782.700052:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64860160, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1eef0 00010000:00010000:1.0:1789760782.700157:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b33 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.700172:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760782.700175:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.700180:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64860160->64864255] (req 64860160->64864255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.700197:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64860160->64864255] (req 64860160->64864255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.706600:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983144 is committed 00002000:00010000:1.0:1789760782.706908:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64860160 -> 64864256 00002000:00010000:1.0:1789760782.706913:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126704 -> 126712 00010000:00010000:1.0:1789760782.706928:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b33 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64860160->64864255] (req 64860160->64864255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.706941:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b33 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64860160->64864255] (req 64860160->64864255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.706949:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64860160->64864255] (req 64860160->64864255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.706957:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760782.706962:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b33 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64860160->64864255] (req 64860160->64864255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.706970:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b33 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64860160->64864255] (req 64860160->64864255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.706995:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b33 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64860160->64864255] (req 64860160->64864255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.707003:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b33 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64860160->64864255] (req 64860160->64864255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.707203:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554792960:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7183us (7457us total) trans 4294983144 rc 0/0 00000100:00100000:1.0:1789760782.707214:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15846 00000100:00100000:1.0:1789760782.707223:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725938217244 time 725945615714 state 0x2 ref 0 00000100:00100000:3.0:1789760782.716344:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760782.716781:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554793088 00000100:00100000:1.0:1789760782.716791:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554793088 00000100:00100000:1.0:1789760782.716847:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725938217244 time 725945615714 state 0x0 ref 1 00000100:00100000:1.0:1789760782.716862:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725945876130 time 725945615714 state 0x1 ref 1 00000100:00100000:1.0:1789760782.716882:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725945876130 time 725962501135 state 0x0 ref 1 00000100:00100000:1.0:1789760782.716893:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06490a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554793088:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760782.716948:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64864256, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1eef8 00010000:00010000:1.0:1789760782.716958:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b3a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.716980:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:1.0:1789760782.716986:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.716993:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64864256->64868351] (req 64864256->64868351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.717020:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64864256->64868351] (req 64864256->64868351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.727653:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983145 is committed 00002000:00010000:1.0:1789760782.727812:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64864256 -> 64868352 00002000:00010000:1.0:1789760782.727817:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126712 -> 126720 00010000:00010000:1.0:1789760782.727830:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b3a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64864256->64868351] (req 64864256->64868351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.727842:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b3a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64864256->64868351] (req 64864256->64868351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.727851:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64864256->64868351] (req 64864256->64868351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.727858:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760782.727862:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b3a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64864256->64868351] (req 64864256->64868351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.727871:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b3a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64864256->64868351] (req 64864256->64868351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.727891:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b3a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64864256->64868351] (req 64864256->64868351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.727903:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b3a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64864256->64868351] (req 64864256->64868351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.727996:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06490a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554793088:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11100us (11665us total) trans 4294983145 rc 0/0 00000100:00100000:1.0:1789760782.728008:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15847 00000100:00100000:1.0:1789760782.728017:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725945876130 time 725962501135 state 0x2 ref 0 00000100:00100000:3.0:1789760782.737241:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760782.737427:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554793216 00000100:00100000:0.0:1789760782.737437:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554793216 00000100:00100000:0.0:1789760782.737491:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725945876130 time 725962501135 state 0x0 ref 1 00000100:00100000:0.0:1789760782.737508:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725962761551 time 725962501135 state 0x1 ref 1 00000100:00100000:0.0:1789760782.737531:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725962761551 time 725983149666 state 0x0 ref 1 00000100:00100000:0.0:1789760782.737545:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06491c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554793216:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760782.737597:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64868352, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1ef00 00010000:00010000:0.0:1789760782.737606:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5b41 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.737622:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:0.0:1789760782.737627:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.737634:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5b41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64868352->64872447] (req 64868352->64872447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.737647:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5b41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64868352->64872447] (req 64868352->64872447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.742756:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983146 is committed 00002000:00010000:0.0:1789760782.743158:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64868352 -> 64872448 00002000:00010000:0.0:1789760782.743163:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126720 -> 126728 00010000:00010000:0.0:1789760782.743188:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5b41 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64868352->64872447] (req 64868352->64872447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.743201:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5b41 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64868352->64872447] (req 64868352->64872447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.743210:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5b41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64868352->64872447] (req 64868352->64872447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.743217:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760782.743222:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5b41 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64868352->64872447] (req 64868352->64872447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.743230:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5b41 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64868352->64872447] (req 64868352->64872447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.743254:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5b41 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64868352->64872447] (req 64868352->64872447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.743261:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5b41 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64868352->64872447] (req 64868352->64872447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.743454:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06491c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554793216:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5907us (6218us total) trans 4294983146 rc 0/0 00000100:00100000:0.0:1789760782.743464:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15848 00000100:00100000:0.0:1789760782.743473:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725962761551 time 725983149666 state 0x2 ref 0 00000100:00100000:0.0:1789760782.749496:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760782.751873:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554793344 00000100:00100000:3.0:1789760782.751887:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554793344 00000100:00100000:3.0:1789760782.751951:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725962761551 time 725983149666 state 0x0 ref 1 00000100:00100000:3.0:1789760782.751966:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725983410082 time 725983149666 state 0x1 ref 1 00000100:00100000:3.0:1789760782.752059:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725983410082 time 725997676676 state 0x0 ref 1 00000100:00100000:3.0:1789760782.752076:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554793344:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760782.752124:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64872448, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1ef08 00010000:00010000:3.0:1789760782.752133:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5b48 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.752149:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=725s v=(0 0) 00010000:00010000:3.0:1789760782.752153:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.752160:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5b48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64872448->64876543] (req 64872448->64876543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.752174:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5b48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64872448->64876543] (req 64872448->64876543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760782.757143:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983147 is committed 00002000:00010000:3.0:1789760782.757651:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64872448 -> 64876544 00002000:00010000:3.0:1789760782.757657:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126728 -> 126736 00010000:00010000:3.0:1789760782.757670:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5b48 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64872448->64876543] (req 64872448->64876543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.757693:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5b48 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64872448->64876543] (req 64872448->64876543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.757701:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5b48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64872448->64876543] (req 64872448->64876543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.757709:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760782.757714:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5b48 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64872448->64876543] (req 64872448->64876543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.757723:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5b48 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64872448->64876543] (req 64872448->64876543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.757744:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5b48 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64872448->64876543] (req 64872448->64876543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.757752:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5b48 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64872448->64876543] (req 64872448->64876543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.757999:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554793344:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5922us (8510us total) trans 4294983147 rc 0/0 00000100:00100000:3.0:1789760782.758007:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15849 00000100:00100000:3.0:1789760782.758015:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725983410082 time 725997676676 state 0x2 ref 0 00000100:00100000:3.0:1789760782.765477:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760782.765613:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554793472 00000100:00100000:2.0:1789760782.765624:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554793472 00000100:00100000:2.0:1789760782.765689:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725983410082 time 725997676676 state 0x0 ref 1 00000100:00100000:2.0:1789760782.765706:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725997937092 time 725997676676 state 0x1 ref 1 00000100:00100000:2.0:1789760782.765728:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725997937092 time 726011346996 state 0x0 ref 1 00000100:00100000:2.0:1789760782.765741:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06490000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554793472:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760782.765800:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64876544, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1ef10 00010000:00010000:2.0:1789760782.765810:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b4f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.765825:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760782.765830:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.765836:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64876544->64880639] (req 64876544->64880639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.765856:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64876544->64880639] (req 64876544->64880639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.772640:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983148 is committed 00002000:00010000:2.0:1789760782.773638:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64876544 -> 64880640 00002000:00010000:2.0:1789760782.773646:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126736 -> 126744 00010000:00010000:2.0:1789760782.773659:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b4f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64876544->64880639] (req 64876544->64880639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.773688:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b4f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64876544->64880639] (req 64876544->64880639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.773986:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64876544->64880639] (req 64876544->64880639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.773998:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760782.774003:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b4f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64876544->64880639] (req 64876544->64880639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.774012:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b4f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64876544->64880639] (req 64876544->64880639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.774030:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b4f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64876544->64880639] (req 64876544->64880639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.774035:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b4f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64876544->64880639] (req 64876544->64880639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.774137:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06490000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554793472:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8393us (8667us total) trans 4294983148 rc 0/0 00000100:00100000:2.0:1789760782.774150:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15850 00000100:00100000:2.0:1789760782.774159:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725997937092 time 726011346996 state 0x2 ref 0 00000100:00100000:0.0:1789760782.778477:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760782.779328:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554793600 00000100:00100000:1.0:1789760782.779338:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554793600 00000100:00100000:1.0:1789760782.779407:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 725997937092 time 726011346996 state 0x0 ref 1 00000100:00100000:1.0:1789760782.779422:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726011607412 time 726011346996 state 0x1 ref 1 00000100:00100000:1.0:1789760782.779531:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726011607412 time 726025147844 state 0x0 ref 1 00000100:00100000:1.0:1789760782.779546:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554793600:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760782.779595:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64880640, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1ef18 00010000:00010000:1.0:1789760782.779602:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b56 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.779659:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760782.779664:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.779687:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64880640->64884735] (req 64880640->64884735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.779708:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64880640->64884735] (req 64880640->64884735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.786995:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983149 is committed 00002000:00010000:2.0:1789760782.789056:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64880640 -> 64884736 00002000:00010000:2.0:1789760782.789065:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126744 -> 126752 00010000:00010000:2.0:1789760782.789078:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b56 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64880640->64884735] (req 64880640->64884735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.789090:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b56 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64880640->64884735] (req 64880640->64884735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.789099:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64880640->64884735] (req 64880640->64884735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.789106:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760782.789111:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b56 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64880640->64884735] (req 64880640->64884735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.789120:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b56 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64880640->64884735] (req 64880640->64884735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.789139:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b56 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64880640->64884735] (req 64880640->64884735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.789308:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c5b56 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64880640->64884735] (req 64880640->64884735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.789443:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554793600:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9894us (10969us total) trans 4294983149 rc 0/0 00000100:00100000:2.0:1789760782.789457:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15851 00000100:00100000:2.0:1789760782.789466:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726011607412 time 726025147844 state 0x2 ref 0 00000100:00100000:0.0:1789760782.794421:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760782.794683:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554793728 00000100:00100000:3.0:1789760782.794700:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554793728 00000100:00100000:3.0:1789760782.794908:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726011607412 time 726025147844 state 0x0 ref 1 00000100:00100000:3.0:1789760782.794929:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726025408260 time 726025147844 state 0x1 ref 1 00000100:00100000:3.0:1789760782.794966:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726025408260 time 726040583743 state 0x0 ref 1 00000100:00100000:3.0:1789760782.794981:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554793728:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760782.796739:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64884736, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1ef20 00010000:00010000:3.0:1789760782.796877:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5b5d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.796900:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760782.796905:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.796914:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5b5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64884736->64888831] (req 64884736->64888831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.796932:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5b5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64884736->64888831] (req 64884736->64888831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.802954:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983150 is committed 00002000:00010000:3.0:1789760782.803210:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64884736 -> 64888832 00002000:00010000:3.0:1789760782.803216:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126752 -> 126760 00010000:00010000:3.0:1789760782.803233:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5b5d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64884736->64888831] (req 64884736->64888831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.803247:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5b5d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64884736->64888831] (req 64884736->64888831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.803257:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5b5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64884736->64888831] (req 64884736->64888831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.803264:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760782.803270:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5b5d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64884736->64888831] (req 64884736->64888831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.803284:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5b5d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64884736->64888831] (req 64884736->64888831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.803306:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5b5d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64884736->64888831] (req 64884736->64888831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.803313:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5b5d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64884736->64888831] (req 64884736->64888831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.806588:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554793728:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11590us (12156us total) trans 4294983150 rc 0/0 00000100:00100000:3.0:1789760782.806607:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15852 00000100:00100000:3.0:1789760782.806621:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726025408260 time 726040583743 state 0x2 ref 0 00000100:00100000:3.0:1789760782.812881:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760782.813081:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554793856 00000100:00100000:0.0:1789760782.813095:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554793856 00000100:00100000:0.0:1789760782.813281:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726025408260 time 726040583743 state 0x0 ref 1 00000100:00100000:0.0:1789760782.813301:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726040844159 time 726040583743 state 0x1 ref 1 00000100:00100000:0.0:1789760782.813433:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726040844159 time 726059049635 state 0x0 ref 1 00000100:00100000:0.0:1789760782.813451:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554793856:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760782.813532:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64888832, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1ef28 00010000:00010000:0.0:1789760782.813543:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5b64 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.813564:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760782.813569:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.813578:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5b64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64888832->64892927] (req 64888832->64892927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.813595:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5b64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64888832->64892927] (req 64888832->64892927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.819988:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983151 is committed 00002000:00010000:3.0:1789760782.820353:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64888832 -> 64892928 00002000:00010000:3.0:1789760782.820359:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126760 -> 126768 00010000:00010000:3.0:1789760782.820727:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5b64 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64888832->64892927] (req 64888832->64892927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.820744:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5b64 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64888832->64892927] (req 64888832->64892927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.820753:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5b64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64888832->64892927] (req 64888832->64892927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.820767:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760782.820774:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5b64 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64888832->64892927] (req 64888832->64892927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.820784:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5b64 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64888832->64892927] (req 64888832->64892927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.820809:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5b64 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64888832->64892927] (req 64888832->64892927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.820818:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5b64 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64888832->64892927] (req 64888832->64892927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.820942:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554793856:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7486us (8068us total) trans 4294983151 rc 0/0 00000100:00100000:3.0:1789760782.820954:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15853 00000100:00100000:3.0:1789760782.820976:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726040844159 time 726059049635 state 0x2 ref 0 00000100:00100000:0.0:1789760782.830644:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760782.831248:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554793984 00000100:00100000:2.0:1789760782.831263:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554793984 00000100:00100000:2.0:1789760782.831337:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726040844159 time 726059049635 state 0x0 ref 1 00000100:00100000:2.0:1789760782.831358:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726059310051 time 726059049635 state 0x1 ref 1 00000100:00100000:3.0:1789760782.831931:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726059310051 time 726077543131 state 0x0 ref 1 00000100:00100000:3.0:1789760782.831955:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554793984:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760782.832284:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64892928, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1ef30 00010000:00010000:3.0:1789760782.832296:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5b6b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.832322:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760782.832328:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.832336:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5b6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64892928->64897023] (req 64892928->64897023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.832352:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5b6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64892928->64897023] (req 64892928->64897023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.844471:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983152 is committed 00002000:00010000:2.0:1789760782.844636:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64892928 -> 64897024 00002000:00010000:2.0:1789760782.844642:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126768 -> 126776 00010000:00010000:2.0:1789760782.844658:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5b6b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64892928->64897023] (req 64892928->64897023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.844674:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5b6b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64892928->64897023] (req 64892928->64897023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.844697:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5b6b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64892928->64897023] (req 64892928->64897023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.844706:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760782.844712:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5b6b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64892928->64897023] (req 64892928->64897023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.844722:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5b6b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64892928->64897023] (req 64892928->64897023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.844753:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5b6b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64892928->64897023] (req 64892928->64897023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.844761:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5b6b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64892928->64897023] (req 64892928->64897023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.844900:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554793984:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12943us (14261us total) trans 4294983152 rc 0/0 00000100:00100000:2.0:1789760782.844915:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15854 00000100:00100000:2.0:1789760782.844929:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726059310051 time 726077543131 state 0x2 ref 0 00000100:00100000:0.0:1789760782.854049:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760782.854584:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554794112 00000100:00100000:2.0:1789760782.854597:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554794112 00000100:00100000:2.0:1789760782.854672:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726059310051 time 726077543131 state 0x0 ref 1 00000100:00100000:2.0:1789760782.854701:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726077803547 time 726077543131 state 0x1 ref 1 00000100:00100000:2.0:1789760782.854884:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726077803547 time 726100500938 state 0x0 ref 1 00000100:00100000:2.0:1789760782.854901:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe18050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554794112:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760782.854972:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64897024, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1ef38 00010000:00010000:2.0:1789760782.854983:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b72 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.855002:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760782.855008:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.855016:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64897024->64901119] (req 64897024->64901119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.855033:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64897024->64901119] (req 64897024->64901119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.872105:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983153 is committed 00002000:00010000:2.0:1789760782.872280:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64897024 -> 64901120 00002000:00010000:2.0:1789760782.872287:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126776 -> 126784 00010000:00010000:2.0:1789760782.872305:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b72 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64897024->64901119] (req 64897024->64901119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.872319:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b72 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64897024->64901119] (req 64897024->64901119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.872328:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b72 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64897024->64901119] (req 64897024->64901119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.872337:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760782.872342:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b72 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64897024->64901119] (req 64897024->64901119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.872353:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b72 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64897024->64901119] (req 64897024->64901119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.872418:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b72 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64897024->64901119] (req 64897024->64901119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.872433:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b72 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64897024->64901119] (req 64897024->64901119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.872551:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe18050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554794112:12345-192.168.203.22@tcp:4:dd.0 Request processed in 17645us (18509us total) trans 4294983153 rc 0/0 00000100:00100000:2.0:1789760782.872563:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15855 00000100:00100000:2.0:1789760782.872575:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726077803547 time 726100500938 state 0x2 ref 0 00000100:00100000:3.0:1789760782.880994:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760782.881235:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554794240 00000100:00100000:2.0:1789760782.881252:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554794240 00000100:00100000:2.0:1789760782.881329:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726077803547 time 726100500938 state 0x0 ref 1 00000100:00100000:2.0:1789760782.881357:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726100761354 time 726100500938 state 0x1 ref 1 00000100:00100000:2.0:1789760782.881428:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726100761354 time 726127045974 state 0x0 ref 1 00000100:00100000:2.0:1789760782.881442:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06492680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554794240:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760782.881502:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64901120, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1ef40 00010000:00010000:2.0:1789760782.881519:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b79 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.881541:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760782.881546:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.881554:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64901120->64905215] (req 64901120->64905215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.881576:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64901120->64905215] (req 64901120->64905215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.890759:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983154 is committed 00002000:00010000:3.0:1789760782.890878:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64901120 -> 64905216 00002000:00010000:3.0:1789760782.890883:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126784 -> 126792 00010000:00010000:3.0:1789760782.890896:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b79 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64901120->64905215] (req 64901120->64905215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.890910:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b79 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64901120->64905215] (req 64901120->64905215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.890918:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64901120->64905215] (req 64901120->64905215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.890925:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760782.890931:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b79 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64901120->64905215] (req 64901120->64905215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.890940:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b79 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64901120->64905215] (req 64901120->64905215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.890966:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b79 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64901120->64905215] (req 64901120->64905215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.890974:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5b79 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64901120->64905215] (req 64901120->64905215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760782.891081:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06492680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554794240:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9637us (10093us total) trans 4294983154 rc 0/0 00000100:00100000:3.0:1789760782.891091:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15856 00000100:00100000:3.0:1789760782.891099:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726100761354 time 726127045974 state 0x2 ref 0 00000100:00100000:0.0:1789760782.894929:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760782.895033:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554794368 00000100:00100000:3.0:1789760782.895048:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554794368 00000100:00100000:3.0:1789760782.895185:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726100761354 time 726127045974 state 0x0 ref 1 00000100:00100000:3.0:1789760782.895203:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726127306390 time 726127045974 state 0x1 ref 1 00000100:00100000:3.0:1789760782.895239:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726127306390 time 726140857984 state 0x0 ref 1 00000100:00100000:3.0:1789760782.895254:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554794368:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760782.895320:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64905216, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1ef48 00010000:00010000:3.0:1789760782.895332:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5b80 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.895351:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760782.895356:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760782.895364:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5b80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64905216->64909311] (req 64905216->64909311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760782.895720:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5b80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64905216->64909311] (req 64905216->64909311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.905945:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983155 is committed 00002000:00010000:0.0:1789760782.906349:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64905216 -> 64909312 00002000:00010000:0.0:1789760782.906356:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126792 -> 126800 00010000:00010000:0.0:1789760782.906373:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5b80 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64905216->64909311] (req 64905216->64909311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.906410:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5b80 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64905216->64909311] (req 64905216->64909311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.906421:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5b80 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64905216->64909311] (req 64905216->64909311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.906509:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760782.906515:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5b80 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64905216->64909311] (req 64905216->64909311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.906525:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5b80 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64905216->64909311] (req 64905216->64909311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.906550:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5b80 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64905216->64909311] (req 64905216->64909311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.906558:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c5b80 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64905216->64909311] (req 64905216->64909311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.907849:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554794368:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12031us (12368us total) trans 4294983155 rc 0/0 00000100:00100000:0.0:1789760782.907868:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15857 00000100:00100000:0.0:1789760782.907882:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726127306390 time 726140857984 state 0x2 ref 0 00000100:00100000:0.0:1789760782.912614:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760782.912740:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554794496 00000100:00100000:1.0:1789760782.912753:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554794496 00000100:00100000:1.0:1789760782.912827:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726127306390 time 726140857984 state 0x0 ref 1 00000100:00100000:1.0:1789760782.912846:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726141118400 time 726140857984 state 0x1 ref 1 00000100:00100000:1.0:1789760782.912883:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726141118400 time 726158501339 state 0x0 ref 1 00000100:00100000:1.0:1789760782.912896:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554794496:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760782.912963:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64909312, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1ef50 00010000:00010000:1.0:1789760782.912974:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5b87 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.912996:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760782.913001:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.913009:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5b87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64909312->64913407] (req 64909312->64913407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.913026:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5b87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64909312->64913407] (req 64909312->64913407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.919907:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983156 is committed 00002000:00010000:0.0:1789760782.920370:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64909312 -> 64913408 00002000:00010000:0.0:1789760782.920468:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126800 -> 126808 00010000:00010000:0.0:1789760782.920488:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5b87 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64909312->64913407] (req 64909312->64913407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.920504:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5b87 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64909312->64913407] (req 64909312->64913407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.920516:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5b87 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64909312->64913407] (req 64909312->64913407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.920523:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760782.920529:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5b87 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64909312->64913407] (req 64909312->64913407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.920540:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5b87 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64909312->64913407] (req 64909312->64913407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.920565:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5b87 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64909312->64913407] (req 64909312->64913407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.920573:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5b87 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64909312->64913407] (req 64909312->64913407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.920757:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554794496:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7854us (8150us total) trans 4294983156 rc 0/0 00000100:00100000:0.0:1789760782.920769:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15858 00000100:00100000:0.0:1789760782.920781:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726141118400 time 726158501339 state 0x2 ref 0 00000100:00100000:3.0:1789760782.925866:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760782.926338:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554794624 00000100:00100000:2.0:1789760782.926353:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554794624 00000100:00100000:2.0:1789760782.926543:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726141118400 time 726158501339 state 0x0 ref 1 00000100:00100000:2.0:1789760782.926563:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726158761755 time 726158501339 state 0x1 ref 1 00000100:00100000:2.0:1789760782.927361:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726158761755 time 726172972495 state 0x0 ref 1 00000100:00100000:2.0:1789760782.927425:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554794624:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760782.927513:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64913408, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1ef58 00010000:00010000:2.0:1789760782.927524:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5b8e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.927546:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760782.927551:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760782.927558:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5b8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64913408->64917503] (req 64913408->64917503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760782.927576:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5b8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64913408->64917503] (req 64913408->64917503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.932416:0:6508:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760782.939581:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983157 is committed 00002000:00010000:1.0:1789760782.939761:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64913408 -> 64917504 00002000:00010000:1.0:1789760782.939768:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126808 -> 126816 00010000:00010000:1.0:1789760782.939789:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5b8e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64913408->64917503] (req 64913408->64917503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.939812:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5b8e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64913408->64917503] (req 64913408->64917503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.939823:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5b8e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64913408->64917503] (req 64913408->64917503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.939835:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760782.939843:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5b8e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64913408->64917503] (req 64913408->64917503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.939856:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5b8e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64913408->64917503] (req 64913408->64917503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.939884:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5b8e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64913408->64917503] (req 64913408->64917503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.939895:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5b8e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64913408->64917503] (req 64913408->64917503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760782.940210:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554794624:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12654us (14183us total) trans 4294983157 rc 0/0 00000100:00100000:1.0:1789760782.940225:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15859 00000100:00100000:1.0:1789760782.940241:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726158761755 time 726172972495 state 0x2 ref 0 00000100:00100000:3.0:1789760782.943920:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760782.944067:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554794752 00000100:00100000:1.0:1789760782.944081:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554794752 00000100:00100000:1.0:1789760782.944156:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726158761755 time 726172972495 state 0x0 ref 1 00000100:00100000:1.0:1789760782.944176:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726173232911 time 726172972495 state 0x1 ref 1 00000100:00100000:1.0:1789760782.944214:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726173232911 time 726189831476 state 0x0 ref 1 00000100:00100000:1.0:1789760782.944228:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06491180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554794752:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760782.944296:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64917504, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1ef60 00010000:00010000:1.0:1789760782.944308:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5b95 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.944327:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760782.944331:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.944339:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5b95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64917504->64921599] (req 64917504->64921599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.944356:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5b95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64917504->64921599] (req 64917504->64921599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760782.951593:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983158 is committed 00002000:00010000:0.0:1789760782.951955:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64917504 -> 64921600 00002000:00010000:0.0:1789760782.951962:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126816 -> 126824 00010000:00010000:0.0:1789760782.951978:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5b95 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64917504->64921599] (req 64917504->64921599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.951993:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5b95 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64917504->64921599] (req 64917504->64921599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.952003:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5b95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64917504->64921599] (req 64917504->64921599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.952011:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760782.952017:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5b95 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64917504->64921599] (req 64917504->64921599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.952028:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5b95 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64917504->64921599] (req 64917504->64921599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.952100:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5b95 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64917504->64921599] (req 64917504->64921599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.952109:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5b95 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64917504->64921599] (req 64917504->64921599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760782.952268:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06491180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554794752:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8034us (8355us total) trans 4294983158 rc 0/0 00000100:00100000:0.0:1789760782.952281:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15860 00000100:00100000:0.0:1789760782.952294:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726173232911 time 726189831476 state 0x2 ref 0 00000100:00100000:0.0:1789760782.963557:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760782.963741:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554794880 00000100:00100000:3.0:1789760782.963754:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554794880 00000100:00100000:3.0:1789760782.963832:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726173232911 time 726189831476 state 0x0 ref 1 00000100:00100000:3.0:1789760782.963852:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726190091892 time 726189831476 state 0x1 ref 1 00000100:00100000:3.0:1789760782.963892:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726190091892 time 726209509831 state 0x0 ref 1 00000100:00100000:3.0:1789760782.963905:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554794880:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760782.964051:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64921600, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1ef68 00010000:00010000:1.0:1789760782.964065:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5b9c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.964085:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760782.964089:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760782.964097:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5b9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64921600->64925695] (req 64921600->64925695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760782.964116:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5b9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64921600->64925695] (req 64921600->64925695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760782.971408:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983159 is committed 00002000:00010000:0.0:1789760782.977918:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64921600 -> 64925696 00002000:00010000:0.0:1789760782.977933:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126824 -> 126832 00010000:00010000:0.0:1789760782.977952:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5b9c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64921600->64925695] (req 64921600->64925695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.977968:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5b9c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64921600->64925695] (req 64921600->64925695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.977979:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5b9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64921600->64925695] (req 64921600->64925695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.977987:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760782.977992:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5b9c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64921600->64925695] (req 64921600->64925695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.978004:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5b9c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64921600->64925695] (req 64921600->64925695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.978032:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5b9c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64921600->64925695] (req 64921600->64925695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.978041:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5b9c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64921600->64925695] (req 64921600->64925695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760782.978831:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554794880:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14915us (15279us total) trans 4294983159 rc 0/0 00000100:00100000:2.0:1789760782.978848:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15861 00000100:00100000:2.0:1789760782.978861:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726190091892 time 726209509831 state 0x2 ref 0 00000100:00100000:2.0:1789760782.989002:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760782.989264:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554795008 00000100:00100000:0.0:1789760782.989278:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554795008 00000100:00100000:0.0:1789760782.989361:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726190091892 time 726209509831 state 0x0 ref 1 00000100:00100000:0.0:1789760782.989435:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726209770247 time 726209509831 state 0x1 ref 1 00000100:00100000:0.0:1789760782.989479:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726209770247 time 726235095774 state 0x0 ref 1 00000100:00100000:0.0:1789760782.989494:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554795008:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760782.989561:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64925696, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1ef70 00010000:00010000:0.0:1789760782.989572:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5ba3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.989594:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760782.989600:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760782.989609:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5ba3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64925696->64929791] (req 64925696->64929791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760782.989628:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5ba3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64925696->64929791] (req 64925696->64929791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760782.998171:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983160 is committed 00002000:00010000:2.0:1789760783.008934:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64925696 -> 64929792 00002000:00010000:2.0:1789760783.008948:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126832 -> 126840 00010000:00010000:2.0:1789760783.008966:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5ba3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64925696->64929791] (req 64925696->64929791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.008981:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5ba3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64925696->64929791] (req 64925696->64929791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.008991:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5ba3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64925696->64929791] (req 64925696->64929791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.008999:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760783.009004:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5ba3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64925696->64929791] (req 64925696->64929791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.009014:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5ba3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64925696->64929791] (req 64925696->64929791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.009046:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5ba3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64925696->64929791] (req 64925696->64929791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.009055:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c5ba3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64925696->64929791] (req 64925696->64929791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.009201:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554795008:12345-192.168.203.22@tcp:4:dd.0 Request processed in 19702us (20210us total) trans 4294983160 rc 0/0 00000100:00100000:2.0:1789760783.009216:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15862 00000100:00100000:2.0:1789760783.009228:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726209770247 time 726235095774 state 0x2 ref 0 00000100:00100000:3.0:1789760783.016203:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760783.016620:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554795136 00000100:00100000:1.0:1789760783.016633:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554795136 00000100:00100000:1.0:1789760783.016743:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726209770247 time 726235095774 state 0x0 ref 1 00000100:00100000:1.0:1789760783.016762:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726235356190 time 726235095774 state 0x1 ref 1 00000100:00100000:1.0:1789760783.016801:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726235356190 time 726262418713 state 0x0 ref 1 00000100:00100000:1.0:1789760783.016815:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554795136:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760783.016882:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64929792, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1ef78 00010000:00010000:1.0:1789760783.016894:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5baa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.016921:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760783.016926:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.016934:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5baa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64929792->64933887] (req 64929792->64933887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.016950:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5baa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64929792->64933887] (req 64929792->64933887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760783.019503:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760783.024374:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983161 is committed 00002000:00010000:1.0:1789760783.024553:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64929792 -> 64933888 00002000:00010000:1.0:1789760783.024559:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126840 -> 126848 00010000:00010000:1.0:1789760783.024578:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5baa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64929792->64933887] (req 64929792->64933887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.024592:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5baa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64929792->64933887] (req 64929792->64933887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.024601:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5baa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64929792->64933887] (req 64929792->64933887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.024609:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760783.024615:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5baa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64929792->64933887] (req 64929792->64933887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.024626:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5baa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64929792->64933887] (req 64929792->64933887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.024657:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5baa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64929792->64933887] (req 64929792->64933887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.024665:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5baa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64929792->64933887] (req 64929792->64933887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.024804:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554795136:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7981us (8608us total) trans 4294983161 rc 0/0 00000100:00100000:1.0:1789760783.024818:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15863 00000100:00100000:1.0:1789760783.024831:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726235356190 time 726262418713 state 0x2 ref 0 00000100:00100000:0.0:1789760783.028862:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760783.029053:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554795264 00000100:00100000:3.0:1789760783.029078:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554795264 00000100:00100000:3.0:1789760783.029160:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726235356190 time 726262418713 state 0x0 ref 1 00000100:00100000:3.0:1789760783.029180:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726262679129 time 726262418713 state 0x1 ref 1 00000100:00100000:3.0:1789760783.029216:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726262679129 time 726274833026 state 0x0 ref 1 00000100:00100000:3.0:1789760783.029229:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554795264:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760783.029298:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64933888, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1ef80 00010000:00010000:3.0:1789760783.029309:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5bb1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.029331:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760783.029336:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.029343:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5bb1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64933888->64937983] (req 64933888->64937983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.029362:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5bb1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64933888->64937983] (req 64933888->64937983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.035858:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983162 is committed 00002000:00010000:3.0:1789760783.042018:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64933888 -> 64937984 00002000:00010000:3.0:1789760783.042033:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126848 -> 126856 00010000:00010000:3.0:1789760783.042100:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5bb1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64933888->64937983] (req 64933888->64937983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.042117:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5bb1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64933888->64937983] (req 64933888->64937983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.042127:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5bb1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64933888->64937983] (req 64933888->64937983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.042142:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760783.042148:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5bb1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64933888->64937983] (req 64933888->64937983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.042158:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5bb1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64933888->64937983] (req 64933888->64937983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.042185:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5bb1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64933888->64937983] (req 64933888->64937983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.042193:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5bb1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64933888->64937983] (req 64933888->64937983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.042339:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554795264:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13104us (13486us total) trans 4294983162 rc 0/0 00000100:00100000:3.0:1789760783.042354:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15864 00000100:00100000:3.0:1789760783.042367:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726262679129 time 726274833026 state 0x2 ref 0 00000100:00100000:2.0:1789760783.052688:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760783.052881:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554795392 00000100:00100000:1.0:1789760783.052894:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554795392 00000100:00100000:1.0:1789760783.052970:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726262679129 time 726274833026 state 0x0 ref 1 00000100:00100000:1.0:1789760783.052990:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726275093442 time 726274833026 state 0x1 ref 1 00000100:00100000:1.0:1789760783.053061:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726275093442 time 726298678156 state 0x0 ref 1 00000100:00100000:1.0:1789760783.053077:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04562d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554795392:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760783.053139:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64937984, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1ef88 00010000:00010000:1.0:1789760783.053150:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5bb8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.053170:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760783.053176:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.053184:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5bb8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64937984->64942079] (req 64937984->64942079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.053201:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5bb8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64937984->64942079] (req 64937984->64942079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.061937:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983163 is committed 00002000:00010000:1.0:1789760783.065740:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64937984 -> 64942080 00002000:00010000:1.0:1789760783.065755:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126856 -> 126864 00010000:00010000:1.0:1789760783.065773:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5bb8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64937984->64942079] (req 64937984->64942079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.065790:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5bb8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64937984->64942079] (req 64937984->64942079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.065802:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5bb8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64937984->64942079] (req 64937984->64942079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.065811:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760783.065818:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5bb8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64937984->64942079] (req 64937984->64942079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.065830:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5bb8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64937984->64942079] (req 64937984->64942079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.065860:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5bb8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64937984->64942079] (req 64937984->64942079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.065875:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5bb8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64937984->64942079] (req 64937984->64942079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.066020:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04562d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554795392:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12940us (13349us total) trans 4294983163 rc 0/0 00000100:00100000:1.0:1789760783.066033:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15865 00000100:00100000:1.0:1789760783.066045:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726275093442 time 726298678156 state 0x2 ref 0 00000100:00100000:0.0:1789760783.070162:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760783.076814:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554795520 00000100:00100000:2.0:1789760783.076836:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554795520 00000100:00100000:2.0:1789760783.076929:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726275093442 time 726298678156 state 0x0 ref 1 00000100:00100000:2.0:1789760783.076948:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726298938572 time 726298678156 state 0x1 ref 1 00000100:00100000:2.0:1789760783.077070:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726298938572 time 726322687666 state 0x0 ref 1 00000100:00100000:2.0:1789760783.077086:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554795520:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760783.077149:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64942080, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1ef90 00010000:00010000:2.0:1789760783.077161:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bbf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.077180:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760783.077186:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.077193:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bbf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64942080->64946175] (req 64942080->64946175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.077217:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bbf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64942080->64946175] (req 64942080->64946175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.082526:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983164 is committed 00002000:00010000:2.0:1789760783.082902:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64942080 -> 64946176 00002000:00010000:2.0:1789760783.082909:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126864 -> 126872 00010000:00010000:2.0:1789760783.082928:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bbf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64942080->64946175] (req 64942080->64946175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.082943:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bbf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64942080->64946175] (req 64942080->64946175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.082952:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bbf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64942080->64946175] (req 64942080->64946175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.082959:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760783.082964:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bbf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64942080->64946175] (req 64942080->64946175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.082973:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bbf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64942080->64946175] (req 64942080->64946175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.082998:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bbf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64942080->64946175] (req 64942080->64946175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.083007:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bbf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64942080->64946175] (req 64942080->64946175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.083133:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554795520:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6042us (12978us total) trans 4294983164 rc 0/0 00000100:00100000:2.0:1789760783.083147:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15866 00000100:00100000:2.0:1789760783.083159:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726298938572 time 726322687666 state 0x2 ref 0 00000100:00100000:0.0:1789760783.087778:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760783.087864:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554795648 00000100:00100000:2.0:1789760783.087877:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554795648 00000100:00100000:2.0:1789760783.087943:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726298938572 time 726322687666 state 0x0 ref 1 00000100:00100000:2.0:1789760783.087961:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726322948082 time 726322687666 state 0x1 ref 1 00000100:00100000:1.0:1789760783.088145:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726322948082 time 726333760353 state 0x0 ref 1 00000100:00100000:1.0:1789760783.088162:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554795648:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760783.088539:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64946176, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1ef98 00010000:00010000:1.0:1789760783.088551:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5bc6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.088570:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760783.088574:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.088583:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5bc6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64946176->64950271] (req 64946176->64950271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.088599:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5bc6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64946176->64950271] (req 64946176->64950271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.098438:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983165 is committed 00002000:00010000:0.0:1789760783.098874:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64946176 -> 64950272 00002000:00010000:0.0:1789760783.098880:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126872 -> 126880 00010000:00010000:0.0:1789760783.098947:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5bc6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64946176->64950271] (req 64946176->64950271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.098969:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5bc6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64946176->64950271] (req 64946176->64950271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.098979:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5bc6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64946176->64950271] (req 64946176->64950271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.098987:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760783.098993:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5bc6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64946176->64950271] (req 64946176->64950271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.099004:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5bc6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64946176->64950271] (req 64946176->64950271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.099031:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5bc6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64946176->64950271] (req 64946176->64950271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.099039:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5bc6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64946176->64950271] (req 64946176->64950271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.099546:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554795648:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11380us (11774us total) trans 4294983165 rc 0/0 00000100:00100000:0.0:1789760783.099559:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15867 00000100:00100000:0.0:1789760783.099571:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726322948082 time 726333760353 state 0x2 ref 0 00000100:00100000:1.0:1789760783.103747:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760783.103888:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554795776 00000100:00100000:3.0:1789760783.103901:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554795776 00000100:00100000:3.0:1789760783.103983:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726322948082 time 726333760353 state 0x0 ref 1 00000100:00100000:3.0:1789760783.104004:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726334020769 time 726333760353 state 0x1 ref 1 00000100:00100000:3.0:1789760783.104057:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726334020769 time 726349674447 state 0x0 ref 1 00000100:00100000:3.0:1789760783.104071:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554795776:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760783.104138:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64950272, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1efa0 00010000:00010000:3.0:1789760783.104149:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5bcd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.104168:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760783.104174:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.104181:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5bcd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64950272->64954367] (req 64950272->64954367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.104207:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5bcd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64950272->64954367] (req 64950272->64954367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.111916:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983166 is committed 00002000:00010000:3.0:1789760783.112259:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64950272 -> 64954368 00002000:00010000:3.0:1789760783.112266:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126880 -> 126888 00010000:00010000:3.0:1789760783.112283:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5bcd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64950272->64954367] (req 64950272->64954367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.112299:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5bcd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64950272->64954367] (req 64950272->64954367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.112309:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5bcd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64950272->64954367] (req 64950272->64954367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.112323:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760783.112329:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5bcd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64950272->64954367] (req 64950272->64954367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.112340:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5bcd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64950272->64954367] (req 64950272->64954367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.112364:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5bcd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64950272->64954367] (req 64950272->64954367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.112373:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5bcd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64950272->64954367] (req 64950272->64954367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.112548:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554795776:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8471us (8814us total) trans 4294983166 rc 0/0 00000100:00100000:3.0:1789760783.112562:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15868 00000100:00100000:3.0:1789760783.112576:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726334020769 time 726349674447 state 0x2 ref 0 00000100:00100000:3.0:1789760783.117632:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760783.117735:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554795904 00000100:00100000:2.0:1789760783.117750:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554795904 00000100:00100000:2.0:1789760783.117824:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726334020769 time 726349674447 state 0x0 ref 1 00000100:00100000:2.0:1789760783.117843:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726349934863 time 726349674447 state 0x1 ref 1 00000100:00100000:2.0:1789760783.117884:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726349934863 time 726363502150 state 0x0 ref 1 00000100:00100000:2.0:1789760783.117898:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06490700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554795904:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760783.117964:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64954368, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1efa8 00010000:00010000:2.0:1789760783.117976:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bd4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.118586:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760783.118593:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.118601:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bd4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64954368->64958463] (req 64954368->64958463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.118620:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bd4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64954368->64958463] (req 64954368->64958463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.125455:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983167 is committed 00002000:00010000:2.0:1789760783.125605:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64954368 -> 64958464 00002000:00010000:2.0:1789760783.125610:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126888 -> 126896 00010000:00010000:2.0:1789760783.125628:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bd4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64954368->64958463] (req 64954368->64958463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.125643:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bd4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64954368->64958463] (req 64954368->64958463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.125653:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bd4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64954368->64958463] (req 64954368->64958463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.125662:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760783.125668:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bd4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64954368->64958463] (req 64954368->64958463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.125689:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bd4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64954368->64958463] (req 64954368->64958463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.125714:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bd4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64954368->64958463] (req 64954368->64958463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.125727:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bd4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64954368->64958463] (req 64954368->64958463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.125971:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06490700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554795904:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8066us (8346us total) trans 4294983167 rc 0/0 00000100:00100000:2.0:1789760783.125984:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15869 00000100:00100000:2.0:1789760783.125998:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726349934863 time 726363502150 state 0x2 ref 0 00000100:00100000:0.0:1789760783.131651:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760783.131757:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554796032 00000100:00100000:2.0:1789760783.131770:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554796032 00000100:00100000:2.0:1789760783.131842:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726349934863 time 726363502150 state 0x0 ref 1 00000100:00100000:2.0:1789760783.131861:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726363762566 time 726363502150 state 0x1 ref 1 00000100:00100000:2.0:1789760783.131905:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726363762566 time 726377523263 state 0x0 ref 1 00000100:00100000:2.0:1789760783.131919:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554796032:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760783.131991:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64958464, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1efb0 00010000:00010000:2.0:1789760783.132001:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bdb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.132022:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760783.132027:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.132034:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bdb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64958464->64962559] (req 64958464->64962559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.132064:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bdb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64958464->64962559] (req 64958464->64962559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.138841:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983168 is committed 00002000:00010000:0.0:1789760783.139636:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64958464 -> 64962560 00002000:00010000:0.0:1789760783.139644:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126896 -> 126904 00010000:00010000:0.0:1789760783.139661:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bdb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64958464->64962559] (req 64958464->64962559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.139688:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bdb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64958464->64962559] (req 64958464->64962559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.139698:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bdb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64958464->64962559] (req 64958464->64962559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.139707:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760783.139713:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bdb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64958464->64962559] (req 64958464->64962559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.139724:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bdb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64958464->64962559] (req 64958464->64962559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.139749:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bdb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64958464->64962559] (req 64958464->64962559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.139758:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5bdb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64958464->64962559] (req 64958464->64962559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.139947:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554796032:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8022us (8303us total) trans 4294983168 rc 0/0 00000100:00100000:0.0:1789760783.139960:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15870 00000100:00100000:0.0:1789760783.139974:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726363762566 time 726377523263 state 0x2 ref 0 00000100:00100000:1.0:1789760783.147797:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760783.147969:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554796160 00000100:00100000:0.0:1789760783.147983:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554796160 00000100:00100000:0.0:1789760783.148069:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726363762566 time 726377523263 state 0x0 ref 1 00000100:00100000:0.0:1789760783.148088:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726377783679 time 726377523263 state 0x1 ref 1 00000100:00100000:0.0:1789760783.148127:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726377783679 time 726393745041 state 0x0 ref 1 00000100:00100000:0.0:1789760783.148142:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554796160:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760783.148207:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64962560, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1efb8 00010000:00010000:0.0:1789760783.148222:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5be2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.148243:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760783.148248:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.148255:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5be2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64962560->64966655] (req 64962560->64966655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.148282:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5be2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64962560->64966655] (req 64962560->64966655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.155978:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983169 is committed 00002000:00010000:0.0:1789760783.156117:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64962560 -> 64966656 00002000:00010000:0.0:1789760783.156123:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126904 -> 126912 00010000:00010000:0.0:1789760783.156139:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5be2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64962560->64966655] (req 64962560->64966655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.156162:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5be2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64962560->64966655] (req 64962560->64966655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.156171:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5be2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64962560->64966655] (req 64962560->64966655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.156179:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760783.156185:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5be2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64962560->64966655] (req 64962560->64966655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.156195:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5be2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64962560->64966655] (req 64962560->64966655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.156220:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5be2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64962560->64966655] (req 64962560->64966655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.156229:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5be2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64962560->64966655] (req 64962560->64966655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.156416:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554796160:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8270us (8625us total) trans 4294983169 rc 0/0 00000100:00100000:0.0:1789760783.156429:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15871 00000100:00100000:0.0:1789760783.156440:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726377783679 time 726393745041 state 0x2 ref 0 00000100:00100000:0.0:1789760783.162279:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760783.164393:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554796288 00000100:00100000:3.0:1789760783.164412:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554796288 00000100:00100000:3.0:1789760783.164494:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726377783679 time 726393745041 state 0x0 ref 1 00000100:00100000:3.0:1789760783.164514:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726394005457 time 726393745041 state 0x1 ref 1 00000100:00100000:3.0:1789760783.164557:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726394005457 time 726410174615 state 0x0 ref 1 00000100:00100000:3.0:1789760783.164572:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554796288:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760783.164635:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64966656, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1efc0 00010000:00010000:3.0:1789760783.164647:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5be9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.164667:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760783.164684:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.164693:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5be9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64966656->64970751] (req 64966656->64970751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.164716:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5be9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64966656->64970751] (req 64966656->64970751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.170734:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983170 is committed 00002000:00010000:3.0:1789760783.173954:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64966656 -> 64970752 00002000:00010000:3.0:1789760783.173966:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126912 -> 126920 00010000:00010000:3.0:1789760783.173984:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5be9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64966656->64970751] (req 64966656->64970751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.173999:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5be9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64966656->64970751] (req 64966656->64970751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.174008:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5be9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64966656->64970751] (req 64966656->64970751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.174021:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760783.174027:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5be9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64966656->64970751] (req 64966656->64970751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.175609:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5be9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64966656->64970751] (req 64966656->64970751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.175653:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5be9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64966656->64970751] (req 64966656->64970751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.175664:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5be9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64966656->64970751] (req 64966656->64970751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.176023:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554796288:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11445us (13748us total) trans 4294983170 rc 0/0 00000100:00100000:3.0:1789760783.176046:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15872 00000100:00100000:3.0:1789760783.176058:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726394005457 time 726410174615 state 0x2 ref 0 00000100:00100000:1.0:1789760783.179866:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760783.180021:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554796416 00000100:00100000:2.0:1789760783.180033:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554796416 00000100:00100000:2.0:1789760783.180106:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726394005457 time 726410174615 state 0x0 ref 1 00000100:00100000:2.0:1789760783.180126:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726410435031 time 726410174615 state 0x1 ref 1 00000100:00100000:2.0:1789760783.180227:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726410435031 time 726425843373 state 0x0 ref 1 00000100:00100000:2.0:1789760783.180242:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554796416:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760783.180313:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64970752, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1efc8 00010000:00010000:2.0:1789760783.180324:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5bf0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.180344:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760783.180348:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.180355:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5bf0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64970752->64974847] (req 64970752->64974847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.180371:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5bf0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64970752->64974847] (req 64970752->64974847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.187245:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983171 is committed 00002000:00010000:0.0:1789760783.187803:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64970752 -> 64974848 00002000:00010000:0.0:1789760783.187811:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126920 -> 126928 00010000:00010000:0.0:1789760783.187828:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5bf0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64970752->64974847] (req 64970752->64974847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.187843:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5bf0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64970752->64974847] (req 64970752->64974847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.187853:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5bf0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64970752->64974847] (req 64970752->64974847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.187861:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760783.187867:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5bf0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64970752->64974847] (req 64970752->64974847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.187877:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5bf0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64970752->64974847] (req 64970752->64974847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.187901:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5bf0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64970752->64974847] (req 64970752->64974847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.187909:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5bf0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64970752->64974847] (req 64970752->64974847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.188503:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554796416:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8254us (8642us total) trans 4294983171 rc 0/0 00000100:00100000:0.0:1789760783.188518:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15873 00000100:00100000:0.0:1789760783.188531:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726410435031 time 726425843373 state 0x2 ref 0 00000100:00100000:1.0:1789760783.193549:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760783.193619:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554796544 00000100:00100000:0.0:1789760783.193632:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554796544 00000100:00100000:0.0:1789760783.193724:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726410435031 time 726425843373 state 0x0 ref 1 00000100:00100000:0.0:1789760783.193744:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726426103789 time 726425843373 state 0x1 ref 1 00000100:00100000:0.0:1789760783.193780:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726426103789 time 726439398019 state 0x0 ref 1 00000100:00100000:0.0:1789760783.193794:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554796544:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760783.193859:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64974848, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1efd0 00010000:00010000:0.0:1789760783.193870:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5bf7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.193892:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760783.193897:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.193904:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5bf7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64974848->64978943] (req 64974848->64978943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.193919:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5bf7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64974848->64978943] (req 64974848->64978943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.200839:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983172 is committed 00002000:00010000:0.0:1789760783.201041:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64974848 -> 64978944 00002000:00010000:0.0:1789760783.201100:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126928 -> 126936 00010000:00010000:0.0:1789760783.201119:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5bf7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64974848->64978943] (req 64974848->64978943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.201141:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5bf7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64974848->64978943] (req 64974848->64978943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.201151:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5bf7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64974848->64978943] (req 64974848->64978943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.201159:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760783.201164:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5bf7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64974848->64978943] (req 64974848->64978943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.201174:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5bf7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64974848->64978943] (req 64974848->64978943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.201199:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5bf7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64974848->64978943] (req 64974848->64978943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.201208:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5bf7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64974848->64978943] (req 64974848->64978943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.201339:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554796544:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7541us (7800us total) trans 4294983172 rc 0/0 00000100:00100000:0.0:1789760783.201351:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15874 00000100:00100000:0.0:1789760783.201363:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726426103789 time 726439398019 state 0x2 ref 0 00000100:00100000:1.0:1789760783.205705:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760783.205866:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554796672 00000100:00100000:0.0:1789760783.205879:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554796672 00000100:00100000:0.0:1789760783.205946:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726426103789 time 726439398019 state 0x0 ref 1 00000100:00100000:0.0:1789760783.205964:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726439658435 time 726439398019 state 0x1 ref 1 00000100:00100000:0.0:1789760783.206038:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726439658435 time 726451655672 state 0x0 ref 1 00000100:00100000:0.0:1789760783.206060:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554796672:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760783.206123:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64978944, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1efd8 00010000:00010000:0.0:1789760783.206135:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5bfe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.206154:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760783.206160:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.206168:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5bfe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64978944->64983039] (req 64978944->64983039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.206184:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5bfe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64978944->64983039] (req 64978944->64983039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.215582:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983173 is committed 00002000:00010000:1.0:1789760783.220957:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64978944 -> 64983040 00002000:00010000:1.0:1789760783.221205:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126936 -> 126944 00010000:00010000:1.0:1789760783.221226:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5bfe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64978944->64983039] (req 64978944->64983039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.221241:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5bfe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64978944->64983039] (req 64978944->64983039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.221251:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5bfe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64978944->64983039] (req 64978944->64983039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.221324:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760783.221330:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5bfe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64978944->64983039] (req 64978944->64983039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.221341:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5bfe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64978944->64983039] (req 64978944->64983039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.221366:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5bfe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64978944->64983039] (req 64978944->64983039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.221374:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5bfe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64978944->64983039] (req 64978944->64983039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.221658:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554796672:12345-192.168.203.22@tcp:4:dd.0 Request processed in 15601us (15958us total) trans 4294983173 rc 0/0 00000100:00100000:1.0:1789760783.221670:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15875 00000100:00100000:1.0:1789760783.221692:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726439658435 time 726451655672 state 0x2 ref 0 00000100:00100000:1.0:1789760783.225719:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760783.225855:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554796800 00000100:00100000:3.0:1789760783.225869:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554796800 00000100:00100000:3.0:1789760783.225926:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726439658435 time 726451655672 state 0x0 ref 1 00000100:00100000:3.0:1789760783.225940:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726451916088 time 726451655672 state 0x1 ref 1 00000100:00100000:3.0:1789760783.225980:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726451916088 time 726471598332 state 0x0 ref 1 00000100:00100000:3.0:1789760783.226006:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554796800:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760783.226057:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64983040, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1efe0 00010000:00010000:3.0:1789760783.226066:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5c05 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.226082:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760783.226086:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.226092:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5c05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64983040->64987135] (req 64983040->64987135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.226106:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5c05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64983040->64987135] (req 64983040->64987135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.231437:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983174 is committed 00002000:00010000:0.0:1789760783.232320:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64983040 -> 64987136 00002000:00010000:0.0:1789760783.232329:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126944 -> 126952 00010000:00010000:0.0:1789760783.232526:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5c05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64983040->64987135] (req 64983040->64987135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.232541:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5c05 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64983040->64987135] (req 64983040->64987135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.232556:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5c05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64983040->64987135] (req 64983040->64987135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.232563:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760783.232568:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5c05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64983040->64987135] (req 64983040->64987135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.232580:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5c05 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64983040->64987135] (req 64983040->64987135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.232606:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5c05 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64983040->64987135] (req 64983040->64987135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.232614:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c5c05 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64983040->64987135] (req 64983040->64987135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.232899:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554796800:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6893us (7182us total) trans 4294983174 rc 0/0 00000100:00100000:1.0:1789760783.232915:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15876 00000100:00100000:1.0:1789760783.232928:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726451916088 time 726471598332 state 0x2 ref 0 00000100:00100000:1.0:1789760783.236530:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760783.236620:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554796928 00000100:00100000:0.0:1789760783.236632:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554796928 00000100:00100000:0.0:1789760783.236720:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726451916088 time 726471598332 state 0x0 ref 1 00000100:00100000:0.0:1789760783.236740:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726471858748 time 726471598332 state 0x1 ref 1 00000100:00100000:0.0:1789760783.236777:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726471858748 time 726482395103 state 0x0 ref 1 00000100:00100000:0.0:1789760783.236791:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554796928:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760783.236859:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64987136, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1efe8 00010000:00010000:0.0:1789760783.236871:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c0c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.236890:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760783.236896:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.236903:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64987136->64991231] (req 64987136->64991231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.236919:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64987136->64991231] (req 64987136->64991231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.246459:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983175 is committed 00002000:00010000:0.0:1789760783.246616:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64987136 -> 64991232 00002000:00010000:0.0:1789760783.246624:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126952 -> 126960 00010000:00010000:0.0:1789760783.246644:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64987136->64991231] (req 64987136->64991231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.246664:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c0c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64987136->64991231] (req 64987136->64991231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.246681:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64987136->64991231] (req 64987136->64991231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.246690:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760783.246696:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64987136->64991231] (req 64987136->64991231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.246711:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c0c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64987136->64991231] (req 64987136->64991231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.246736:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c0c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64987136->64991231] (req 64987136->64991231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.246745:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c0c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64987136->64991231] (req 64987136->64991231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.246928:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554796928:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10132us (10544us total) trans 4294983175 rc 0/0 00000100:00100000:0.0:1789760783.246941:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15877 00000100:00100000:0.0:1789760783.246953:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726471858748 time 726482395103 state 0x2 ref 0 00000100:00100000:1.0:1789760783.250262:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760783.250717:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554797056 00000100:00100000:0.0:1789760783.250731:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554797056 00000100:00100000:1.0:1789760783.251209:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726471858748 time 726482395103 state 0x0 ref 1 00000100:00100000:1.0:1789760783.251231:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726482655519 time 726482395103 state 0x1 ref 1 00000100:00100000:1.0:1789760783.251325:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726482655519 time 726496941723 state 0x0 ref 1 00000100:00100000:1.0:1789760783.251341:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554797056:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760783.251454:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64991232, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1eff0 00010000:00010000:1.0:1789760783.251466:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5c13 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.251493:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760783.251498:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.251506:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5c13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64991232->64995327] (req 64991232->64995327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.251523:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5c13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64991232->64995327] (req 64991232->64995327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.258909:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983176 is committed 00002000:00010000:1.0:1789760783.259147:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64991232 -> 64995328 00002000:00010000:1.0:1789760783.259154:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126960 -> 126968 00010000:00010000:1.0:1789760783.259172:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5c13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64991232->64995327] (req 64991232->64995327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.259188:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5c13 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64991232->64995327] (req 64991232->64995327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.259197:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5c13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64991232->64995327] (req 64991232->64995327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.259205:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760783.259211:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5c13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64991232->64995327] (req 64991232->64995327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.259222:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5c13 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64991232->64995327] (req 64991232->64995327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.259594:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5c13 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64991232->64995327] (req 64991232->64995327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.259605:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5c13 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64991232->64995327] (req 64991232->64995327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.259853:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554797056:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8507us (9598us total) trans 4294983176 rc 0/0 00000100:00100000:1.0:1789760783.259865:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15878 00000100:00100000:1.0:1789760783.259876:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726482655519 time 726496941723 state 0x2 ref 0 00000100:00100000:1.0:1789760783.264914:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760783.265976:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554797184 00000100:00100000:2.0:1789760783.265991:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554797184 00000100:00100000:2.0:1789760783.266125:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726482655519 time 726496941723 state 0x0 ref 1 00000100:00100000:2.0:1789760783.266145:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726497202139 time 726496941723 state 0x1 ref 1 00000100:00100000:2.0:1789760783.266321:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726497202139 time 726511938425 state 0x0 ref 1 00000100:00100000:2.0:1789760783.266337:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554797184:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760783.266661:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64995328, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1eff8 00010000:00010000:2.0:1789760783.266683:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5c1a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.266704:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760783.266709:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.266717:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5c1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64995328->64999423] (req 64995328->64999423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.266735:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5c1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64995328->64999423] (req 64995328->64999423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.275946:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983177 is committed 00002000:00010000:2.0:1789760783.276247:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64995328 -> 64999424 00002000:00010000:2.0:1789760783.276253:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126968 -> 126976 00010000:00010000:2.0:1789760783.276270:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5c1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64995328->64999423] (req 64995328->64999423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.276283:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5c1a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64995328->64999423] (req 64995328->64999423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.276292:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5c1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64995328->64999423] (req 64995328->64999423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.276299:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760783.276305:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5c1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64995328->64999423] (req 64995328->64999423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.276314:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5c1a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64995328->64999423] (req 64995328->64999423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.276338:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5c1a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64995328->64999423] (req 64995328->64999423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.276346:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5c1a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64995328->64999423] (req 64995328->64999423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.276852:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554797184:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10511us (11945us total) trans 4294983177 rc 0/0 00000100:00100000:2.0:1789760783.276867:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15879 00000100:00100000:2.0:1789760783.276880:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726497202139 time 726511938425 state 0x2 ref 0 00000100:00100000:2.0:1789760783.282029:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760783.282184:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554797312 00000100:00100000:3.0:1789760783.282196:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554797312 00000100:00100000:3.0:1789760783.282271:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726497202139 time 726511938425 state 0x0 ref 1 00000100:00100000:3.0:1789760783.282291:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726512198841 time 726511938425 state 0x1 ref 1 00000100:00100000:3.0:1789760783.282329:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726512198841 time 726527947444 state 0x0 ref 1 00000100:00100000:3.0:1789760783.282344:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04563800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554797312:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760783.282438:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 64999424, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1f000 00010000:00010000:3.0:1789760783.282450:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5c21 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.282469:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760783.282474:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.282482:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5c21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64999424->65003519] (req 64999424->65003519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.282499:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5c21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64999424->65003519] (req 64999424->65003519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.290930:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983178 is committed 00002000:00010000:3.0:1789760783.291277:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 64999424 -> 65003520 00002000:00010000:3.0:1789760783.291283:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126976 -> 126984 00010000:00010000:3.0:1789760783.291301:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5c21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64999424->65003519] (req 64999424->65003519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.291316:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5c21 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64999424->65003519] (req 64999424->65003519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.291327:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5c21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64999424->65003519] (req 64999424->65003519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.291335:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760783.291341:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5c21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64999424->65003519] (req 64999424->65003519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.291362:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5c21 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64999424->65003519] (req 64999424->65003519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.291423:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5c21 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64999424->65003519] (req 64999424->65003519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.291441:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5c21 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [64999424->65003519] (req 64999424->65003519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.291567:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04563800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554797312:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9218us (9552us total) trans 4294983178 rc 0/0 00000100:00100000:3.0:1789760783.291581:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15880 00000100:00100000:3.0:1789760783.291592:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726512198841 time 726527947444 state 0x2 ref 0 00000100:00100000:1.0:1789760783.296557:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760783.296818:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554797440 00000100:00100000:0.0:1789760783.296832:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554797440 00000100:00100000:0.0:1789760783.296907:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726512198841 time 726527947444 state 0x0 ref 1 00000100:00100000:0.0:1789760783.296927:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726528207860 time 726527947444 state 0x1 ref 1 00000100:00100000:0.0:1789760783.296967:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726528207860 time 726542584327 state 0x0 ref 1 00000100:00100000:0.0:1789760783.296981:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554797440:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760783.297046:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65003520, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1f008 00010000:00010000:0.0:1789760783.297058:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c28 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.297078:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760783.297083:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.297090:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65003520->65007615] (req 65003520->65007615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.297105:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65003520->65007615] (req 65003520->65007615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.305805:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983179 is committed 00002000:00010000:0.0:1789760783.305948:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65003520 -> 65007616 00002000:00010000:0.0:1789760783.305955:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126984 -> 126992 00010000:00010000:0.0:1789760783.305972:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65003520->65007615] (req 65003520->65007615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.305988:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c28 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65003520->65007615] (req 65003520->65007615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.305998:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65003520->65007615] (req 65003520->65007615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.306006:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760783.306012:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65003520->65007615] (req 65003520->65007615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.306029:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c28 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65003520->65007615] (req 65003520->65007615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.306066:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c28 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65003520->65007615] (req 65003520->65007615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.306075:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c28 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65003520->65007615] (req 65003520->65007615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.306198:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554797440:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9212us (9648us total) trans 4294983179 rc 0/0 00000100:00100000:0.0:1789760783.306211:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15881 00000100:00100000:0.0:1789760783.306224:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726528207860 time 726542584327 state 0x2 ref 0 00000100:00100000:1.0:1789760783.310201:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760783.310796:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554797568 00000100:00100000:2.0:1789760783.310810:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554797568 00000100:00100000:2.0:1789760783.310885:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726528207860 time 726542584327 state 0x0 ref 1 00000100:00100000:2.0:1789760783.310903:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726542844743 time 726542584327 state 0x1 ref 1 00000100:00100000:2.0:1789760783.310942:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726542844743 time 726556560118 state 0x0 ref 1 00000100:00100000:2.0:1789760783.310954:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554797568:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760783.311014:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65007616, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1f010 00010000:00010000:2.0:1789760783.311026:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5c2f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.311044:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760783.311048:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.311055:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5c2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65007616->65011711] (req 65007616->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.311071:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5c2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65007616->65011711] (req 65007616->65011711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.320225:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983180 is committed 00002000:00010000:2.0:1789760783.320364:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65007616 -> 65011712 00002000:00010000:2.0:1789760783.320370:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 126992 -> 127000 00010000:00010000:2.0:1789760783.320415:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5c2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65007616->65011711] (req 65007616->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.320431:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5c2f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65007616->65011711] (req 65007616->65011711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.320442:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5c2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65007616->65011711] (req 65007616->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.320451:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760783.320457:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5c2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65007616->65011711] (req 65007616->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.320467:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5c2f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65007616->65011711] (req 65007616->65011711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.320492:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5c2f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65007616->65011711] (req 65007616->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.320715:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5c2f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65007616->65011711] (req 65007616->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.320850:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554797568:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9890us (10659us total) trans 4294983180 rc 0/0 00000100:00100000:2.0:1789760783.320861:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15882 00000100:00100000:2.0:1789760783.320881:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726542844743 time 726556560118 state 0x2 ref 0 00000100:00100000:2.0:1789760783.324838:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760783.325180:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554797696 00000100:00100000:1.0:1789760783.325193:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554797696 00000100:00100000:1.0:1789760783.325262:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726542844743 time 726556560118 state 0x0 ref 1 00000100:00100000:1.0:1789760783.325280:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726556820534 time 726556560118 state 0x1 ref 1 00000100:00100000:1.0:1789760783.325315:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726556820534 time 726570933394 state 0x0 ref 1 00000100:00100000:1.0:1789760783.325330:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554797696:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760783.325412:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65011712, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1f018 00010000:00010000:1.0:1789760783.325424:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5c36 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.325445:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760783.325450:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.325458:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5c36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65011712->65015807] (req 65011712->65015807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.325472:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5c36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65011712->65015807] (req 65011712->65015807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.332055:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983181 is committed 00002000:00010000:1.0:1789760783.332225:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65011712 -> 65015808 00002000:00010000:1.0:1789760783.332231:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127000 -> 127008 00010000:00010000:1.0:1789760783.332248:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5c36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65011712->65015807] (req 65011712->65015807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.332264:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5c36 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65011712->65015807] (req 65011712->65015807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.332273:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5c36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65011712->65015807] (req 65011712->65015807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.332282:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760783.332287:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5c36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65011712->65015807] (req 65011712->65015807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.332298:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5c36 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65011712->65015807] (req 65011712->65015807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.332329:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5c36 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65011712->65015807] (req 65011712->65015807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.332338:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5c36 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65011712->65015807] (req 65011712->65015807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.332514:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554797696:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7180us (7684us total) trans 4294983181 rc 0/0 00000100:00100000:1.0:1789760783.332527:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15883 00000100:00100000:1.0:1789760783.332539:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726556820534 time 726570933394 state 0x2 ref 0 00000100:00100000:1.0:1789760783.336086:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760783.336225:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554797824 00000100:00100000:3.0:1789760783.336237:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554797824 00000100:00100000:3.0:1789760783.336311:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726556820534 time 726570933394 state 0x0 ref 1 00000100:00100000:3.0:1789760783.336330:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726571193810 time 726570933394 state 0x1 ref 1 00000100:00100000:3.0:1789760783.336367:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726571193810 time 726581984678 state 0x0 ref 1 00000100:00100000:3.0:1789760783.336424:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554797824:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760783.336487:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65015808, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1f020 00010000:00010000:3.0:1789760783.336500:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5c3d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.336522:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760783.336527:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.336535:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5c3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65015808->65019903] (req 65015808->65019903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.336553:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5c3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65015808->65019903] (req 65015808->65019903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.345210:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983182 is committed 00002000:00010000:1.0:1789760783.345418:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65015808 -> 65019904 00002000:00010000:1.0:1789760783.345424:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127008 -> 127016 00010000:00010000:1.0:1789760783.345499:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5c3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65015808->65019903] (req 65015808->65019903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.345515:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5c3d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65015808->65019903] (req 65015808->65019903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.345525:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5c3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65015808->65019903] (req 65015808->65019903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.345533:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760783.345539:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5c3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65015808->65019903] (req 65015808->65019903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.345549:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5c3d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65015808->65019903] (req 65015808->65019903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.345574:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5c3d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65015808->65019903] (req 65015808->65019903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.345582:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c5c3d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65015808->65019903] (req 65015808->65019903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.345765:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554797824:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9379us (9685us total) trans 4294983182 rc 0/0 00000100:00100000:1.0:1789760783.345778:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15884 00000100:00100000:1.0:1789760783.345790:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726571193810 time 726581984678 state 0x2 ref 0 00000100:00100000:2.0:1789760783.350832:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760783.351660:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554797952 00000100:00100000:0.0:1789760783.351685:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554797952 00000100:00100000:0.0:1789760783.351764:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726571193810 time 726581984678 state 0x0 ref 1 00000100:00100000:0.0:1789760783.351784:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726582245094 time 726581984678 state 0x1 ref 1 00000100:00100000:0.0:1789760783.351859:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726582245094 time 726597477232 state 0x0 ref 1 00000100:00100000:0.0:1789760783.351940:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554797952:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760783.352017:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65019904, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1f028 00010000:00010000:0.0:1789760783.352029:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c44 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.352060:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760783.352066:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.352074:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65019904->65023999] (req 65019904->65023999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.352092:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65019904->65023999] (req 65019904->65023999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.358892:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983183 is committed 00002000:00010000:1.0:1789760783.359209:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65019904 -> 65024000 00002000:00010000:1.0:1789760783.359215:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127016 -> 127024 00010000:00010000:1.0:1789760783.359232:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65019904->65023999] (req 65019904->65023999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.359247:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c44 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65019904->65023999] (req 65019904->65023999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.359257:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65019904->65023999] (req 65019904->65023999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.359271:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760783.359278:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65019904->65023999] (req 65019904->65023999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.359288:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c44 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65019904->65023999] (req 65019904->65023999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.359313:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c44 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65019904->65023999] (req 65019904->65023999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.359320:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5c44 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65019904->65023999] (req 65019904->65023999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.360044:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554797952:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8099us (9216us total) trans 4294983183 rc 0/0 00000100:00100000:1.0:1789760783.360068:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15885 00000100:00100000:1.0:1789760783.360080:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726582245094 time 726597477232 state 0x2 ref 0 00000100:00100000:2.0:1789760783.366798:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760783.366925:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554798080 00000100:00100000:1.0:1789760783.366950:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554798080 00000100:00100000:1.0:1789760783.367022:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726582245094 time 726597477232 state 0x0 ref 1 00000100:00100000:1.0:1789760783.367041:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726597737648 time 726597477232 state 0x1 ref 1 00000100:00100000:1.0:1789760783.367191:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726597737648 time 726612807762 state 0x0 ref 1 00000100:00100000:1.0:1789760783.367209:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04562a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554798080:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760783.368245:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65024000, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1f030 00010000:00010000:1.0:1789760783.368263:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5c4b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.368564:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760783.368571:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.368579:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5c4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65024000->65028095] (req 65024000->65028095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.368598:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5c4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65024000->65028095] (req 65024000->65028095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.375144:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983184 is committed 00002000:00010000:1.0:1789760783.375269:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65024000 -> 65028096 00002000:00010000:1.0:1789760783.375275:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127024 -> 127032 00010000:00010000:1.0:1789760783.375292:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5c4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65024000->65028095] (req 65024000->65028095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.375307:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5c4b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65024000->65028095] (req 65024000->65028095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.375317:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5c4b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65024000->65028095] (req 65024000->65028095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.375325:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760783.375331:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5c4b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65024000->65028095] (req 65024000->65028095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.375467:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5c4b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65024000->65028095] (req 65024000->65028095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.375492:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5c4b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65024000->65028095] (req 65024000->65028095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.375557:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5c4b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65024000->65028095] (req 65024000->65028095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.375724:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04562a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554798080:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8511us (8934us total) trans 4294983184 rc 0/0 00000100:00100000:1.0:1789760783.375737:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15886 00000100:00100000:1.0:1789760783.375749:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726597737648 time 726612807762 state 0x2 ref 0 00000100:00100000:2.0:1789760783.383512:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760783.384452:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554798208 00000100:00100000:1.0:1789760783.384469:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554798208 00000100:00100000:1.0:1789760783.384557:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726597737648 time 726612807762 state 0x0 ref 1 00000100:00100000:1.0:1789760783.384578:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726613068178 time 726612807762 state 0x1 ref 1 00000100:00100000:1.0:1789760783.384610:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726613068178 time 726630227631 state 0x0 ref 1 00000100:00100000:1.0:1789760783.384625:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04562300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554798208:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760783.384706:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65028096, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1f038 00010000:00010000:1.0:1789760783.384719:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5c52 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.384740:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760783.384745:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.384753:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5c52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65028096->65032191] (req 65028096->65032191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.384768:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5c52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65028096->65032191] (req 65028096->65032191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.395007:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983185 is committed 00002000:00010000:1.0:1789760783.395590:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65028096 -> 65032192 00002000:00010000:1.0:1789760783.395597:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127032 -> 127040 00010000:00010000:1.0:1789760783.395614:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5c52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65028096->65032191] (req 65028096->65032191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.395630:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5c52 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65028096->65032191] (req 65028096->65032191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.395641:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5c52 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65028096->65032191] (req 65028096->65032191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.395649:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760783.395655:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5c52 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65028096->65032191] (req 65028096->65032191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.395665:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5c52 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65028096->65032191] (req 65028096->65032191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.395702:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5c52 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65028096->65032191] (req 65028096->65032191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.395712:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c5c52 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65028096->65032191] (req 65028096->65032191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.395837:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04562300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554798208:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11208us (12334us total) trans 4294983185 rc 0/0 00000100:00100000:1.0:1789760783.395850:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15887 00000100:00100000:1.0:1789760783.395862:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726613068178 time 726630227631 state 0x2 ref 0 00000100:00100000:2.0:1789760783.399308:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760783.399410:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554798336 00000100:00100000:3.0:1789760783.399424:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554798336 00000100:00100000:3.0:1789760783.399498:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726613068178 time 726630227631 state 0x0 ref 1 00000100:00100000:3.0:1789760783.399587:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726630488047 time 726630227631 state 0x1 ref 1 00000100:00100000:3.0:1789760783.399623:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726630488047 time 726645240745 state 0x0 ref 1 00000100:00100000:3.0:1789760783.399637:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04560380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554798336:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760783.399709:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65032192, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1f040 00010000:00010000:3.0:1789760783.399721:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5c59 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.399741:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760783.399746:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.399754:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5c59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65032192->65036287] (req 65032192->65036287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.399771:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5c59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65032192->65036287] (req 65032192->65036287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.408513:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983186 is committed 00002000:00010000:2.0:1789760783.411302:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65032192 -> 65036288 00002000:00010000:2.0:1789760783.411317:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127040 -> 127048 00010000:00010000:2.0:1789760783.411335:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5c59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65032192->65036287] (req 65032192->65036287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.411351:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5c59 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65032192->65036287] (req 65032192->65036287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.411362:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5c59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65032192->65036287] (req 65032192->65036287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.411372:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760783.411419:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5c59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65032192->65036287] (req 65032192->65036287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.411431:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5c59 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65032192->65036287] (req 65032192->65036287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.411456:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5c59 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65032192->65036287] (req 65032192->65036287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.411464:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c5c59 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65032192->65036287] (req 65032192->65036287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.411612:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04560380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554798336:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11969us (12310us total) trans 4294983186 rc 0/0 00000100:00100000:2.0:1789760783.411626:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15888 00000100:00100000:2.0:1789760783.411638:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726630488047 time 726645240745 state 0x2 ref 0 00000100:00100000:2.0:1789760783.416424:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760783.416566:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554798464 00000100:00100000:0.0:1789760783.416578:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554798464 00000100:00100000:0.0:1789760783.416655:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726630488047 time 726645240745 state 0x0 ref 1 00000100:00100000:0.0:1789760783.416683:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726645501161 time 726645240745 state 0x1 ref 1 00000100:00100000:0.0:1789760783.416720:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726645501161 time 726662337580 state 0x0 ref 1 00000100:00100000:0.0:1789760783.416735:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04563b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554798464:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760783.416801:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65036288, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1f048 00010000:00010000:0.0:1789760783.416812:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5c60 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.416831:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760783.416836:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.416845:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5c60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65036288->65040383] (req 65036288->65040383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.416862:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5c60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65036288->65040383] (req 65036288->65040383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.425062:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983187 is committed 00002000:00010000:0.0:1789760783.425273:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65036288 -> 65040384 00002000:00010000:0.0:1789760783.425280:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127048 -> 127056 00010000:00010000:0.0:1789760783.425298:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5c60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65036288->65040383] (req 65036288->65040383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.425314:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5c60 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65036288->65040383] (req 65036288->65040383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.425325:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5c60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65036288->65040383] (req 65036288->65040383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.425333:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760783.425339:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5c60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65036288->65040383] (req 65036288->65040383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.425349:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5c60 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65036288->65040383] (req 65036288->65040383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.425374:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5c60 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65036288->65040383] (req 65036288->65040383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.425641:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5c60 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65036288->65040383] (req 65036288->65040383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.425776:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04563b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554798464:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9037us (9407us total) trans 4294983187 rc 0/0 00000100:00100000:0.0:1789760783.425789:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15889 00000100:00100000:0.0:1789760783.425801:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726645501161 time 726662337580 state 0x2 ref 0 00000100:00100000:1.0:1789760783.431316:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760783.431533:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554798592 00000100:00100000:2.0:1789760783.431547:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554798592 00000100:00100000:2.0:1789760783.431621:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726645501161 time 726662337580 state 0x0 ref 1 00000100:00100000:2.0:1789760783.431640:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726662597996 time 726662337580 state 0x1 ref 1 00000100:00100000:2.0:1789760783.431683:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726662597996 time 726677301266 state 0x0 ref 1 00000100:00100000:2.0:1789760783.431698:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d9c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554798592:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760783.431767:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65040384, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1f050 00010000:00010000:2.0:1789760783.431780:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5c67 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.431803:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760783.431809:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.431818:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5c67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65040384->65044479] (req 65040384->65044479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.431841:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5c67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65040384->65044479] (req 65040384->65044479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.438367:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983188 is committed 00002000:00010000:2.0:1789760783.438688:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65040384 -> 65044480 00002000:00010000:2.0:1789760783.438694:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127056 -> 127064 00010000:00010000:2.0:1789760783.438711:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5c67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65040384->65044479] (req 65040384->65044479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.438726:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5c67 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65040384->65044479] (req 65040384->65044479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.438738:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5c67 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65040384->65044479] (req 65040384->65044479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.438747:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760783.438753:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5c67 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65040384->65044479] (req 65040384->65044479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.438763:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5c67 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65040384->65044479] (req 65040384->65044479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.438799:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5c67 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65040384->65044479] (req 65040384->65044479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.438808:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5c67 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65040384->65044479] (req 65040384->65044479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.438959:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d9c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554798592:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7243us (7638us total) trans 4294983188 rc 0/0 00000100:00100000:2.0:1789760783.438973:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15890 00000100:00100000:2.0:1789760783.438991:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726662597996 time 726677301266 state 0x2 ref 0 00000100:00100000:2.0:1789760783.442206:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760783.442309:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554798720 00000100:00100000:1.0:1789760783.442322:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554798720 00000100:00100000:0.0:1789760783.443134:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726662597996 time 726677301266 state 0x0 ref 1 00000100:00100000:0.0:1789760783.443157:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726677561682 time 726677301266 state 0x1 ref 1 00000100:00100000:0.0:1789760783.443202:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726677561682 time 726688820129 state 0x0 ref 1 00000100:00100000:0.0:1789760783.443221:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04561880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554798720:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760783.443286:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65044480, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1f058 00010000:00010000:0.0:1789760783.443565:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5c6e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.443585:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760783.443590:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.443598:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5c6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65044480->65048575] (req 65044480->65048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.443619:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5c6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65044480->65048575] (req 65044480->65048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.453743:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983189 is committed 00002000:00010000:2.0:1789760783.453919:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65044480 -> 65048576 00002000:00010000:2.0:1789760783.453927:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127064 -> 127072 00010000:00010000:2.0:1789760783.453945:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5c6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65044480->65048575] (req 65044480->65048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.453962:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5c6e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65044480->65048575] (req 65044480->65048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.453972:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5c6e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65044480->65048575] (req 65044480->65048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.453980:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760783.453986:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5c6e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65044480->65048575] (req 65044480->65048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.453997:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5c6e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65044480->65048575] (req 65044480->65048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.454023:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5c6e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65044480->65048575] (req 65044480->65048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.454032:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5c6e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65044480->65048575] (req 65044480->65048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.454172:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04561880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554798720:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10951us (12051us total) trans 4294983189 rc 0/0 00000100:00100000:2.0:1789760783.454186:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15891 00000100:00100000:2.0:1789760783.454199:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726677561682 time 726688820129 state 0x2 ref 0 00000100:00100000:1.0:1789760783.459165:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760783.459231:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554798848 00000100:00100000:2.0:1789760783.459244:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554798848 00000100:00100000:2.0:1789760783.459319:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726677561682 time 726688820129 state 0x0 ref 1 00000100:00100000:2.0:1789760783.459339:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726689080545 time 726688820129 state 0x1 ref 1 00000100:00100000:2.0:1789760783.459400:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726689080545 time 726704995800 state 0x0 ref 1 00000100:00100000:2.0:1789760783.459416:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04546300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554798848:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760783.459497:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65048576, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1f060 00010000:00010000:2.0:1789760783.459509:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5c75 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.459530:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760783.459535:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.459543:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5c75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65048576->65052671] (req 65048576->65052671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.459568:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5c75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65048576->65052671] (req 65048576->65052671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.465556:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983190 is committed 00002000:00010000:2.0:1789760783.467018:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65048576 -> 65052672 00002000:00010000:2.0:1789760783.467031:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127072 -> 127080 00010000:00010000:2.0:1789760783.467065:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5c75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65048576->65052671] (req 65048576->65052671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.467081:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5c75 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65048576->65052671] (req 65048576->65052671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.467092:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5c75 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65048576->65052671] (req 65048576->65052671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.467106:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760783.467111:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5c75 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65048576->65052671] (req 65048576->65052671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.467120:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5c75 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65048576->65052671] (req 65048576->65052671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.467146:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5c75 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65048576->65052671] (req 65048576->65052671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.467154:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5c75 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65048576->65052671] (req 65048576->65052671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.467293:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04546300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554798848:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7872us (8135us total) trans 4294983190 rc 0/0 00000100:00100000:2.0:1789760783.467307:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15892 00000100:00100000:2.0:1789760783.467320:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726689080545 time 726704995800 state 0x2 ref 0 00000100:00100000:1.0:1789760783.470803:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760783.470924:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554798976 00000100:00100000:0.0:1789760783.470938:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554798976 00000100:00100000:0.0:1789760783.471015:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726689080545 time 726704995800 state 0x0 ref 1 00000100:00100000:0.0:1789760783.471033:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726705256216 time 726704995800 state 0x1 ref 1 00000100:00100000:0.0:1789760783.471067:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726705256216 time 726716685199 state 0x0 ref 1 00000100:00100000:0.0:1789760783.471082:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04544380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554798976:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760783.471149:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65052672, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1f068 00010000:00010000:0.0:1789760783.471161:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c7c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.471182:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760783.471187:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.471195:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65052672->65056767] (req 65052672->65056767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.471211:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65052672->65056767] (req 65052672->65056767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.480023:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983191 is committed 00002000:00010000:1.0:1789760783.480180:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65052672 -> 65056768 00002000:00010000:1.0:1789760783.480186:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127080 -> 127088 00010000:00010000:1.0:1789760783.480204:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65052672->65056767] (req 65052672->65056767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.480219:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c7c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65052672->65056767] (req 65052672->65056767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.480229:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65052672->65056767] (req 65052672->65056767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.480238:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760783.480243:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65052672->65056767] (req 65052672->65056767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.480254:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c7c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65052672->65056767] (req 65052672->65056767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.480279:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c7c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65052672->65056767] (req 65052672->65056767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.480287:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c7c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65052672->65056767] (req 65052672->65056767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.480480:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04544380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554798976:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9393us (9685us total) trans 4294983191 rc 0/0 00000100:00100000:1.0:1789760783.480493:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15893 00000100:00100000:1.0:1789760783.480506:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726705256216 time 726716685199 state 0x2 ref 0 00000100:00100000:1.0:1789760783.483135:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760783.483238:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554799104 00000100:00100000:2.0:1789760783.483251:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554799104 00000100:00100000:2.0:1789760783.483323:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726705256216 time 726716685199 state 0x0 ref 1 00000100:00100000:2.0:1789760783.483341:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726716945615 time 726716685199 state 0x1 ref 1 00000100:00100000:2.0:1789760783.483375:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726716945615 time 726728993220 state 0x0 ref 1 00000100:00100000:3.0:1789760783.484125:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04546680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554799104:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760783.484199:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65056768, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1f070 00010000:00010000:3.0:1789760783.484212:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5c83 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.484234:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760783.484240:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.484247:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5c83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65056768->65060863] (req 65056768->65060863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.484264:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5c83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65056768->65060863] (req 65056768->65060863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.490230:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983192 is committed 00002000:00010000:3.0:1789760783.490374:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65056768 -> 65060864 00002000:00010000:3.0:1789760783.490402:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127088 -> 127096 00010000:00010000:3.0:1789760783.490419:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5c83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65056768->65060863] (req 65056768->65060863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.490436:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5c83 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65056768->65060863] (req 65056768->65060863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.490446:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5c83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65056768->65060863] (req 65056768->65060863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.490455:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760783.490461:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5c83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65056768->65060863] (req 65056768->65060863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.490471:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5c83 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65056768->65060863] (req 65056768->65060863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.490496:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5c83 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65056768->65060863] (req 65056768->65060863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.490505:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5c83 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65056768->65060863] (req 65056768->65060863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.490631:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04546680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554799104:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6785us (7501us total) trans 4294983192 rc 0/0 00000100:00100000:3.0:1789760783.490643:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15894 00000100:00100000:3.0:1789760783.490655:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726716945615 time 726728993220 state 0x2 ref 0 00000100:00100000:2.0:1789760783.494359:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760783.494903:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554799232 00000100:00100000:0.0:1789760783.494917:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554799232 00000100:00100000:0.0:1789760783.494990:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726716945615 time 726728993220 state 0x0 ref 1 00000100:00100000:0.0:1789760783.495008:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726729253636 time 726728993220 state 0x1 ref 1 00000100:00100000:0.0:1789760783.495043:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726729253636 time 726740662033 state 0x0 ref 1 00000100:00100000:0.0:1789760783.495201:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04f86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554799232:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760783.495270:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65060864, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1f078 00010000:00010000:0.0:1789760783.495283:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c8a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.495303:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760783.495308:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.495316:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65060864->65064959] (req 65060864->65064959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.495343:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65060864->65064959] (req 65060864->65064959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.501102:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983193 is committed 00002000:00010000:0.0:1789760783.501354:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65060864 -> 65064960 00002000:00010000:0.0:1789760783.501360:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127096 -> 127104 00010000:00010000:0.0:1789760783.501402:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65060864->65064959] (req 65060864->65064959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.501419:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c8a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65060864->65064959] (req 65060864->65064959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.501429:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65060864->65064959] (req 65060864->65064959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.501437:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760783.501443:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65060864->65064959] (req 65060864->65064959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.501453:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c8a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65060864->65064959] (req 65060864->65064959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.501478:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c8a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65060864->65064959] (req 65060864->65064959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.501487:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c8a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65060864->65064959] (req 65060864->65064959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.501614:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04f86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554799232:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6552us (7261us total) trans 4294983193 rc 0/0 00000100:00100000:0.0:1789760783.501629:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15895 00000100:00100000:0.0:1789760783.501642:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726729253636 time 726740662033 state 0x2 ref 0 00000100:00100000:1.0:1789760783.507912:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760783.508012:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554799360 00000100:00100000:2.0:1789760783.508024:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554799360 00000100:00100000:2.0:1789760783.508108:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726729253636 time 726740662033 state 0x0 ref 1 00000100:00100000:2.0:1789760783.508127:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726740922449 time 726740662033 state 0x1 ref 1 00000100:00100000:2.0:1789760783.508172:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726740922449 time 726753789562 state 0x0 ref 1 00000100:00100000:2.0:1789760783.508187:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04546a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554799360:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760783.508253:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65064960, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1f080 00010000:00010000:2.0:1789760783.508265:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5c91 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.508286:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760783.508292:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.508301:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5c91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65064960->65069055] (req 65064960->65069055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.508319:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5c91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65064960->65069055] (req 65064960->65069055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.518628:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983194 is committed 00002000:00010000:2.0:1789760783.519250:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65064960 -> 65069056 00002000:00010000:2.0:1789760783.519258:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127104 -> 127112 00010000:00010000:2.0:1789760783.519274:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5c91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65064960->65069055] (req 65064960->65069055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.519289:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5c91 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65064960->65069055] (req 65064960->65069055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.519300:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5c91 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65064960->65069055] (req 65064960->65069055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.519308:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760783.519313:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5c91 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65064960->65069055] (req 65064960->65069055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.519322:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5c91 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65064960->65069055] (req 65064960->65069055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.519363:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5c91 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65064960->65069055] (req 65064960->65069055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.519371:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5c91 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65064960->65069055] (req 65064960->65069055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.520011:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04546a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554799360:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11817us (12104us total) trans 4294983194 rc 0/0 00000100:00100000:2.0:1789760783.520025:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15896 00000100:00100000:2.0:1789760783.520036:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726740922449 time 726753789562 state 0x2 ref 0 00000100:00100000:1.0:1789760783.523345:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760783.523694:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554799488 00000100:00100000:0.0:1789760783.523710:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554799488 00000100:00100000:0.0:1789760783.523786:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726740922449 time 726753789562 state 0x0 ref 1 00000100:00100000:0.0:1789760783.523805:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726754049978 time 726753789562 state 0x1 ref 1 00000100:00100000:0.0:1789760783.523845:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726754049978 time 726769462561 state 0x0 ref 1 00000100:00100000:0.0:1789760783.523862:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04547480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554799488:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760783.523933:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65069056, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1f088 00010000:00010000:0.0:1789760783.523945:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c98 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.523965:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760783.523971:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.523979:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65069056->65073151] (req 65069056->65073151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.523998:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65069056->65073151] (req 65069056->65073151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.534301:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983195 is committed 00002000:00010000:0.0:1789760783.534485:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65069056 -> 65073152 00002000:00010000:0.0:1789760783.534492:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127112 -> 127120 00010000:00010000:0.0:1789760783.534509:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65069056->65073151] (req 65069056->65073151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.534524:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c98 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65069056->65073151] (req 65069056->65073151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.534540:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65069056->65073151] (req 65069056->65073151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.534549:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760783.534555:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65069056->65073151] (req 65069056->65073151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.534572:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c98 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65069056->65073151] (req 65069056->65073151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.534600:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c98 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65069056->65073151] (req 65069056->65073151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.534609:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5c98 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65069056->65073151] (req 65069056->65073151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.534755:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04547480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554799488:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10890us (11423us total) trans 4294983195 rc 0/0 00000100:00100000:0.0:1789760783.534768:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15897 00000100:00100000:0.0:1789760783.534779:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726754049978 time 726769462561 state 0x2 ref 0 00000100:00100000:1.0:1789760783.537941:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760783.538048:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554799616 00000100:00100000:3.0:1789760783.538067:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554799616 00000100:00100000:3.0:1789760783.538146:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726754049978 time 726769462561 state 0x0 ref 1 00000100:00100000:3.0:1789760783.538166:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726769722977 time 726769462561 state 0x1 ref 1 00000100:00100000:3.0:1789760783.538203:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726769722977 time 726783821752 state 0x0 ref 1 00000100:00100000:3.0:1789760783.538220:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04547800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554799616:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760783.538282:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65073152, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1f090 00010000:00010000:3.0:1789760783.538294:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5c9f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.538316:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760783.538321:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.538329:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5c9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65073152->65077247] (req 65073152->65077247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.538347:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5c9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65073152->65077247] (req 65073152->65077247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.543880:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983196 is committed 00002000:00010000:3.0:1789760783.544067:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65073152 -> 65077248 00002000:00010000:3.0:1789760783.544074:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127120 -> 127128 00010000:00010000:3.0:1789760783.544091:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5c9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65073152->65077247] (req 65073152->65077247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.544119:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5c9f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65073152->65077247] (req 65073152->65077247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.544130:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5c9f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65073152->65077247] (req 65073152->65077247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.544137:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760783.544144:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5c9f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65073152->65077247] (req 65073152->65077247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.544154:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5c9f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65073152->65077247] (req 65073152->65077247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.544177:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5c9f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65073152->65077247] (req 65073152->65077247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.544306:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5c9f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65073152->65077247] (req 65073152->65077247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.544514:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04547800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554799616:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6288us (6579us total) trans 4294983196 rc 0/0 00000100:00100000:3.0:1789760783.544527:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15898 00000100:00100000:3.0:1789760783.544540:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726769722977 time 726783821752 state 0x2 ref 0 00000100:00100000:1.0:1789760783.548936:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760783.549063:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554799744 00000100:00100000:2.0:1789760783.549076:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554799744 00000100:00100000:2.0:1789760783.549156:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726769722977 time 726783821752 state 0x0 ref 1 00000100:00100000:2.0:1789760783.549175:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726784082168 time 726783821752 state 0x1 ref 1 00000100:00100000:2.0:1789760783.549212:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726784082168 time 726794829312 state 0x0 ref 1 00000100:00100000:2.0:1789760783.549227:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04545500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554799744:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760783.549296:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65077248, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1f098 00010000:00010000:2.0:1789760783.549307:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5ca6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.549334:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760783.549340:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.549348:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5ca6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65077248->65081343] (req 65077248->65081343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.549366:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5ca6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65077248->65081343] (req 65077248->65081343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.554937:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983197 is committed 00002000:00010000:0.0:1789760783.555281:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65077248 -> 65081344 00002000:00010000:0.0:1789760783.555288:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127128 -> 127136 00010000:00010000:0.0:1789760783.555305:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5ca6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65077248->65081343] (req 65077248->65081343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.555326:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5ca6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65077248->65081343] (req 65077248->65081343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.555337:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5ca6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65077248->65081343] (req 65077248->65081343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.555346:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760783.555351:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5ca6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65077248->65081343] (req 65077248->65081343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.555361:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5ca6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65077248->65081343] (req 65077248->65081343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.555421:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5ca6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65077248->65081343] (req 65077248->65081343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.555430:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5ca6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65077248->65081343] (req 65077248->65081343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.555540:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04545500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554799744:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6311us (6614us total) trans 4294983197 rc 0/0 00000100:00100000:0.0:1789760783.555552:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15899 00000100:00100000:0.0:1789760783.555564:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726784082168 time 726794829312 state 0x2 ref 0 00000100:00100000:1.0:1789760783.559081:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760783.559163:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554799872 00000100:00100000:2.0:1789760783.559176:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554799872 00000100:00100000:2.0:1789760783.559250:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726784082168 time 726794829312 state 0x0 ref 1 00000100:00100000:2.0:1789760783.559269:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726795089728 time 726794829312 state 0x1 ref 1 00000100:00100000:2.0:1789760783.559304:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726795089728 time 726804921737 state 0x0 ref 1 00000100:00100000:2.0:1789760783.559330:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04547100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554799872:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760783.559441:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65081344, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1f0a0 00010000:00010000:2.0:1789760783.559454:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5cad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.559473:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760783.559478:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.559485:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5cad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65081344->65085439] (req 65081344->65085439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.559503:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5cad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65081344->65085439] (req 65081344->65085439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.564360:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983198 is committed 00002000:00010000:2.0:1789760783.564692:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65081344 -> 65085440 00002000:00010000:2.0:1789760783.564698:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127136 -> 127144 00010000:00010000:2.0:1789760783.564777:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5cad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65081344->65085439] (req 65081344->65085439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.564794:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5cad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65081344->65085439] (req 65081344->65085439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.564805:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5cad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65081344->65085439] (req 65081344->65085439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.564814:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760783.564819:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5cad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65081344->65085439] (req 65081344->65085439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.564830:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5cad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65081344->65085439] (req 65081344->65085439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.564856:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5cad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65081344->65085439] (req 65081344->65085439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.564865:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5cad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65081344->65085439] (req 65081344->65085439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.565123:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04547100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554799872:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5732us (5981us total) trans 4294983198 rc 0/0 00000100:00100000:2.0:1789760783.565137:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15900 00000100:00100000:2.0:1789760783.565149:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726795089728 time 726804921737 state 0x2 ref 0 00000100:00100000:1.0:1789760783.570242:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760783.573528:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554800000 00000100:00100000:0.0:1789760783.573551:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554800000 00000100:00100000:0.0:1789760783.573653:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726795089728 time 726804921737 state 0x0 ref 1 00000100:00100000:0.0:1789760783.574368:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726805182153 time 726804921737 state 0x1 ref 1 00000100:00100000:0.0:1789760783.575230:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726805182153 time 726820841879 state 0x0 ref 1 00000100:00100000:0.0:1789760783.575260:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04545f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554800000:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760783.575332:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65085440, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1f0a8 00010000:00010000:0.0:1789760783.575345:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5cb4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.575365:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760783.575370:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.575416:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5cb4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65085440->65089535] (req 65085440->65089535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.575435:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5cb4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65085440->65089535] (req 65085440->65089535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.581971:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983199 is committed 00002000:00010000:1.0:1789760783.582301:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65085440 -> 65089536 00002000:00010000:1.0:1789760783.582307:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127144 -> 127152 00010000:00010000:1.0:1789760783.582324:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5cb4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65085440->65089535] (req 65085440->65089535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.582337:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5cb4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65085440->65089535] (req 65085440->65089535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.582347:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5cb4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65085440->65089535] (req 65085440->65089535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.582354:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760783.582360:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5cb4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65085440->65089535] (req 65085440->65089535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.582369:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5cb4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65085440->65089535] (req 65085440->65089535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.582443:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5cb4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65085440->65089535] (req 65085440->65089535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.582453:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c5cb4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65085440->65089535] (req 65085440->65089535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.582598:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04545f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554800000:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7342us (12361us total) trans 4294983199 rc 0/0 00000100:00100000:1.0:1789760783.582612:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15901 00000100:00100000:1.0:1789760783.582626:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726805182153 time 726820841879 state 0x2 ref 0 00000100:00100000:1.0:1789760783.585944:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760783.586071:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554800128 00000100:00100000:3.0:1789760783.586084:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554800128 00000100:00100000:3.0:1789760783.586173:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726805182153 time 726820841879 state 0x0 ref 1 00000100:00100000:3.0:1789760783.586196:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726821102295 time 726820841879 state 0x1 ref 1 00000100:00100000:3.0:1789760783.586231:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726821102295 time 726831849655 state 0x0 ref 1 00000100:00100000:3.0:1789760783.586249:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04544a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554800128:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760783.586318:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65089536, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1f0b0 00010000:00010000:3.0:1789760783.586332:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5cbb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.586352:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760783.586358:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.586366:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5cbb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65089536->65093631] (req 65089536->65093631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.586438:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5cbb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65089536->65093631] (req 65089536->65093631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.595791:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983200 is committed 00002000:00010000:3.0:1789760783.600548:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65089536 -> 65093632 00002000:00010000:3.0:1789760783.600560:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127152 -> 127160 00010000:00010000:3.0:1789760783.600574:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5cbb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65089536->65093631] (req 65089536->65093631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.600587:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5cbb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65089536->65093631] (req 65089536->65093631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.600597:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5cbb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65089536->65093631] (req 65089536->65093631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.600604:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760783.600610:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5cbb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65089536->65093631] (req 65089536->65093631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.600619:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5cbb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65089536->65093631] (req 65089536->65093631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.600644:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5cbb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65089536->65093631] (req 65089536->65093631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.600908:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5cbb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65089536->65093631] (req 65089536->65093631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.601030:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04544a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554800128:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14778us (15097us total) trans 4294983200 rc 0/0 00000100:00100000:3.0:1789760783.601042:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15902 00000100:00100000:3.0:1789760783.601055:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726821102295 time 726831849655 state 0x2 ref 0 00000100:00100000:2.0:1789760783.610710:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760783.613332:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554800256 00000100:00100000:0.0:1789760783.613358:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554800256 00000100:00100000:0.0:1789760783.613471:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726821102295 time 726831849655 state 0x0 ref 1 00000100:00100000:0.0:1789760783.613491:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726832110071 time 726831849655 state 0x1 ref 1 00000100:00100000:0.0:1789760783.613524:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726832110071 time 726859141722 state 0x0 ref 1 00000100:00100000:0.0:1789760783.613541:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554800256:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760783.613618:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65093632, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1f0b8 00010000:00010000:0.0:1789760783.613630:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5cc2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.613652:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:0.0:1789760783.613657:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.613665:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5cc2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65093632->65097727] (req 65093632->65097727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.613692:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5cc2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65093632->65097727] (req 65093632->65097727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.616175:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760783.616235:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760783.616269:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760783.616307:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554800384 00000100:00100000:3.0:1789760783.616320:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554800384 00000100:00100000:3.0:1789760783.616421:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 472 00000100:00100000:3.0:1789760783.616434:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc320b7c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9765eb0b-55b9-40b6-bfdf-ffee156c3226+13:2411:x1876699554800384:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:1.0:1789760783.616453:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760783.616540:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc320b7c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9765eb0b-55b9-40b6-bfdf-ffee156c3226+13:2411:x1876699554800384:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 108us (303us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760783.616549:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 472 00000100:00100000:2.0:1789760783.623089:0:7470:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554800512 00000100:00100000:3.0:1789760783.623089:0:6521:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554800640 00000100:00100000:3.0:1789760783.623112:0:6521:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554800640 00000100:00100000:2.0:1789760783.623153:0:7470:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554800512 00000100:00100000:3.0:1789760783.623215:0:6521:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 885 00000100:00100000:3.0:1789760783.623228:0:6521:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04547b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554800640:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:3.0:1789760783.623372:0:6521:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04547b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554800640:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 145us (7100us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760783.623402:0:6521:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 885 00000100:00100000:2.0:1789760783.623432:0:7470:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554800896 00000100:00100000:2.0:1789760783.623438:0:7470:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554800896 00000100:00080000:3.0:1789760783.623458:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789760783.623465:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:2.0:1789760783.623470:0:7470:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 886 00000100:00100000:2.0:1789760783.623480:0:7470:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb04544000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+8:2412:x1876699554800512:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:3.0:1789760783.623485:0:6522:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 887 00000100:00100000:3.0:1789760783.623497:0:6522:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb064ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2411:x1876699554800896:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:2.0:1789760783.623521:0:7470:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb04544000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+8:2412:x1876699554800512:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 42us (7360us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760783.623527:0:7470:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 886 00000100:00100000:3.0:1789760783.623550:0:6522:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb064ece00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2411:x1876699554800896:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 55us (7097us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760783.623558:0:6522:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 887 00000100:00100000:2.0:1789760783.625632:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000001:00080000:3.0:1789760783.625904:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983201 is committed 00002000:00010000:2.0:1789760783.626063:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65093632 -> 65097728 00002000:00010000:2.0:1789760783.626068:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127160 -> 127168 00010000:00010000:2.0:1789760783.626087:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5cc2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65093632->65097727] (req 65093632->65097727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.626101:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5cc2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65093632->65097727] (req 65093632->65097727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.626117:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5cc2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65093632->65097727] (req 65093632->65097727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.626126:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760783.626132:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5cc2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65093632->65097727] (req 65093632->65097727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.626144:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5cc2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65093632->65097727] (req 65093632->65097727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.626169:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5cc2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65093632->65097727] (req 65093632->65097727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.626173:0:7470:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554800768 00010000:00010000:2.0:1789760783.626181:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c5cc2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65093632->65097727] (req 65093632->65097727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.626240:0:7470:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554800768 00000100:00100000:2.0:1789760783.626287:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554800256:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12748us (15590us total) trans 4294983201 rc 0/0 00000100:00100000:2.0:1789760783.626302:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15903 00000100:00100000:2.0:1789760783.626316:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726832110071 time 726859141722 state 0x2 ref 0 00000100:00100000:1.0:1789760783.626420:0:7470:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 888 00000100:00100000:1.0:1789760783.626432:0:7470:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f84a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2412:x1876699554800768:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:1.0:1789760783.626484:0:7470:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f84a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2412:x1876699554800768:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 54us (867us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760783.626493:0:7470:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 888 00000100:00080000:3.0:1789760783.626861:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789760783.635858:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760783.635956:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554801024 00000100:00100000:1.0:1789760783.635969:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554801024 00000100:00100000:1.0:1789760783.636118:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726832110071 time 726859141722 state 0x0 ref 1 00000100:00100000:1.0:1789760783.636138:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726859402138 time 726859141722 state 0x1 ref 1 00000100:00100000:1.0:1789760783.636175:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726859402138 time 726881793484 state 0x0 ref 1 00000100:00100000:1.0:1789760783.636190:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554801024:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760783.636250:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65097728, mtime: 0x6aad950e, atime: 0x0, ctime: 0x6aad950e, blocks: 0x1f0c0 00010000:00010000:1.0:1789760783.636265:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5cc9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.636286:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760783.636292:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.636300:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5cc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65097728->65101823] (req 65097728->65101823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.636317:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5cc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65097728->65101823] (req 65097728->65101823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.643819:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983202 is committed 00002000:00010000:1.0:1789760783.643952:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65097728 -> 65101824 00002000:00010000:1.0:1789760783.643957:0:8441:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760782 -> 1789760783 00002000:00010000:1.0:1789760783.643960:0:8441:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760782 -> 1789760783 00002000:00010000:1.0:1789760783.643963:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127168 -> 127176 00010000:00010000:1.0:1789760783.643979:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5cc9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65097728->65101823] (req 65097728->65101823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.643993:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5cc9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65097728->65101823] (req 65097728->65101823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.644003:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5cc9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65097728->65101823] (req 65097728->65101823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.644011:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760783.644026:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5cc9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65097728->65101823] (req 65097728->65101823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.644042:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5cc9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65097728->65101823] (req 65097728->65101823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.644071:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5cc9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65097728->65101823] (req 65097728->65101823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.644079:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5cc9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65097728->65101823] (req 65097728->65101823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.644189:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554801024:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7995us (8338us total) trans 4294983202 rc 0/0 00000100:00100000:1.0:1789760783.644200:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15904 00000100:00100000:1.0:1789760783.644211:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726859402138 time 726881793484 state 0x2 ref 0 00000100:00100000:3.0:1789760783.648308:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760783.648414:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554801152 00000100:00100000:1.0:1789760783.648428:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554801152 00000100:00100000:1.0:1789760783.648499:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726859402138 time 726881793484 state 0x0 ref 1 00000100:00100000:1.0:1789760783.648518:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726882053900 time 726881793484 state 0x1 ref 1 00000100:00100000:1.0:1789760783.648557:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726882053900 time 726894175186 state 0x0 ref 1 00000100:00100000:1.0:1789760783.648570:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06493800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554801152:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760783.648636:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65101824, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f0c8 00010000:00010000:1.0:1789760783.648648:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5cd0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.648680:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760783.648686:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.648694:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5cd0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65101824->65105919] (req 65101824->65105919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.648710:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5cd0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65101824->65105919] (req 65101824->65105919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.654446:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983203 is committed 00002000:00010000:1.0:1789760783.654775:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65101824 -> 65105920 00002000:00010000:1.0:1789760783.654782:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127176 -> 127184 00010000:00010000:1.0:1789760783.654801:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5cd0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65101824->65105919] (req 65101824->65105919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.654818:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5cd0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65101824->65105919] (req 65101824->65105919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.654829:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5cd0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65101824->65105919] (req 65101824->65105919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.654838:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760783.654844:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5cd0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65101824->65105919] (req 65101824->65105919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.654855:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5cd0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65101824->65105919] (req 65101824->65105919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.654890:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5cd0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65101824->65105919] (req 65101824->65105919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.654899:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5cd0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65101824->65105919] (req 65101824->65105919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.655029:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06493800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554801152:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6453us (6728us total) trans 4294983203 rc 0/0 00000100:00100000:1.0:1789760783.655040:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15905 00000100:00100000:1.0:1789760783.655062:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726882053900 time 726894175186 state 0x2 ref 0 00000100:00100000:2.0:1789760783.667288:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760783.667511:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554801280 00000100:00100000:3.0:1789760783.667524:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554801280 00000100:00100000:3.0:1789760783.667594:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726882053900 time 726894175186 state 0x0 ref 1 00000100:00100000:3.0:1789760783.667613:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726894435602 time 726894175186 state 0x1 ref 1 00000100:00100000:3.0:1789760783.667650:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726894435602 time 726913267936 state 0x0 ref 1 00000100:00100000:3.0:1789760783.667664:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554801280:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760783.667737:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65105920, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f0d0 00010000:00010000:3.0:1789760783.667749:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5cd7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.667777:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760783.667783:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.667792:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5cd7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65105920->65110015] (req 65105920->65110015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.667808:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5cd7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65105920->65110015] (req 65105920->65110015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.674928:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983204 is committed 00002000:00010000:3.0:1789760783.679523:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65105920 -> 65110016 00002000:00010000:3.0:1789760783.679533:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127184 -> 127192 00010000:00010000:3.0:1789760783.679548:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5cd7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65105920->65110015] (req 65105920->65110015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.679559:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5cd7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65105920->65110015] (req 65105920->65110015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.679567:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5cd7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65105920->65110015] (req 65105920->65110015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.679575:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760783.679579:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5cd7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65105920->65110015] (req 65105920->65110015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.679988:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5cd7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65105920->65110015] (req 65105920->65110015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.680009:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5cd7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65105920->65110015] (req 65105920->65110015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.680022:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5cd7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65105920->65110015] (req 65105920->65110015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.680138:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554801280:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12472us (12863us total) trans 4294983204 rc 0/0 00000100:00100000:3.0:1789760783.680150:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15906 00000100:00100000:3.0:1789760783.680159:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726894435602 time 726913267936 state 0x2 ref 0 00000100:00100000:2.0:1789760783.683231:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760783.683307:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554801408 00000100:00100000:3.0:1789760783.683320:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554801408 00000100:00100000:3.0:1789760783.683418:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726894435602 time 726913267936 state 0x0 ref 1 00000100:00100000:3.0:1789760783.683434:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726913528352 time 726913267936 state 0x1 ref 1 00000100:00100000:3.0:1789760783.683460:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726913528352 time 726929078963 state 0x0 ref 1 00000100:00100000:3.0:1789760783.683515:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554801408:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760783.683559:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65110016, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f0d8 00010000:00010000:3.0:1789760783.683568:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5cde lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.683583:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:3.0:1789760783.683587:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.683593:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5cde lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65110016->65114111] (req 65110016->65114111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.683612:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5cde lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65110016->65114111] (req 65110016->65114111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.692102:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983205 is committed 00002000:00010000:3.0:1789760783.692557:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65110016 -> 65114112 00002000:00010000:3.0:1789760783.692565:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127192 -> 127200 00010000:00010000:3.0:1789760783.692578:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5cde lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65110016->65114111] (req 65110016->65114111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.692591:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5cde lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65110016->65114111] (req 65110016->65114111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.692806:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5cde lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65110016->65114111] (req 65110016->65114111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.692815:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760783.692820:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5cde lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65110016->65114111] (req 65110016->65114111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.692829:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5cde lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65110016->65114111] (req 65110016->65114111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.692853:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5cde lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65110016->65114111] (req 65110016->65114111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.692861:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5cde lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65110016->65114111] (req 65110016->65114111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.692978:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554801408:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9460us (9756us total) trans 4294983205 rc 0/0 00000100:00100000:3.0:1789760783.692999:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15907 00000100:00100000:3.0:1789760783.693010:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726913528352 time 726929078963 state 0x2 ref 0 00000100:00100000:1.0:1789760783.704177:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760783.704281:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554801536 00000100:00100000:2.0:1789760783.704301:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554801536 00000100:00100000:2.0:1789760783.704436:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726913528352 time 726929078963 state 0x0 ref 1 00000100:00100000:2.0:1789760783.704456:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726929339379 time 726929078963 state 0x1 ref 1 00000100:00100000:2.0:1789760783.704491:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726929339379 time 726950109937 state 0x0 ref 1 00000100:00100000:2.0:1789760783.704508:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb064edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554801536:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760783.704573:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65114112, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f0e0 00010000:00010000:2.0:1789760783.704585:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5ce5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.704604:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760783.704609:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.704617:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5ce5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65114112->65118207] (req 65114112->65118207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.704633:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5ce5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65114112->65118207] (req 65114112->65118207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.715621:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983206 is committed 00002000:00010000:0.0:1789760783.715763:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65114112 -> 65118208 00002000:00010000:0.0:1789760783.715768:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127200 -> 127208 00010000:00010000:0.0:1789760783.715786:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5ce5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65114112->65118207] (req 65114112->65118207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.715803:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5ce5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65114112->65118207] (req 65114112->65118207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.715814:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5ce5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65114112->65118207] (req 65114112->65118207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.715823:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760783.715830:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5ce5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65114112->65118207] (req 65114112->65118207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.715841:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5ce5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65114112->65118207] (req 65114112->65118207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.715866:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5ce5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65114112->65118207] (req 65114112->65118207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.715874:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5ce5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65114112->65118207] (req 65114112->65118207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.716038:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb064edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554801536:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11526us (11863us total) trans 4294983206 rc 0/0 00000100:00100000:0.0:1789760783.716051:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15908 00000100:00100000:0.0:1789760783.716062:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726929339379 time 726950109937 state 0x2 ref 0 00000100:00100000:2.0:1789760783.719958:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760783.720073:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554801664 00000100:00100000:1.0:1789760783.720086:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554801664 00000100:00100000:1.0:1789760783.720169:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726929339379 time 726950109937 state 0x0 ref 1 00000100:00100000:1.0:1789760783.720188:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726950370353 time 726950109937 state 0x1 ref 1 00000100:00100000:1.0:1789760783.720224:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726950370353 time 726965841646 state 0x0 ref 1 00000100:00100000:1.0:1789760783.720237:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554801664:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760783.720297:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65118208, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f0e8 00010000:00010000:1.0:1789760783.720308:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5cec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.720333:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760783.720338:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.720346:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5cec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65118208->65122303] (req 65118208->65122303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.720361:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5cec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65118208->65122303] (req 65118208->65122303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760783.725980:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983207 is committed 00002000:00010000:1.0:1789760783.726313:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65118208 -> 65122304 00002000:00010000:1.0:1789760783.726319:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127208 -> 127216 00010000:00010000:1.0:1789760783.726337:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5cec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65118208->65122303] (req 65118208->65122303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.726354:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5cec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65118208->65122303] (req 65118208->65122303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.726367:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5cec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65118208->65122303] (req 65118208->65122303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.726393:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760783.726401:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5cec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65118208->65122303] (req 65118208->65122303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.726412:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5cec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65118208->65122303] (req 65118208->65122303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.726512:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5cec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65118208->65122303] (req 65118208->65122303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.726522:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5cec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65118208->65122303] (req 65118208->65122303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.726704:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554801664:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6462us (6756us total) trans 4294983207 rc 0/0 00000100:00100000:1.0:1789760783.726717:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15909 00000100:00100000:1.0:1789760783.726730:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726950370353 time 726965841646 state 0x2 ref 0 00000100:00100000:3.0:1789760783.729839:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760783.730041:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554801792 00000100:00100000:1.0:1789760783.730062:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554801792 00000100:00100000:1.0:1789760783.730147:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726950370353 time 726965841646 state 0x0 ref 1 00000100:00100000:1.0:1789760783.730166:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726966102062 time 726965841646 state 0x1 ref 1 00000100:00100000:1.0:1789760783.730201:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726966102062 time 726975818911 state 0x0 ref 1 00000100:00100000:1.0:1789760783.730215:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06490a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554801792:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760783.730280:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65122304, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f0f0 00010000:00010000:1.0:1789760783.730290:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5cf3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.730310:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:1.0:1789760783.730316:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.730323:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5cf3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65122304->65126399] (req 65122304->65126399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.730340:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5cf3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65122304->65126399] (req 65122304->65126399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.738223:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983208 is committed 00002000:00010000:1.0:1789760783.741398:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65122304 -> 65126400 00002000:00010000:1.0:1789760783.741412:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127216 -> 127224 00010000:00010000:1.0:1789760783.741498:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5cf3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65122304->65126399] (req 65122304->65126399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.741515:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5cf3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65122304->65126399] (req 65122304->65126399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.741526:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5cf3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65122304->65126399] (req 65122304->65126399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.741534:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760783.741540:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5cf3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65122304->65126399] (req 65122304->65126399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.741551:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5cf3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65122304->65126399] (req 65122304->65126399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.741579:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5cf3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65122304->65126399] (req 65122304->65126399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.741589:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5cf3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65122304->65126399] (req 65122304->65126399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.741972:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06490a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554801792:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11750us (12142us total) trans 4294983208 rc 0/0 00000100:00100000:1.0:1789760783.741987:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15910 00000100:00100000:1.0:1789760783.742000:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726966102062 time 726975818911 state 0x2 ref 0 00000100:00100000:3.0:1789760783.746201:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760783.746345:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554801920 00000100:00100000:2.0:1789760783.746358:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554801920 00000100:00100000:2.0:1789760783.746658:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726966102062 time 726975818911 state 0x0 ref 1 00000100:00100000:2.0:1789760783.746692:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726976079327 time 726975818911 state 0x1 ref 1 00000100:00100000:2.0:1789760783.746810:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726976079327 time 726992426977 state 0x0 ref 1 00000100:00100000:2.0:1789760783.746826:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06491c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554801920:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760783.746889:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65126400, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f0f8 00010000:00010000:2.0:1789760783.746900:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5cfa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.746922:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=726s v=(0 0) 00010000:00010000:2.0:1789760783.746927:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.746934:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5cfa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65126400->65130495] (req 65126400->65130495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.746952:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5cfa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65126400->65130495] (req 65126400->65130495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.755345:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983209 is committed 00002000:00010000:2.0:1789760783.755637:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65126400 -> 65130496 00002000:00010000:2.0:1789760783.755643:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127224 -> 127232 00010000:00010000:2.0:1789760783.755661:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5cfa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65126400->65130495] (req 65126400->65130495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.755685:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5cfa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65126400->65130495] (req 65126400->65130495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.755696:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5cfa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65126400->65130495] (req 65126400->65130495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.755705:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760783.755711:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5cfa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65126400->65130495] (req 65126400->65130495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.755722:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5cfa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65126400->65130495] (req 65126400->65130495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.755749:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5cfa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65126400->65130495] (req 65126400->65130495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.755759:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5cfa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65126400->65130495] (req 65126400->65130495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.755942:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06491c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554801920:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9112us (9750us total) trans 4294983209 rc 0/0 00000100:00100000:2.0:1789760783.755954:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15911 00000100:00100000:2.0:1789760783.755967:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726976079327 time 726992426977 state 0x2 ref 0 00000100:00100000:2.0:1789760783.759995:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760783.760194:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554802048 00000100:00100000:0.0:1789760783.760206:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554802048 00000100:00100000:0.0:1789760783.760277:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726976079327 time 726992426977 state 0x0 ref 1 00000100:00100000:0.0:1789760783.760294:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726992687393 time 726992426977 state 0x1 ref 1 00000100:00100000:0.0:1789760783.760361:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726992687393 time 727005978572 state 0x0 ref 1 00000100:00100000:0.0:1789760783.760417:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe1b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554802048:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760783.760487:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65130496, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f100 00010000:00010000:0.0:1789760783.760499:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5d01 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.760518:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760783.760523:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.760531:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5d01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65130496->65134591] (req 65130496->65134591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.760547:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5d01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65130496->65134591] (req 65130496->65134591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.770573:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983210 is committed 00002000:00010000:1.0:1789760783.771236:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65130496 -> 65134592 00002000:00010000:1.0:1789760783.771242:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127232 -> 127240 00010000:00010000:1.0:1789760783.771260:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5d01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65130496->65134591] (req 65130496->65134591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.771276:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5d01 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65130496->65134591] (req 65130496->65134591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.771285:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5d01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65130496->65134591] (req 65130496->65134591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.771293:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760783.771299:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5d01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65130496->65134591] (req 65130496->65134591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.771310:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5d01 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65130496->65134591] (req 65130496->65134591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.771337:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5d01 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65130496->65134591] (req 65130496->65134591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.771346:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c5d01 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65130496->65134591] (req 65130496->65134591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.772266:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe1b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554802048:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11882us (12275us total) trans 4294983210 rc 0/0 00000100:00100000:1.0:1789760783.772280:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15912 00000100:00100000:1.0:1789760783.772293:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726992687393 time 727005978572 state 0x2 ref 0 00000100:00100000:3.0:1789760783.776197:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760783.776323:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554802176 00000100:00100000:0.0:1789760783.776335:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554802176 00000100:00100000:0.0:1789760783.776433:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 726992687393 time 727005978572 state 0x0 ref 1 00000100:00100000:0.0:1789760783.776453:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727006238988 time 727005978572 state 0x1 ref 1 00000100:00100000:0.0:1789760783.776488:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727006238988 time 727022105809 state 0x0 ref 1 00000100:00100000:0.0:1789760783.776503:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06490000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554802176:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760783.776567:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65134592, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f108 00010000:00010000:0.0:1789760783.776579:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d08 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.776599:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760783.776604:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.776612:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65134592->65138687] (req 65134592->65138687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.776629:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65134592->65138687] (req 65134592->65138687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.785790:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983211 is committed 00002000:00010000:0.0:1789760783.786246:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65134592 -> 65138688 00002000:00010000:0.0:1789760783.786254:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127240 -> 127248 00010000:00010000:0.0:1789760783.786272:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65134592->65138687] (req 65134592->65138687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.786287:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d08 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65134592->65138687] (req 65134592->65138687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.786298:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65134592->65138687] (req 65134592->65138687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.786307:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760783.786313:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65134592->65138687] (req 65134592->65138687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.786322:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d08 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65134592->65138687] (req 65134592->65138687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.789933:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d08 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65134592->65138687] (req 65134592->65138687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.789960:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d08 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65134592->65138687] (req 65134592->65138687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.790488:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06490000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554802176:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13979us (14296us total) trans 4294983211 rc 0/0 00000100:00100000:0.0:1789760783.790504:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15913 00000100:00100000:0.0:1789760783.790516:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727006238988 time 727022105809 state 0x2 ref 0 00000100:00100000:3.0:1789760783.797460:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760783.797660:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554802304 00000100:00100000:1.0:1789760783.797682:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554802304 00000100:00100000:1.0:1789760783.797753:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727006238988 time 727022105809 state 0x0 ref 1 00000100:00100000:1.0:1789760783.797773:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727022366225 time 727022105809 state 0x1 ref 1 00000100:00100000:1.0:1789760783.797810:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727022366225 time 727043427646 state 0x0 ref 1 00000100:00100000:1.0:1789760783.797825:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554802304:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760783.797893:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65138688, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f110 00010000:00010000:1.0:1789760783.797905:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5d0f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.797932:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760783.797938:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.797945:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5d0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65138688->65142783] (req 65138688->65142783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.797962:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5d0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65138688->65142783] (req 65138688->65142783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760783.803755:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983212 is committed 00002000:00010000:2.0:1789760783.803942:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65138688 -> 65142784 00002000:00010000:2.0:1789760783.803948:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127248 -> 127256 00010000:00010000:2.0:1789760783.803966:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5d0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65138688->65142783] (req 65138688->65142783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.803987:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5d0f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65138688->65142783] (req 65138688->65142783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.803996:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5d0f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65138688->65142783] (req 65138688->65142783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.804003:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760783.804008:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5d0f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65138688->65142783] (req 65138688->65142783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.804018:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5d0f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65138688->65142783] (req 65138688->65142783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.804152:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5d0f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65138688->65142783] (req 65138688->65142783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.804162:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5d0f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65138688->65142783] (req 65138688->65142783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.804321:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06490380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554802304:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6492us (6868us total) trans 4294983212 rc 0/0 00000100:00100000:2.0:1789760783.804334:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15914 00000100:00100000:2.0:1789760783.804348:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727022366225 time 727043427646 state 0x2 ref 0 00000100:00100000:1.0:1789760783.812788:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760783.813093:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554802432 00000100:00100000:3.0:1789760783.813107:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554802432 00000100:00100000:3.0:1789760783.813181:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727022366225 time 727043427646 state 0x0 ref 1 00000100:00100000:3.0:1789760783.813200:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727043688062 time 727043427646 state 0x1 ref 1 00000100:00100000:3.0:1789760783.813239:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727043688062 time 727058857618 state 0x0 ref 1 00000100:00100000:3.0:1789760783.813254:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb064edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554802432:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760783.813331:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65142784, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f118 00010000:00010000:3.0:1789760783.813345:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d16 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.813367:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760783.813374:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.813430:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65142784->65146879] (req 65142784->65146879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.813448:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65142784->65146879] (req 65142784->65146879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.822861:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983213 is committed 00002000:00010000:3.0:1789760783.823270:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65142784 -> 65146880 00002000:00010000:3.0:1789760783.823279:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127256 -> 127264 00010000:00010000:3.0:1789760783.823297:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65142784->65146879] (req 65142784->65146879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.823312:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d16 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65142784->65146879] (req 65142784->65146879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.823322:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65142784->65146879] (req 65142784->65146879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.823329:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760783.823334:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65142784->65146879] (req 65142784->65146879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.823344:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d16 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65142784->65146879] (req 65142784->65146879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.823368:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d16 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65142784->65146879] (req 65142784->65146879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.823426:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d16 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65142784->65146879] (req 65142784->65146879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.823606:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb064edf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554802432:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10347us (10824us total) trans 4294983213 rc 0/0 00000100:00100000:3.0:1789760783.823619:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15915 00000100:00100000:3.0:1789760783.823633:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727043688062 time 727058857618 state 0x2 ref 0 00000100:00100000:3.0:1789760783.836530:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760783.836751:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554802560 00000100:00100000:0.0:1789760783.836764:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554802560 00000100:00100000:0.0:1789760783.837021:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727043688062 time 727058857618 state 0x0 ref 1 00000100:00100000:0.0:1789760783.837040:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727059118034 time 727058857618 state 0x1 ref 1 00000100:00100000:0.0:1789760783.837082:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727059118034 time 727082700004 state 0x0 ref 1 00000100:00100000:0.0:1789760783.837110:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06492680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554802560:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760783.837174:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65146880, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f120 00010000:00010000:0.0:1789760783.837187:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d1d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.837208:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760783.837213:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.837221:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65146880->65150975] (req 65146880->65150975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.837238:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65146880->65150975] (req 65146880->65150975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.847984:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983214 is committed 00002000:00010000:0.0:1789760783.848154:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65146880 -> 65150976 00002000:00010000:0.0:1789760783.848160:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127264 -> 127272 00010000:00010000:0.0:1789760783.848178:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65146880->65150975] (req 65146880->65150975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.848194:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d1d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65146880->65150975] (req 65146880->65150975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.848204:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65146880->65150975] (req 65146880->65150975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.848213:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760783.848219:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65146880->65150975] (req 65146880->65150975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.848229:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d1d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65146880->65150975] (req 65146880->65150975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.848255:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d1d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65146880->65150975] (req 65146880->65150975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.848265:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d1d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65146880->65150975] (req 65146880->65150975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760783.848443:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06492680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554802560:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11331us (11921us total) trans 4294983214 rc 0/0 00000100:00100000:0.0:1789760783.848458:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15916 00000100:00100000:0.0:1789760783.848470:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727059118034 time 727082700004 state 0x2 ref 0 00000100:00100000:1.0:1789760783.864282:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760783.866995:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554802688 00000100:00100000:0.0:1789760783.867021:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554802688 00000100:00100000:0.0:1789760783.867117:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727059118034 time 727082700004 state 0x0 ref 1 00000100:00100000:0.0:1789760783.867137:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727082960420 time 727082700004 state 0x1 ref 1 00000100:00100000:0.0:1789760783.867181:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727082960420 time 727112799094 state 0x0 ref 1 00000100:00100000:0.0:1789760783.867196:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb064edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554802688:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760783.867278:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65150976, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f128 00010000:00010000:0.0:1789760783.867291:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d24 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.867312:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760783.867317:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.867325:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65150976->65155071] (req 65150976->65155071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.867342:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65150976->65155071] (req 65150976->65155071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.884437:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983215 is committed 00002000:00010000:1.0:1789760783.884685:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65150976 -> 65155072 00002000:00010000:1.0:1789760783.884692:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127272 -> 127280 00010000:00010000:1.0:1789760783.884711:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65150976->65155071] (req 65150976->65155071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.884727:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d24 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65150976->65155071] (req 65150976->65155071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.884738:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65150976->65155071] (req 65150976->65155071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.884747:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760783.884753:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65150976->65155071] (req 65150976->65155071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.884769:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d24 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65150976->65155071] (req 65150976->65155071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.884794:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d24 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65150976->65155071] (req 65150976->65155071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.884803:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c5d24 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65150976->65155071] (req 65150976->65155071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.884926:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb064edc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554802688:12345-192.168.203.22@tcp:4:dd.0 Request processed in 17726us (20659us total) trans 4294983215 rc 0/0 00000100:00100000:1.0:1789760783.884938:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15917 00000100:00100000:1.0:1789760783.884951:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727082960420 time 727112799094 state 0x2 ref 0 00000100:00100000:1.0:1789760783.890283:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760783.890357:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554802816 00000100:00100000:2.0:1789760783.890370:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554802816 00000100:00100000:2.0:1789760783.890478:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727082960420 time 727112799094 state 0x0 ref 1 00000100:00100000:2.0:1789760783.890498:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727113059510 time 727112799094 state 0x1 ref 1 00000100:00100000:2.0:1789760783.890524:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727113059510 time 727136143090 state 0x0 ref 1 00000100:00100000:2.0:1789760783.890538:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb064ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554802816:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760783.890764:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65155072, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f130 00010000:00010000:2.0:1789760783.890777:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d2b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.890798:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760783.890803:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.890811:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65155072->65159167] (req 65155072->65159167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.890830:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65155072->65159167] (req 65155072->65159167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.899813:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983216 is committed 00002000:00010000:2.0:1789760783.906356:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65155072 -> 65159168 00002000:00010000:2.0:1789760783.906371:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127280 -> 127288 00010000:00010000:2.0:1789760783.906404:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65155072->65159167] (req 65155072->65159167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.906426:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d2b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65155072->65159167] (req 65155072->65159167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.906436:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65155072->65159167] (req 65155072->65159167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.906445:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760783.906451:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65155072->65159167] (req 65155072->65159167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.906462:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d2b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65155072->65159167] (req 65155072->65159167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.906488:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d2b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65155072->65159167] (req 65155072->65159167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.906497:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d2b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65155072->65159167] (req 65155072->65159167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.906636:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb064ef800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554802816:12345-192.168.203.22@tcp:4:dd.0 Request processed in 16092us (16358us total) trans 4294983216 rc 0/0 00000100:00100000:2.0:1789760783.906649:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15918 00000100:00100000:2.0:1789760783.906661:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727113059510 time 727136143090 state 0x2 ref 0 00000100:00100000:3.0:1789760783.909887:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760783.910421:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554802944 00000100:00100000:2.0:1789760783.910436:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554802944 00000100:00100000:2.0:1789760783.910514:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727113059510 time 727136143090 state 0x0 ref 1 00000100:00100000:2.0:1789760783.910535:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727136403506 time 727136143090 state 0x1 ref 1 00000100:00100000:2.0:1789760783.910574:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727136403506 time 727156191727 state 0x0 ref 1 00000100:00100000:2.0:1789760783.910589:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554802944:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760783.910653:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65159168, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f138 00010000:00010000:2.0:1789760783.910665:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5d32 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.910698:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760783.910704:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.910712:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5d32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65159168->65163263] (req 65159168->65163263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.910728:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5d32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65159168->65163263] (req 65159168->65163263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.919014:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983217 is committed 00002000:00010000:3.0:1789760783.919231:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65159168 -> 65163264 00002000:00010000:3.0:1789760783.919238:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127288 -> 127296 00010000:00010000:3.0:1789760783.919255:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5d32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65159168->65163263] (req 65159168->65163263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.919270:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5d32 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65159168->65163263] (req 65159168->65163263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.919280:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5d32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65159168->65163263] (req 65159168->65163263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.919288:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760783.919294:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5d32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65159168->65163263] (req 65159168->65163263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.919308:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5d32 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65159168->65163263] (req 65159168->65163263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.919332:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5d32 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65159168->65163263] (req 65159168->65163263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.919341:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5d32 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65159168->65163263] (req 65159168->65163263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.919627:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06491500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554802944:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9033us (9746us total) trans 4294983217 rc 0/0 00000100:00100000:3.0:1789760783.919641:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15919 00000100:00100000:3.0:1789760783.919655:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727136403506 time 727156191727 state 0x2 ref 0 00000100:00100000:1.0:1789760783.931893:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760783.932010:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554803072 00000100:00100000:0.0:1789760783.932032:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554803072 00000100:00100000:0.0:1789760783.932116:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727136403506 time 727156191727 state 0x0 ref 1 00000100:00100000:0.0:1789760783.932136:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727156452143 time 727156191727 state 0x1 ref 1 00000100:00100000:0.0:1789760783.932174:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727156452143 time 727177791567 state 0x0 ref 1 00000100:00100000:0.0:1789760783.932189:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb064ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554803072:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760783.932255:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65163264, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f140 00010000:00010000:0.0:1789760783.932266:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5d39 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.932286:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760783.932292:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760783.932299:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5d39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65163264->65167359] (req 65163264->65167359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760783.932315:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5d39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65163264->65167359] (req 65163264->65167359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.941831:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983218 is committed 00002000:00010000:1.0:1789760783.942279:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65163264 -> 65167360 00002000:00010000:1.0:1789760783.942285:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127296 -> 127304 00010000:00010000:1.0:1789760783.942304:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5d39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65163264->65167359] (req 65163264->65167359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.942320:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5d39 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65163264->65167359] (req 65163264->65167359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.942329:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5d39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65163264->65167359] (req 65163264->65167359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.942338:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760783.942344:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5d39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65163264->65167359] (req 65163264->65167359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.942353:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5d39 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65163264->65167359] (req 65163264->65167359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.942409:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5d39 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65163264->65167359] (req 65163264->65167359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.942418:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c5d39 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65163264->65167359] (req 65163264->65167359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760783.942557:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb064ee300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554803072:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10363us (10671us total) trans 4294983218 rc 0/0 00000100:00100000:1.0:1789760783.942570:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15920 00000100:00100000:1.0:1789760783.942582:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727156452143 time 727177791567 state 0x2 ref 0 00000100:00100000:1.0:1789760783.945850:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760783.945966:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554803200 00000100:00100000:2.0:1789760783.945980:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554803200 00000100:00100000:2.0:1789760783.946065:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727156452143 time 727177791567 state 0x0 ref 1 00000100:00100000:2.0:1789760783.946083:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727178051983 time 727177791567 state 0x1 ref 1 00000100:00100000:2.0:1789760783.946118:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727178051983 time 727191736533 state 0x0 ref 1 00000100:00100000:2.0:1789760783.946136:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb064ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554803200:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760783.946198:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65167360, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f148 00010000:00010000:2.0:1789760783.946208:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d40 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.946228:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760783.946233:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.946241:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65167360->65171455] (req 65167360->65171455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.946257:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65167360->65171455] (req 65167360->65171455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.953544:0:8453:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789760783.955970:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983219 is committed 00002000:00010000:2.0:1789760783.956234:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65167360 -> 65171456 00002000:00010000:2.0:1789760783.956240:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127304 -> 127312 00010000:00010000:2.0:1789760783.956257:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65167360->65171455] (req 65167360->65171455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.956274:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d40 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65167360->65171455] (req 65167360->65171455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.956285:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65167360->65171455] (req 65167360->65171455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.956293:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760783.956300:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65167360->65171455] (req 65167360->65171455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.956310:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d40 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65167360->65171455] (req 65167360->65171455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.956335:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d40 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65167360->65171455] (req 65167360->65171455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.956344:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d40 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65167360->65171455] (req 65167360->65171455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.956639:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb064ee680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554803200:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10496us (10793us total) trans 4294983219 rc 0/0 00000100:00100000:2.0:1789760783.956652:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15921 00000100:00100000:2.0:1789760783.956664:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727178051983 time 727191736533 state 0x2 ref 0 00000100:00100000:3.0:1789760783.967278:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760783.967881:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554803328 00000100:00100000:2.0:1789760783.967895:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554803328 00000100:00100000:2.0:1789760783.967973:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727178051983 time 727191736533 state 0x0 ref 1 00000100:00100000:2.0:1789760783.967994:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727191996949 time 727191736533 state 0x1 ref 1 00000100:00100000:2.0:1789760783.968053:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727191996949 time 727213661509 state 0x0 ref 1 00000100:00100000:2.0:1789760783.968068:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06491180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554803328:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760783.968134:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65171456, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f150 00010000:00010000:2.0:1789760783.968146:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d47 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.968166:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760783.968172:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760783.968179:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65171456->65175551] (req 65171456->65175551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.968197:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65171456->65175551] (req 65171456->65175551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760783.975995:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983220 is committed 00002000:00010000:2.0:1789760783.976237:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65171456 -> 65175552 00002000:00010000:2.0:1789760783.976244:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127312 -> 127320 00010000:00010000:2.0:1789760783.976263:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65171456->65175551] (req 65171456->65175551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.976280:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d47 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65171456->65175551] (req 65171456->65175551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.976291:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d47 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65171456->65175551] (req 65171456->65175551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.976300:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760783.976306:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d47 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65171456->65175551] (req 65171456->65175551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.976315:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d47 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65171456->65175551] (req 65171456->65175551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.976340:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d47 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65171456->65175551] (req 65171456->65175551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760783.976348:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5d47 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65171456->65175551] (req 65171456->65175551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760783.976526:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06491180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554803328:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8455us (9257us total) trans 4294983220 rc 0/0 00000100:00100000:2.0:1789760783.976539:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15922 00000100:00100000:2.0:1789760783.976551:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727191996949 time 727213661509 state 0x2 ref 0 00000100:00100000:1.0:1789760783.979764:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760783.980239:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554803456 00000100:00100000:3.0:1789760783.980251:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554803456 00000100:00100000:3.0:1789760783.980325:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727191996949 time 727213661509 state 0x0 ref 1 00000100:00100000:3.0:1789760783.980344:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727213921925 time 727213661509 state 0x1 ref 1 00000100:00100000:3.0:1789760783.980430:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727213921925 time 727226039921 state 0x0 ref 1 00000100:00100000:3.0:1789760783.980445:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb064eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554803456:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760783.980509:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65175552, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f158 00010000:00010000:3.0:1789760783.980521:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d4e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.980542:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760783.980548:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760783.980556:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65175552->65179647] (req 65175552->65179647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.980572:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65175552->65179647] (req 65175552->65179647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760783.989433:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983221 is committed 00002000:00010000:3.0:1789760783.989620:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65175552 -> 65179648 00002000:00010000:3.0:1789760783.989627:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127320 -> 127328 00010000:00010000:3.0:1789760783.989643:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65175552->65179647] (req 65175552->65179647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.989658:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d4e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65175552->65179647] (req 65175552->65179647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.989909:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65175552->65179647] (req 65175552->65179647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.989921:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760783.989928:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65175552->65179647] (req 65175552->65179647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.989945:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d4e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65175552->65179647] (req 65175552->65179647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.989971:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d4e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65175552->65179647] (req 65175552->65179647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760783.989980:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d4e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65175552->65179647] (req 65175552->65179647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760783.990234:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb064eed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554803456:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9785us (10477us total) trans 4294983221 rc 0/0 00000100:00100000:3.0:1789760783.990248:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15923 00000100:00100000:3.0:1789760783.990260:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727213921925 time 727226039921 state 0x2 ref 0 00000100:00100000:1.0:1789760783.998298:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760783.998435:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554803584 00000100:00100000:0.0:1789760783.998449:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554803584 00000100:00100000:0.0:1789760783.998521:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727213921925 time 727226039921 state 0x0 ref 1 00000100:00100000:0.0:1789760783.998544:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727226300337 time 727226039921 state 0x1 ref 1 00000100:00100000:0.0:1789760783.998743:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727226300337 time 727244198321 state 0x0 ref 1 00000100:00100000:1.0:1789760783.998816:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554803584:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760783.998885:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65179648, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f160 00010000:00010000:1.0:1789760783.998896:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d55 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.998918:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760783.998924:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760783.998932:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65179648->65183743] (req 65179648->65183743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760783.998948:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65179648->65183743] (req 65179648->65183743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.006890:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983222 is committed 00002000:00010000:1.0:1789760784.007094:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65179648 -> 65183744 00002000:00010000:1.0:1789760784.007101:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127328 -> 127336 00010000:00010000:1.0:1789760784.007122:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65179648->65183743] (req 65179648->65183743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.007138:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d55 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65179648->65183743] (req 65179648->65183743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.007148:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65179648->65183743] (req 65179648->65183743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.007156:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.007162:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65179648->65183743] (req 65179648->65183743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.007171:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d55 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65179648->65183743] (req 65179648->65183743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.007201:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d55 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65179648->65183743] (req 65179648->65183743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.007209:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d55 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65179648->65183743] (req 65179648->65183743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.007353:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554803584:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8588us (9062us total) trans 4294983222 rc 0/0 00000100:00100000:1.0:1789760784.007367:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15924 00000100:00100000:1.0:1789760784.007891:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727226300337 time 727244198321 state 0x2 ref 0 00000100:00100000:2.0:1789760784.016360:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760784.020552:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554803712 00000100:00100000:1.0:1789760784.020576:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554803712 00000100:00100000:1.0:1789760784.020662:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727226300337 time 727244198321 state 0x0 ref 1 00000100:00100000:1.0:1789760784.020695:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727244458737 time 727244198321 state 0x1 ref 1 00000100:00100000:1.0:1789760784.020811:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727244458737 time 727266426792 state 0x0 ref 1 00000100:00100000:1.0:1789760784.020829:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554803712:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760784.020968:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65183744, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f168 00010000:00010000:1.0:1789760784.020980:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d5c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.021002:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760784.021009:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.021016:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65183744->65187839] (req 65183744->65187839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.021035:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65183744->65187839] (req 65183744->65187839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.029182:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983223 is committed 00002000:00010000:1.0:1789760784.029448:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65183744 -> 65187840 00002000:00010000:1.0:1789760784.029453:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127336 -> 127344 00010000:00010000:1.0:1789760784.029470:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65183744->65187839] (req 65183744->65187839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.029486:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d5c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65183744->65187839] (req 65183744->65187839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.029496:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65183744->65187839] (req 65183744->65187839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.029505:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.029511:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65183744->65187839] (req 65183744->65187839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.029521:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d5c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65183744->65187839] (req 65183744->65187839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.029545:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d5c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65183744->65187839] (req 65183744->65187839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.029558:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d5c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65183744->65187839] (req 65183744->65187839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.029706:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554803712:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8874us (13357us total) trans 4294983223 rc 0/0 00000100:00100000:1.0:1789760784.029719:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15925 00000100:00100000:1.0:1789760784.029731:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727244458737 time 727266426792 state 0x2 ref 0 00000100:00100000:3.0:1789760784.032808:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760784.033036:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554803840 00000100:00100000:2.0:1789760784.033055:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554803840 00000100:00100000:2.0:1789760784.033123:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727244458737 time 727266426792 state 0x0 ref 1 00000100:00100000:2.0:1789760784.033139:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727266687208 time 727266426792 state 0x1 ref 1 00000100:00100000:2.0:1789760784.033169:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727266687208 time 727278787777 state 0x0 ref 1 00000100:00100000:2.0:1789760784.033181:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554803840:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760784.033254:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65187840, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f170 00010000:00010000:2.0:1789760784.033266:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5d63 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.033288:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760784.033294:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.033301:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5d63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65187840->65191935] (req 65187840->65191935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.033317:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5d63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65187840->65191935] (req 65187840->65191935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.039970:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983224 is committed 00002000:00010000:1.0:1789760784.040341:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65187840 -> 65191936 00002000:00010000:1.0:1789760784.040347:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127344 -> 127352 00010000:00010000:1.0:1789760784.040365:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5d63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65187840->65191935] (req 65187840->65191935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.040406:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5d63 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65187840->65191935] (req 65187840->65191935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.040417:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5d63 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65187840->65191935] (req 65187840->65191935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.040423:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.040428:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5d63 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65187840->65191935] (req 65187840->65191935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.040436:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5d63 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65187840->65191935] (req 65187840->65191935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.040462:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5d63 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65187840->65191935] (req 65187840->65191935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.040468:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5d63 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65187840->65191935] (req 65187840->65191935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.040575:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06493480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554803840:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7388us (7772us total) trans 4294983224 rc 0/0 00000100:00100000:1.0:1789760784.040658:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15926 00000100:00100000:1.0:1789760784.040681:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727266687208 time 727278787777 state 0x2 ref 0 00000100:00100000:2.0:1789760784.044461:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760784.044656:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554803968 00000100:00100000:3.0:1789760784.044677:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554803968 00000100:00100000:3.0:1789760784.044747:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727266687208 time 727278787777 state 0x0 ref 1 00000100:00100000:3.0:1789760784.044766:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727279048193 time 727278787777 state 0x1 ref 1 00000100:00100000:3.0:1789760784.044805:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727279048193 time 727290422759 state 0x0 ref 1 00000100:00100000:3.0:1789760784.044817:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554803968:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760784.044879:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65191936, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f178 00010000:00010000:3.0:1789760784.044891:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d6a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.044911:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760784.044916:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.044923:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65191936->65196031] (req 65191936->65196031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.044940:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65191936->65196031] (req 65191936->65196031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.052764:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983225 is committed 00002000:00010000:3.0:1789760784.052960:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65191936 -> 65196032 00002000:00010000:3.0:1789760784.052966:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127352 -> 127360 00010000:00010000:3.0:1789760784.052985:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65191936->65196031] (req 65191936->65196031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.053000:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d6a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65191936->65196031] (req 65191936->65196031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.053010:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d6a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65191936->65196031] (req 65191936->65196031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.053018:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760784.053024:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d6a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65191936->65196031] (req 65191936->65196031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.053035:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d6a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65191936->65196031] (req 65191936->65196031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.053072:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d6a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65191936->65196031] (req 65191936->65196031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.053081:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5d6a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65191936->65196031] (req 65191936->65196031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.053232:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554803968:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8401us (8771us total) trans 4294983225 rc 0/0 00000100:00100000:3.0:1789760784.053245:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15927 00000100:00100000:3.0:1789760784.053257:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727279048193 time 727290422759 state 0x2 ref 0 00000100:00100000:2.0:1789760784.056518:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760784.056653:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554804096 00000100:00100000:1.0:1789760784.056677:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554804096 00000100:00100000:1.0:1789760784.056761:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727279048193 time 727290422759 state 0x0 ref 1 00000100:00100000:1.0:1789760784.056780:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727290683175 time 727290422759 state 0x1 ref 1 00000100:00100000:1.0:1789760784.056816:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727290683175 time 727302433959 state 0x0 ref 1 00000100:00100000:1.0:1789760784.056830:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554804096:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760784.056904:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65196032, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f180 00010000:00010000:1.0:1789760784.056917:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d71 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.056938:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760784.056944:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.056954:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65196032->65200127] (req 65196032->65200127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.056970:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65196032->65200127] (req 65196032->65200127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.064013:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983226 is committed 00002000:00010000:1.0:1789760784.064211:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65196032 -> 65200128 00002000:00010000:1.0:1789760784.064217:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127360 -> 127368 00010000:00010000:1.0:1789760784.064236:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65196032->65200127] (req 65196032->65200127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.064251:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d71 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65196032->65200127] (req 65196032->65200127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.064261:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65196032->65200127] (req 65196032->65200127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.064269:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.064275:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65196032->65200127] (req 65196032->65200127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.064284:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d71 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65196032->65200127] (req 65196032->65200127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.064309:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d71 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65196032->65200127] (req 65196032->65200127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.064324:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d71 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65196032->65200127] (req 65196032->65200127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.064520:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f86300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554804096:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7685us (8142us total) trans 4294983226 rc 0/0 00000100:00100000:1.0:1789760784.064533:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15928 00000100:00100000:1.0:1789760784.064545:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727290683175 time 727302433959 state 0x2 ref 0 00000100:00100000:1.0:1789760784.068511:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760784.068872:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554804224 00000100:00100000:2.0:1789760784.068885:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554804224 00000100:00100000:2.0:1789760784.068958:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727290683175 time 727302433959 state 0x0 ref 1 00000100:00100000:2.0:1789760784.068979:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727302694375 time 727302433959 state 0x1 ref 1 00000100:00100000:2.0:1789760784.069012:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727302694375 time 727314630843 state 0x0 ref 1 00000100:00100000:2.0:1789760784.069025:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554804224:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760784.069109:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65200128, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f188 00010000:00010000:2.0:1789760784.069120:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5d78 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.069148:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760784.069154:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.069162:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5d78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65200128->65204223] (req 65200128->65204223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.069179:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5d78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65200128->65204223] (req 65200128->65204223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.078803:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983227 is committed 00002000:00010000:2.0:1789760784.079269:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65200128 -> 65204224 00002000:00010000:2.0:1789760784.079276:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127368 -> 127376 00010000:00010000:2.0:1789760784.079295:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5d78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65200128->65204223] (req 65200128->65204223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.079311:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5d78 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65200128->65204223] (req 65200128->65204223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.079322:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5d78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65200128->65204223] (req 65200128->65204223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.079331:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760784.079337:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5d78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65200128->65204223] (req 65200128->65204223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.079448:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5d78 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65200128->65204223] (req 65200128->65204223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.079480:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5d78 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65200128->65204223] (req 65200128->65204223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.079489:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5d78 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65200128->65204223] (req 65200128->65204223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.079753:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554804224:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10721us (11250us total) trans 4294983227 rc 0/0 00000100:00100000:2.0:1789760784.079766:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15929 00000100:00100000:2.0:1789760784.079779:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727302694375 time 727314630843 state 0x2 ref 0 00000100:00100000:2.0:1789760784.083076:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760784.083258:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554804352 00000100:00100000:1.0:1789760784.083271:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554804352 00000100:00100000:1.0:1789760784.083540:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727302694375 time 727314630843 state 0x0 ref 1 00000100:00100000:1.0:1789760784.083561:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727314891259 time 727314630843 state 0x1 ref 1 00000100:00100000:1.0:1789760784.083731:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727314891259 time 727329347288 state 0x0 ref 1 00000100:00100000:1.0:1789760784.083747:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554804352:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760784.083810:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65204224, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f190 00010000:00010000:1.0:1789760784.083821:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d7f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.083840:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760784.083845:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.083854:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65204224->65208319] (req 65204224->65208319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.083871:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65204224->65208319] (req 65204224->65208319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.092257:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983228 is committed 00002000:00010000:1.0:1789760784.092595:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65204224 -> 65208320 00002000:00010000:1.0:1789760784.092602:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127376 -> 127384 00010000:00010000:1.0:1789760784.092621:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65204224->65208319] (req 65204224->65208319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.092637:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d7f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65204224->65208319] (req 65204224->65208319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.092648:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d7f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65204224->65208319] (req 65204224->65208319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.092656:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.092662:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d7f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65204224->65208319] (req 65204224->65208319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.092685:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d7f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65204224->65208319] (req 65204224->65208319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.092777:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d7f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65204224->65208319] (req 65204224->65208319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.092793:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d7f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65204224->65208319] (req 65204224->65208319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.092959:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554804352:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9208us (9892us total) trans 4294983228 rc 0/0 00000100:00100000:1.0:1789760784.093035:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15930 00000100:00100000:1.0:1789760784.093048:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727314891259 time 727329347288 state 0x2 ref 0 00000100:00100000:2.0:1789760784.101919:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760784.102293:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554804480 00000100:00100000:0.0:1789760784.102313:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554804480 00000100:00100000:0.0:1789760784.102896:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727314891259 time 727329347288 state 0x0 ref 1 00000100:00100000:0.0:1789760784.102917:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727329607704 time 727329347288 state 0x1 ref 1 00000100:00100000:0.0:1789760784.103195:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727329607704 time 727348811128 state 0x0 ref 1 00000100:00100000:0.0:1789760784.103215:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f84000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554804480:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760784.103287:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65208320, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f198 00010000:00010000:0.0:1789760784.103299:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5d86 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.103318:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760784.103323:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.103331:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5d86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65208320->65212415] (req 65208320->65212415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.103349:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5d86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65208320->65212415] (req 65208320->65212415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.112011:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983229 is committed 00002000:00010000:0.0:1789760784.112209:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65208320 -> 65212416 00002000:00010000:0.0:1789760784.112215:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127384 -> 127392 00010000:00010000:0.0:1789760784.112234:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5d86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65208320->65212415] (req 65208320->65212415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.112249:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5d86 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65208320->65212415] (req 65208320->65212415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.112259:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5d86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65208320->65212415] (req 65208320->65212415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.112267:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760784.112272:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5d86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65208320->65212415] (req 65208320->65212415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.112283:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5d86 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65208320->65212415] (req 65208320->65212415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.112783:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5d86 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65208320->65212415] (req 65208320->65212415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.112798:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c5d86 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65208320->65212415] (req 65208320->65212415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.112989:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f84000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554804480:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9773us (11077us total) trans 4294983229 rc 0/0 00000100:00100000:0.0:1789760784.113004:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15931 00000100:00100000:0.0:1789760784.113016:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727329607704 time 727348811128 state 0x2 ref 0 00000100:00100000:2.0:1789760784.117280:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760784.117485:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554804608 00000100:00100000:1.0:1789760784.117498:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554804608 00000100:00100000:1.0:1789760784.117571:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727329607704 time 727348811128 state 0x0 ref 1 00000100:00100000:1.0:1789760784.117592:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727349071544 time 727348811128 state 0x1 ref 1 00000100:00100000:1.0:1789760784.117636:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727349071544 time 727363253724 state 0x0 ref 1 00000100:00100000:1.0:1789760784.117651:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554804608:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760784.117727:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65212416, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f1a0 00010000:00010000:1.0:1789760784.117739:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d8d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.117760:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760784.117766:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.117774:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65212416->65216511] (req 65212416->65216511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.117794:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65212416->65216511] (req 65212416->65216511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.126994:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983230 is committed 00002000:00010000:2.0:1789760784.131609:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65212416 -> 65216512 00002000:00010000:2.0:1789760784.131622:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127392 -> 127400 00010000:00010000:2.0:1789760784.131641:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65212416->65216511] (req 65212416->65216511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.131655:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d8d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65212416->65216511] (req 65212416->65216511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.131664:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65212416->65216511] (req 65212416->65216511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.131688:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760784.131694:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65212416->65216511] (req 65212416->65216511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.131704:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d8d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65212416->65216511] (req 65212416->65216511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.131809:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d8d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65212416->65216511] (req 65212416->65216511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.131819:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c5d8d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65212416->65216511] (req 65212416->65216511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.131965:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554804608:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14308us (14691us total) trans 4294983230 rc 0/0 00000100:00100000:2.0:1789760784.131979:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15932 00000100:00100000:2.0:1789760784.131991:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727349071544 time 727363253724 state 0x2 ref 0 00000100:00100000:2.0:1789760784.136280:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760784.136412:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554804736 00000100:00100000:1.0:1789760784.136432:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554804736 00000100:00100000:1.0:1789760784.136499:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727349071544 time 727363253724 state 0x0 ref 1 00000100:00100000:1.0:1789760784.136517:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727363514140 time 727363253724 state 0x1 ref 1 00000100:00100000:1.0:1789760784.136548:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727363514140 time 727382166872 state 0x0 ref 1 00000100:00100000:1.0:1789760784.136560:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554804736:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760784.136619:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65216512, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f1a8 00010000:00010000:1.0:1789760784.136630:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d94 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.136650:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760784.136655:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.136661:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65216512->65220607] (req 65216512->65220607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.136689:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65216512->65220607] (req 65216512->65220607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.143884:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983231 is committed 00002000:00010000:1.0:1789760784.144292:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65216512 -> 65220608 00002000:00010000:1.0:1789760784.144300:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127400 -> 127408 00010000:00010000:1.0:1789760784.144319:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65216512->65220607] (req 65216512->65220607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.144336:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d94 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65216512->65220607] (req 65216512->65220607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.144346:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65216512->65220607] (req 65216512->65220607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.144356:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.144362:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65216512->65220607] (req 65216512->65220607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.144372:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d94 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65216512->65220607] (req 65216512->65220607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.144606:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d94 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65216512->65220607] (req 65216512->65220607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.144615:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5d94 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65216512->65220607] (req 65216512->65220607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.144875:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554804736:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8309us (8602us total) trans 4294983231 rc 0/0 00000100:00100000:1.0:1789760784.144890:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15933 00000100:00100000:1.0:1789760784.144903:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727363514140 time 727382166872 state 0x2 ref 0 00000100:00100000:2.0:1789760784.149996:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760784.150328:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554804864 00000100:00100000:0.0:1789760784.150341:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554804864 00000100:00100000:0.0:1789760784.150431:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727363514140 time 727382166872 state 0x0 ref 1 00000100:00100000:0.0:1789760784.150450:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727382427288 time 727382166872 state 0x1 ref 1 00000100:00100000:0.0:1789760784.150488:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727382427288 time 727396105378 state 0x0 ref 1 00000100:00100000:1.0:1789760784.150652:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554804864:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760784.150735:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65220608, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f1b0 00010000:00010000:1.0:1789760784.150746:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5d9b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.150767:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760784.150772:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.150779:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5d9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65220608->65224703] (req 65220608->65224703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.150796:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5d9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65220608->65224703] (req 65220608->65224703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.157067:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983232 is committed 00002000:00010000:1.0:1789760784.157417:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65220608 -> 65224704 00002000:00010000:1.0:1789760784.157424:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127408 -> 127416 00010000:00010000:1.0:1789760784.157443:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5d9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65220608->65224703] (req 65220608->65224703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.157459:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5d9b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65220608->65224703] (req 65220608->65224703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.157470:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5d9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65220608->65224703] (req 65220608->65224703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.157479:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.157485:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5d9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65220608->65224703] (req 65220608->65224703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.157497:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5d9b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65220608->65224703] (req 65220608->65224703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.157524:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5d9b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65220608->65224703] (req 65220608->65224703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.157533:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5d9b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65220608->65224703] (req 65220608->65224703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.157654:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554804864:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7038us (7665us total) trans 4294983232 rc 0/0 00000100:00100000:1.0:1789760784.157678:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15934 00000100:00100000:1.0:1789760784.157690:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727382427288 time 727396105378 state 0x2 ref 0 00000100:00100000:2.0:1789760784.166037:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760784.166733:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554804992 00000100:00100000:0.0:1789760784.166747:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554804992 00000100:00100000:0.0:1789760784.166824:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727382427288 time 727396105378 state 0x0 ref 1 00000100:00100000:0.0:1789760784.166843:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727396365794 time 727396105378 state 0x1 ref 1 00000100:00100000:0.0:1789760784.166888:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727396365794 time 727412505702 state 0x0 ref 1 00000100:00100000:0.0:1789760784.166902:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554804992:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760784.166964:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65224704, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f1b8 00010000:00010000:0.0:1789760784.166976:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5da2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.166996:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760784.167001:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.167009:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5da2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65224704->65228799] (req 65224704->65228799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.167027:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5da2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65224704->65228799] (req 65224704->65228799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.174778:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983233 is committed 00002000:00010000:0.0:1789760784.174949:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65224704 -> 65228800 00002000:00010000:0.0:1789760784.174955:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127416 -> 127424 00010000:00010000:0.0:1789760784.174973:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5da2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65224704->65228799] (req 65224704->65228799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.174990:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5da2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65224704->65228799] (req 65224704->65228799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.175001:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5da2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65224704->65228799] (req 65224704->65228799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.175010:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760784.175016:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5da2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65224704->65228799] (req 65224704->65228799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.175027:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5da2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65224704->65228799] (req 65224704->65228799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.175062:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5da2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65224704->65228799] (req 65224704->65228799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.175078:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c5da2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65224704->65228799] (req 65224704->65228799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.175323:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554804992:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8416us (9292us total) trans 4294983233 rc 0/0 00000100:00100000:0.0:1789760784.175337:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15935 00000100:00100000:0.0:1789760784.175349:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727396365794 time 727412505702 state 0x2 ref 0 00000100:00100000:0.0:1789760784.178453:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760784.178601:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554805120 00000100:00100000:2.0:1789760784.178614:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554805120 00000100:00100000:2.0:1789760784.178696:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727396365794 time 727412505702 state 0x0 ref 1 00000100:00100000:2.0:1789760784.178714:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727412766118 time 727412505702 state 0x1 ref 1 00000100:00100000:2.0:1789760784.178749:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727412766118 time 727424367859 state 0x0 ref 1 00000100:00100000:2.0:1789760784.178763:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554805120:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760784.178827:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65228800, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f1c0 00010000:00010000:2.0:1789760784.178837:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5da9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.178856:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760784.178861:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.178868:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5da9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65228800->65232895] (req 65228800->65232895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.178883:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5da9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65228800->65232895] (req 65228800->65232895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.188083:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983234 is committed 00002000:00010000:2.0:1789760784.188769:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65228800 -> 65232896 00002000:00010000:2.0:1789760784.188776:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127424 -> 127432 00010000:00010000:2.0:1789760784.188795:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5da9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65228800->65232895] (req 65228800->65232895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.188811:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5da9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65228800->65232895] (req 65228800->65232895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.188821:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5da9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65228800->65232895] (req 65228800->65232895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.188833:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760784.188839:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5da9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65228800->65232895] (req 65228800->65232895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.188849:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5da9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65228800->65232895] (req 65228800->65232895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.188875:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5da9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65228800->65232895] (req 65228800->65232895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.188884:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5da9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65228800->65232895] (req 65228800->65232895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.189012:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554805120:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10244us (10566us total) trans 4294983234 rc 0/0 00000100:00100000:2.0:1789760784.189026:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15936 00000100:00100000:2.0:1789760784.189038:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727412766118 time 727424367859 state 0x2 ref 0 00000100:00100000:2.0:1789760784.194511:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760784.194600:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554805248 00000100:00100000:1.0:1789760784.194614:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554805248 00000100:00100000:1.0:1789760784.194701:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727412766118 time 727424367859 state 0x0 ref 1 00000100:00100000:1.0:1789760784.194721:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727424628275 time 727424367859 state 0x1 ref 1 00000100:00100000:1.0:1789760784.194801:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727424628275 time 727440417688 state 0x0 ref 1 00000100:00100000:1.0:1789760784.194819:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554805248:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760784.194883:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65232896, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f1c8 00010000:00010000:1.0:1789760784.194894:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5db0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.194915:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760784.194920:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.194928:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5db0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65232896->65236991] (req 65232896->65236991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.194945:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5db0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65232896->65236991] (req 65232896->65236991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.203880:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983235 is committed 00002000:00010000:1.0:1789760784.204030:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65232896 -> 65236992 00002000:00010000:1.0:1789760784.204036:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127432 -> 127440 00010000:00010000:1.0:1789760784.204064:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5db0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65232896->65236991] (req 65232896->65236991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.204085:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5db0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65232896->65236991] (req 65232896->65236991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.204095:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5db0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65232896->65236991] (req 65232896->65236991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.204103:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.204111:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5db0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65232896->65236991] (req 65232896->65236991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.204122:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5db0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65232896->65236991] (req 65232896->65236991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.204148:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5db0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65232896->65236991] (req 65232896->65236991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.204156:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5db0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65232896->65236991] (req 65232896->65236991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.204372:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f85500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554805248:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9547us (9944us total) trans 4294983235 rc 0/0 00000100:00100000:1.0:1789760784.204585:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15937 00000100:00100000:1.0:1789760784.204597:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727424628275 time 727440417688 state 0x2 ref 0 00000100:00100000:3.0:1789760784.208240:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760784.208341:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554805376 00000100:00100000:1.0:1789760784.208353:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554805376 00000100:00100000:1.0:1789760784.208442:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727424628275 time 727440417688 state 0x0 ref 1 00000100:00100000:1.0:1789760784.208456:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727440678104 time 727440417688 state 0x1 ref 1 00000100:00100000:1.0:1789760784.208479:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727440678104 time 727454097671 state 0x0 ref 1 00000100:00100000:1.0:1789760784.208493:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06490700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554805376:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760784.208555:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65236992, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f1d0 00010000:00010000:1.0:1789760784.208566:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5db7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.208582:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760784.208586:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.208593:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5db7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65236992->65241087] (req 65236992->65241087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.208606:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5db7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65236992->65241087] (req 65236992->65241087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.219771:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983236 is committed 00002000:00010000:1.0:1789760784.219928:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65236992 -> 65241088 00002000:00010000:1.0:1789760784.219935:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127440 -> 127448 00010000:00010000:1.0:1789760784.219955:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5db7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65236992->65241087] (req 65236992->65241087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.219972:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5db7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65236992->65241087] (req 65236992->65241087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.219983:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5db7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65236992->65241087] (req 65236992->65241087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.219993:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.219999:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5db7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65236992->65241087] (req 65236992->65241087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.220011:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5db7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65236992->65241087] (req 65236992->65241087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.220042:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5db7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65236992->65241087] (req 65236992->65241087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.220051:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5db7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65236992->65241087] (req 65236992->65241087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.220196:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06490700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554805376:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11698us (11963us total) trans 4294983236 rc 0/0 00000100:00100000:1.0:1789760784.220209:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15938 00000100:00100000:1.0:1789760784.220222:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727440678104 time 727454097671 state 0x2 ref 0 00000100:00100000:0.0:1789760784.224889:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760784.225005:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554805504 00000100:00100000:2.0:1789760784.225018:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554805504 00000100:00100000:2.0:1789760784.225093:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727440678104 time 727454097671 state 0x0 ref 1 00000100:00100000:2.0:1789760784.225117:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727454358087 time 727454097671 state 0x1 ref 1 00000100:00100000:2.0:1789760784.226835:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727454358087 time 727472436835 state 0x0 ref 1 00000100:00100000:2.0:1789760784.226878:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554805504:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760784.226953:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65241088, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f1d8 00010000:00010000:2.0:1789760784.226966:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5dbe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.226986:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760784.226992:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.227001:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5dbe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65241088->65245183] (req 65241088->65245183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.227019:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5dbe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65241088->65245183] (req 65241088->65245183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.236084:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983237 is committed 00002000:00010000:2.0:1789760784.236275:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65241088 -> 65245184 00002000:00010000:2.0:1789760784.236281:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127448 -> 127456 00010000:00010000:2.0:1789760784.236299:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5dbe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65241088->65245183] (req 65241088->65245183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.236314:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5dbe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65241088->65245183] (req 65241088->65245183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.236336:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5dbe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65241088->65245183] (req 65241088->65245183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.236359:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760784.236365:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5dbe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65241088->65245183] (req 65241088->65245183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.236414:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5dbe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65241088->65245183] (req 65241088->65245183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.236441:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5dbe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65241088->65245183] (req 65241088->65245183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.236450:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5dbe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65241088->65245183] (req 65241088->65245183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.236580:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554805504:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9714us (11697us total) trans 4294983237 rc 0/0 00000100:00100000:2.0:1789760784.236593:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15939 00000100:00100000:2.0:1789760784.236605:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727454358087 time 727472436835 state 0x2 ref 0 00000100:00100000:2.0:1789760784.240464:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760784.240662:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554805632 00000100:00100000:1.0:1789760784.240686:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554805632 00000100:00100000:1.0:1789760784.240762:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727454358087 time 727472436835 state 0x0 ref 1 00000100:00100000:1.0:1789760784.240783:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727472697251 time 727472436835 state 0x1 ref 1 00000100:00100000:1.0:1789760784.240822:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727472697251 time 727486439841 state 0x0 ref 1 00000100:00100000:1.0:1789760784.240836:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554805632:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760784.240904:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65245184, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f1e0 00010000:00010000:1.0:1789760784.240916:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5dc5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.240944:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760784.240949:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.240957:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5dc5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65245184->65249279] (req 65245184->65249279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.240975:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5dc5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65245184->65249279] (req 65245184->65249279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.247921:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983238 is committed 00002000:00010000:1.0:1789760784.248055:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65245184 -> 65249280 00002000:00010000:1.0:1789760784.248061:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127456 -> 127464 00010000:00010000:1.0:1789760784.248076:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5dc5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65245184->65249279] (req 65245184->65249279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.248090:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5dc5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65245184->65249279] (req 65245184->65249279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.248100:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5dc5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65245184->65249279] (req 65245184->65249279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.248107:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.248113:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5dc5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65245184->65249279] (req 65245184->65249279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.248122:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5dc5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65245184->65249279] (req 65245184->65249279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.248149:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5dc5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65245184->65249279] (req 65245184->65249279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.248158:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5dc5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65245184->65249279] (req 65245184->65249279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.248627:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554805632:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7785us (8171us total) trans 4294983238 rc 0/0 00000100:00100000:1.0:1789760784.248639:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15940 00000100:00100000:1.0:1789760784.248653:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727472697251 time 727486439841 state 0x2 ref 0 00000100:00100000:3.0:1789760784.251283:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760784.251416:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554805760 00000100:00100000:1.0:1789760784.251430:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554805760 00000100:00100000:1.0:1789760784.251505:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727472697251 time 727486439841 state 0x0 ref 1 00000100:00100000:1.0:1789760784.251525:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727486700257 time 727486439841 state 0x1 ref 1 00000100:00100000:1.0:1789760784.251559:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727486700257 time 727497177535 state 0x0 ref 1 00000100:00100000:1.0:1789760784.251572:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06493800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554805760:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760784.251630:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65249280, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f1e8 00010000:00010000:1.0:1789760784.251641:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5dcc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.251678:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760784.251685:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.251692:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5dcc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65249280->65253375] (req 65249280->65253375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.251707:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5dcc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65249280->65253375] (req 65249280->65253375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.259462:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983239 is committed 00002000:00010000:1.0:1789760784.259636:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65249280 -> 65253376 00002000:00010000:1.0:1789760784.259642:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127464 -> 127472 00010000:00010000:1.0:1789760784.259659:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5dcc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65249280->65253375] (req 65249280->65253375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.259685:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5dcc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65249280->65253375] (req 65249280->65253375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.259695:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5dcc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65249280->65253375] (req 65249280->65253375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.259702:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.259707:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5dcc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65249280->65253375] (req 65249280->65253375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.259717:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5dcc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65249280->65253375] (req 65249280->65253375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.259742:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5dcc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65249280->65253375] (req 65249280->65253375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.259756:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5dcc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65249280->65253375] (req 65249280->65253375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.259890:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06493800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554805760:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8312us (8611us total) trans 4294983239 rc 0/0 00000100:00100000:1.0:1789760784.259903:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15941 00000100:00100000:1.0:1789760784.259916:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727486700257 time 727497177535 state 0x2 ref 0 00000100:00100000:0.0:1789760784.270957:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760784.271058:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554805888 00000100:00100000:3.0:1789760784.271076:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554805888 00000100:00100000:3.0:1789760784.271156:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727486700257 time 727497177535 state 0x0 ref 1 00000100:00100000:3.0:1789760784.271175:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727497437951 time 727497177535 state 0x1 ref 1 00000100:00100000:3.0:1789760784.271225:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727497437951 time 727516843138 state 0x0 ref 1 00000100:00100000:3.0:1789760784.271241:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554805888:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760784.271303:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65253376, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f1f0 00010000:00010000:3.0:1789760784.271315:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5dd3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.271336:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760784.271341:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.271350:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5dd3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65253376->65257471] (req 65253376->65257471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.271367:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5dd3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65253376->65257471] (req 65253376->65257471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.278892:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983240 is committed 00002000:00010000:1.0:1789760784.279051:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65253376 -> 65257472 00002000:00010000:1.0:1789760784.279058:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127472 -> 127480 00010000:00010000:1.0:1789760784.279074:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5dd3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65253376->65257471] (req 65253376->65257471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.279090:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5dd3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65253376->65257471] (req 65253376->65257471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.279101:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5dd3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65253376->65257471] (req 65253376->65257471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.279110:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.279116:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5dd3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65253376->65257471] (req 65253376->65257471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.279126:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5dd3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65253376->65257471] (req 65253376->65257471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.279158:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5dd3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65253376->65257471] (req 65253376->65257471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.279167:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5dd3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65253376->65257471] (req 65253376->65257471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.279289:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554805888:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8043us (8334us total) trans 4294983240 rc 0/0 00000100:00100000:1.0:1789760784.279301:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15942 00000100:00100000:1.0:1789760784.279313:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727497437951 time 727516843138 state 0x2 ref 0 00000100:00100000:2.0:1789760784.283773:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760784.283921:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554806016 00000100:00100000:0.0:1789760784.283934:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554806016 00000100:00100000:0.0:1789760784.284013:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727497437951 time 727516843138 state 0x0 ref 1 00000100:00100000:0.0:1789760784.284031:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727517103554 time 727516843138 state 0x1 ref 1 00000100:00100000:0.0:1789760784.284078:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727517103554 time 727529695445 state 0x0 ref 1 00000100:00100000:0.0:1789760784.284093:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554806016:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760784.284160:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65257472, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f1f8 00010000:00010000:0.0:1789760784.284172:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5dda lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.284192:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760784.284199:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.284207:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5dda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65257472->65261567] (req 65257472->65261567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.284223:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5dda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65257472->65261567] (req 65257472->65261567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.295111:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983241 is committed 00002000:00010000:0.0:1789760784.295259:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65257472 -> 65261568 00002000:00010000:0.0:1789760784.295266:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127480 -> 127488 00010000:00010000:0.0:1789760784.295292:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5dda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65257472->65261567] (req 65257472->65261567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.295310:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5dda lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65257472->65261567] (req 65257472->65261567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.295321:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5dda lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65257472->65261567] (req 65257472->65261567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.295330:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760784.295338:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5dda lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65257472->65261567] (req 65257472->65261567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.295348:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5dda lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65257472->65261567] (req 65257472->65261567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.295373:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5dda lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65257472->65261567] (req 65257472->65261567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.295432:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5dda lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65257472->65261567] (req 65257472->65261567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.295565:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f84380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554806016:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11469us (11802us total) trans 4294983241 rc 0/0 00000100:00100000:0.0:1789760784.295579:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15943 00000100:00100000:0.0:1789760784.295591:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727517103554 time 727529695445 state 0x2 ref 0 00000100:00100000:3.0:1789760784.301347:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760784.302962:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554806144 00000100:00100000:2.0:1789760784.302982:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554806144 00000100:00100000:2.0:1789760784.303077:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727517103554 time 727529695445 state 0x0 ref 1 00000100:00100000:2.0:1789760784.303102:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727529955861 time 727529695445 state 0x1 ref 1 00000100:00100000:2.0:1789760784.303208:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727529955861 time 727548824798 state 0x0 ref 1 00000100:00100000:2.0:1789760784.303224:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06490a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554806144:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760784.303289:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65261568, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f200 00010000:00010000:2.0:1789760784.303302:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5de1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.303322:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760784.303327:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.303334:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5de1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65261568->65265663] (req 65261568->65265663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.303352:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5de1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65261568->65265663] (req 65261568->65265663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.311906:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983242 is committed 00002000:00010000:2.0:1789760784.312573:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65261568 -> 65265664 00002000:00010000:2.0:1789760784.312581:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127488 -> 127496 00010000:00010000:2.0:1789760784.312599:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5de1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65261568->65265663] (req 65261568->65265663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.312614:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5de1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65261568->65265663] (req 65261568->65265663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.312624:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5de1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65261568->65265663] (req 65261568->65265663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.312647:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760784.312653:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5de1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65261568->65265663] (req 65261568->65265663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.312663:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5de1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65261568->65265663] (req 65261568->65265663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.312703:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5de1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65261568->65265663] (req 65261568->65265663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.312713:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5de1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65261568->65265663] (req 65261568->65265663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.312847:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06490a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554806144:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9619us (11508us total) trans 4294983242 rc 0/0 00000100:00100000:2.0:1789760784.312862:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15944 00000100:00100000:2.0:1789760784.312875:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727529955861 time 727548824798 state 0x2 ref 0 00000100:00100000:1.0:1789760784.317818:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760784.318004:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554806272 00000100:00100000:2.0:1789760784.318019:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554806272 00000100:00100000:2.0:1789760784.318098:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727529955861 time 727548824798 state 0x0 ref 1 00000100:00100000:2.0:1789760784.318117:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727549085214 time 727548824798 state 0x1 ref 1 00000100:00100000:2.0:1789760784.318146:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727549085214 time 727563764223 state 0x0 ref 1 00000100:00100000:2.0:1789760784.318165:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554806272:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760784.318230:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65265664, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f208 00010000:00010000:2.0:1789760784.318242:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5de8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.318262:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760784.318268:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.318275:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5de8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65265664->65269759] (req 65265664->65269759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.318291:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5de8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65265664->65269759] (req 65265664->65269759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.326643:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983243 is committed 00002000:00010000:2.0:1789760784.326829:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65265664 -> 65269760 00002000:00010000:2.0:1789760784.326836:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127496 -> 127504 00010000:00010000:2.0:1789760784.326854:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5de8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65265664->65269759] (req 65265664->65269759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.326872:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5de8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65265664->65269759] (req 65265664->65269759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.326883:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5de8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65265664->65269759] (req 65265664->65269759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.326896:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760784.326903:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5de8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65265664->65269759] (req 65265664->65269759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.326913:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5de8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65265664->65269759] (req 65265664->65269759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.326939:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5de8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65265664->65269759] (req 65265664->65269759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.326946:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5de8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65265664->65269759] (req 65265664->65269759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.327336:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554806272:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9164us (9524us total) trans 4294983243 rc 0/0 00000100:00100000:0.0:1789760784.327350:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15945 00000100:00100000:0.0:1789760784.327362:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727549085214 time 727563764223 state 0x2 ref 0 00000100:00100000:1.0:1789760784.334879:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760784.334987:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554806400 00000100:00100000:3.0:1789760784.335000:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554806400 00000100:00100000:3.0:1789760784.335193:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727549085214 time 727563764223 state 0x0 ref 1 00000100:00100000:3.0:1789760784.335212:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727564024639 time 727563764223 state 0x1 ref 1 00000100:00100000:3.0:1789760784.335310:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727564024639 time 727580926927 state 0x0 ref 1 00000100:00100000:3.0:1789760784.335326:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554806400:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760784.335440:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65269760, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f210 00010000:00010000:3.0:1789760784.335452:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5def lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.335472:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760784.335478:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.335485:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5def lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65269760->65273855] (req 65269760->65273855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.335515:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5def lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65269760->65273855] (req 65269760->65273855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.359252:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983244 is committed 00002000:00010000:3.0:1789760784.359499:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65269760 -> 65273856 00002000:00010000:3.0:1789760784.359506:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127504 -> 127512 00010000:00010000:3.0:1789760784.359526:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5def lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65269760->65273855] (req 65269760->65273855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.359540:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5def lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65269760->65273855] (req 65269760->65273855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.359549:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5def lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65269760->65273855] (req 65269760->65273855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.359558:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760784.359564:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5def lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65269760->65273855] (req 65269760->65273855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.359579:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5def lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65269760->65273855] (req 65269760->65273855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.359603:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5def lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65269760->65273855] (req 65269760->65273855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.359610:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5def lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65269760->65273855] (req 65269760->65273855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760784.359764:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554806400:12345-192.168.203.22@tcp:4:dd.0 Request processed in 24433us (24891us total) trans 4294983244 rc 0/0 00000100:00100000:3.0:1789760784.359778:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15946 00000100:00100000:3.0:1789760784.359790:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727564024639 time 727580926927 state 0x2 ref 0 00000100:00100000:3.0:1789760784.364526:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760784.364686:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554806528 00000100:00100000:0.0:1789760784.364699:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554806528 00000100:00100000:0.0:1789760784.364774:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727564024639 time 727580926927 state 0x0 ref 1 00000100:00100000:0.0:1789760784.364794:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727581187343 time 727580926927 state 0x1 ref 1 00000100:00100000:0.0:1789760784.364828:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727581187343 time 727610446717 state 0x0 ref 1 00000100:00100000:0.0:1789760784.364842:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06491c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554806528:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760784.364909:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65273856, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f218 00010000:00010000:0.0:1789760784.364921:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5df6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.364943:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760784.364948:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.364962:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5df6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65273856->65277951] (req 65273856->65277951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.364979:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5df6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65273856->65277951] (req 65273856->65277951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.374796:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983245 is committed 00002000:00010000:0.0:1789760784.374994:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65273856 -> 65277952 00002000:00010000:0.0:1789760784.374999:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127512 -> 127520 00010000:00010000:0.0:1789760784.375016:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5df6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65273856->65277951] (req 65273856->65277951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.375029:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5df6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65273856->65277951] (req 65273856->65277951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.375092:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5df6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65273856->65277951] (req 65273856->65277951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.375103:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760784.375108:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5df6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65273856->65277951] (req 65273856->65277951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.375118:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5df6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65273856->65277951] (req 65273856->65277951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.375149:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5df6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65273856->65277951] (req 65273856->65277951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.375158:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5df6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65273856->65277951] (req 65273856->65277951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.375340:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06491c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554806528:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10493us (10822us total) trans 4294983245 rc 0/0 00000100:00100000:0.0:1789760784.375353:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15947 00000100:00100000:0.0:1789760784.375364:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727581187343 time 727610446717 state 0x2 ref 0 00000100:00100000:1.0:1789760784.379900:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760784.380303:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554806656 00000100:00100000:2.0:1789760784.380314:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554806656 00000100:00100000:2.0:1789760784.380411:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727581187343 time 727610446717 state 0x0 ref 1 00000100:00100000:2.0:1789760784.380431:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727610707133 time 727610446717 state 0x1 ref 1 00000100:00100000:2.0:1789760784.380463:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727610707133 time 727626081211 state 0x0 ref 1 00000100:00100000:2.0:1789760784.380476:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554806656:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760784.380535:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65277952, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f220 00010000:00010000:2.0:1789760784.380547:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5dfd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.380565:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760784.380570:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.380578:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5dfd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65277952->65282047] (req 65277952->65282047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.380594:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5dfd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65277952->65282047] (req 65277952->65282047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.391058:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983246 is committed 00002000:00010000:2.0:1789760784.393310:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65277952 -> 65282048 00002000:00010000:2.0:1789760784.393322:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127520 -> 127528 00010000:00010000:2.0:1789760784.393340:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5dfd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65277952->65282047] (req 65277952->65282047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.393356:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5dfd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65277952->65282047] (req 65277952->65282047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.393366:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5dfd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65277952->65282047] (req 65277952->65282047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.393374:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760784.393397:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5dfd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65277952->65282047] (req 65277952->65282047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.393407:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5dfd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65277952->65282047] (req 65277952->65282047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.393442:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5dfd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65277952->65282047] (req 65277952->65282047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.393457:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5dfd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65277952->65282047] (req 65277952->65282047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.393580:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554806656:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13099us (13687us total) trans 4294983246 rc 0/0 00000100:00100000:2.0:1789760784.393594:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15948 00000100:00100000:2.0:1789760784.393606:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727610707133 time 727626081211 state 0x2 ref 0 00000100:00100000:1.0:1789760784.398922:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760784.399033:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554806784 00000100:00100000:2.0:1789760784.399046:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554806784 00000100:00100000:2.0:1789760784.399122:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727610707133 time 727626081211 state 0x0 ref 1 00000100:00100000:2.0:1789760784.399140:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727626341627 time 727626081211 state 0x1 ref 1 00000100:00100000:2.0:1789760784.399177:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727626341627 time 727644794433 state 0x0 ref 1 00000100:00100000:2.0:1789760784.399192:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554806784:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760784.399258:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65282048, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f228 00010000:00010000:2.0:1789760784.399269:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5e04 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.399290:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760784.399296:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.399304:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5e04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65282048->65286143] (req 65282048->65286143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.399327:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5e04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65282048->65286143] (req 65282048->65286143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.408950:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983247 is committed 00002000:00010000:2.0:1789760784.409162:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65282048 -> 65286144 00002000:00010000:2.0:1789760784.409168:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127528 -> 127536 00010000:00010000:2.0:1789760784.409185:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5e04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65282048->65286143] (req 65282048->65286143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.409200:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5e04 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65282048->65286143] (req 65282048->65286143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.409210:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5e04 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65282048->65286143] (req 65282048->65286143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.409218:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760784.409223:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5e04 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65282048->65286143] (req 65282048->65286143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.409233:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5e04 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65282048->65286143] (req 65282048->65286143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.409257:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5e04 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65282048->65286143] (req 65282048->65286143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.409265:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5e04 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65282048->65286143] (req 65282048->65286143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.409444:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554806784:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10247us (10531us total) trans 4294983247 rc 0/0 00000100:00100000:2.0:1789760784.409457:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15949 00000100:00100000:2.0:1789760784.409468:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727626341627 time 727644794433 state 0x2 ref 0 00000100:00100000:2.0:1789760784.415657:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760784.416179:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554806912 00000100:00100000:3.0:1789760784.416194:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554806912 00000100:00100000:3.0:1789760784.416267:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727626341627 time 727644794433 state 0x0 ref 1 00000100:00100000:3.0:1789760784.416286:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727645054849 time 727644794433 state 0x1 ref 1 00000100:00100000:3.0:1789760784.416358:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727645054849 time 727661975184 state 0x0 ref 1 00000100:00100000:3.0:1789760784.416375:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554806912:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760784.416465:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65286144, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f230 00010000:00010000:3.0:1789760784.416476:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5e0b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.416496:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760784.416501:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.416508:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5e0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65286144->65290239] (req 65286144->65290239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.416524:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5e0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65286144->65290239] (req 65286144->65290239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.425190:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983248 is committed 00002000:00010000:1.0:1789760784.425528:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65286144 -> 65290240 00002000:00010000:1.0:1789760784.425533:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127536 -> 127544 00010000:00010000:1.0:1789760784.425546:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5e0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65286144->65290239] (req 65286144->65290239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.425559:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5e0b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65286144->65290239] (req 65286144->65290239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.425568:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5e0b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65286144->65290239] (req 65286144->65290239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.425575:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.425580:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5e0b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65286144->65290239] (req 65286144->65290239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.425588:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5e0b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65286144->65290239] (req 65286144->65290239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.425607:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5e0b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65286144->65290239] (req 65286144->65290239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.425614:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5e0b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65286144->65290239] (req 65286144->65290239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.425716:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554806912:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9338us (10069us total) trans 4294983248 rc 0/0 00000100:00100000:1.0:1789760784.425726:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15950 00000100:00100000:1.0:1789760784.425735:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727645054849 time 727661975184 state 0x2 ref 0 00000100:00100000:1.0:1789760784.432302:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760784.432404:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554807040 00000100:00100000:0.0:1789760784.432414:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554807040 00000100:00100000:0.0:1789760784.432590:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727645054849 time 727661975184 state 0x0 ref 1 00000100:00100000:0.0:1789760784.432606:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727662235600 time 727661975184 state 0x1 ref 1 00000100:00100000:0.0:1789760784.432690:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727662235600 time 727678307775 state 0x0 ref 1 00000100:00100000:0.0:1789760784.432703:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554807040:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760784.432747:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65290240, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f238 00010000:00010000:0.0:1789760784.432756:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e12 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.432772:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760784.432777:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.432783:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65290240->65294335] (req 65290240->65294335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.432796:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65290240->65294335] (req 65290240->65294335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.439301:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983249 is committed 00002000:00010000:0.0:1789760784.439464:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65290240 -> 65294336 00002000:00010000:0.0:1789760784.439470:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127544 -> 127552 00010000:00010000:0.0:1789760784.439483:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65290240->65294335] (req 65290240->65294335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.439496:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e12 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65290240->65294335] (req 65290240->65294335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.439505:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e12 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65290240->65294335] (req 65290240->65294335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.439512:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760784.439516:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e12 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65290240->65294335] (req 65290240->65294335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.439524:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e12 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65290240->65294335] (req 65290240->65294335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.439545:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e12 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65290240->65294335] (req 65290240->65294335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.439558:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e12 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65290240->65294335] (req 65290240->65294335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.439655:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554807040:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6950us (7358us total) trans 4294983249 rc 0/0 00000100:00100000:0.0:1789760784.439675:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15951 00000100:00100000:0.0:1789760784.439685:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727662235600 time 727678307775 state 0x2 ref 0 00000100:00100000:2.0:1789760784.442776:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760784.442923:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554807168 00000100:00100000:0.0:1789760784.442932:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554807168 00000100:00100000:0.0:1789760784.442984:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727662235600 time 727678307775 state 0x0 ref 1 00000100:00100000:0.0:1789760784.442999:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727678568191 time 727678307775 state 0x1 ref 1 00000100:00100000:0.0:1789760784.443028:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727678568191 time 727688647482 state 0x0 ref 1 00000100:00100000:0.0:1789760784.443040:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a272a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554807168:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760784.443097:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65294336, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f240 00010000:00010000:0.0:1789760784.443106:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e19 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.443123:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760784.443127:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.443133:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65294336->65298431] (req 65294336->65298431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.443154:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65294336->65298431] (req 65294336->65298431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.448763:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983250 is committed 00002000:00010000:0.0:1789760784.448881:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65294336 -> 65298432 00002000:00010000:0.0:1789760784.448887:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127552 -> 127560 00010000:00010000:0.0:1789760784.448900:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65294336->65298431] (req 65294336->65298431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.448913:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e19 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65294336->65298431] (req 65294336->65298431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.448923:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e19 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65294336->65298431] (req 65294336->65298431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.448930:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760784.448935:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e19 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65294336->65298431] (req 65294336->65298431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.448943:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e19 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65294336->65298431] (req 65294336->65298431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.448962:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e19 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65294336->65298431] (req 65294336->65298431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.448969:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e19 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65294336->65298431] (req 65294336->65298431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.449477:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a272a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554807168:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6432us (6704us total) trans 4294983250 rc 0/0 00000100:00100000:0.0:1789760784.449488:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15952 00000100:00100000:0.0:1789760784.449497:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727678568191 time 727688647482 state 0x2 ref 0 00000100:00100000:2.0:1789760784.459003:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760784.460124:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554807296 00000100:00100000:1.0:1789760784.460136:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554807296 00000100:00100000:1.0:1789760784.460205:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727678568191 time 727688647482 state 0x0 ref 1 00000100:00100000:1.0:1789760784.460220:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727688907898 time 727688647482 state 0x1 ref 1 00000100:00100000:1.0:1789760784.460253:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727688907898 time 727705871571 state 0x0 ref 1 00000100:00100000:1.0:1789760784.460267:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a270380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554807296:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760784.460311:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65298432, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f248 00010000:00010000:1.0:1789760784.460321:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5e20 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.460336:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760784.460341:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.460348:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5e20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65298432->65302527] (req 65298432->65302527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.460360:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5e20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65298432->65302527] (req 65298432->65302527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.469737:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983251 is committed 00002000:00010000:1.0:1789760784.469840:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65298432 -> 65302528 00002000:00010000:1.0:1789760784.469845:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127560 -> 127568 00010000:00010000:1.0:1789760784.469859:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5e20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65298432->65302527] (req 65298432->65302527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.469871:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5e20 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65298432->65302527] (req 65298432->65302527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.469880:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5e20 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65298432->65302527] (req 65298432->65302527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.469887:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.469893:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5e20 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65298432->65302527] (req 65298432->65302527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.469903:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5e20 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65298432->65302527] (req 65298432->65302527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.469923:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5e20 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65298432->65302527] (req 65298432->65302527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.469931:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5e20 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65298432->65302527] (req 65298432->65302527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.470081:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a270380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554807296:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9810us (11081us total) trans 4294983251 rc 0/0 00000100:00100000:1.0:1789760784.470092:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15953 00000100:00100000:1.0:1789760784.470101:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727688907898 time 727705871571 state 0x2 ref 0 00000100:00100000:1.0:1789760784.478532:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760784.483817:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554807424 00000100:00100000:3.0:1789760784.483832:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554807424 00000100:00100000:3.0:1789760784.483896:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727688907898 time 727705871571 state 0x0 ref 1 00000100:00100000:3.0:1789760784.483913:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727706131987 time 727705871571 state 0x1 ref 1 00000100:00100000:3.0:1789760784.483938:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727706131987 time 727729556697 state 0x0 ref 1 00000100:00100000:3.0:1789760784.483951:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554807424:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760784.484013:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65302528, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f250 00010000:00010000:3.0:1789760784.484023:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5e27 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.484039:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760784.484044:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.484050:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5e27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65302528->65306623] (req 65302528->65306623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.484065:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5e27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65302528->65306623] (req 65302528->65306623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.489419:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983252 is committed 00002000:00010000:1.0:1789760784.489542:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65302528 -> 65306624 00002000:00010000:1.0:1789760784.489547:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127568 -> 127576 00010000:00010000:1.0:1789760784.489560:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5e27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65302528->65306623] (req 65302528->65306623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.489573:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5e27 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65302528->65306623] (req 65302528->65306623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.489582:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5e27 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65302528->65306623] (req 65302528->65306623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.489590:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.489596:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5e27 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65302528->65306623] (req 65302528->65306623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.489604:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5e27 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65302528->65306623] (req 65302528->65306623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.489623:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5e27 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65302528->65306623] (req 65302528->65306623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.489631:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5e27 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65302528->65306623] (req 65302528->65306623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.489822:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554807424:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5868us (11294us total) trans 4294983252 rc 0/0 00000100:00100000:1.0:1789760784.489834:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15954 00000100:00100000:1.0:1789760784.489844:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727706131987 time 727729556697 state 0x2 ref 0 00000100:00100000:2.0:1789760784.494321:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760784.494571:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554807552 00000100:00100000:0.0:1789760784.494582:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554807552 00000100:00100000:0.0:1789760784.494635:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727706131987 time 727729556697 state 0x0 ref 1 00000100:00100000:0.0:1789760784.494649:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727729817113 time 727729556697 state 0x1 ref 1 00000100:00100000:0.0:1789760784.494731:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727729817113 time 727740348266 state 0x0 ref 1 00000100:00100000:0.0:1789760784.494744:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a270700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554807552:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760784.494793:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65306624, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f258 00010000:00010000:0.0:1789760784.494803:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e2e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.494818:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760784.494823:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.494829:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65306624->65310719] (req 65306624->65310719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.494841:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65306624->65310719] (req 65306624->65310719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.499694:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983253 is committed 00002000:00010000:0.0:1789760784.502508:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65306624 -> 65310720 00002000:00010000:0.0:1789760784.502518:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127576 -> 127584 00010000:00010000:0.0:1789760784.502533:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65306624->65310719] (req 65306624->65310719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.502546:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e2e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65306624->65310719] (req 65306624->65310719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.502554:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65306624->65310719] (req 65306624->65310719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.502561:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760784.502567:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65306624->65310719] (req 65306624->65310719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.502576:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e2e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65306624->65310719] (req 65306624->65310719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.502596:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e2e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65306624->65310719] (req 65306624->65310719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.502604:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e2e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65306624->65310719] (req 65306624->65310719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.502700:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a270700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554807552:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7952us (8384us total) trans 4294983253 rc 0/0 00000100:00100000:0.0:1789760784.502712:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15955 00000100:00100000:0.0:1789760784.502721:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727729817113 time 727740348266 state 0x2 ref 0 00000100:00100000:2.0:1789760784.507312:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760784.507444:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554807680 00000100:00100000:0.0:1789760784.507453:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554807680 00000100:00100000:0.0:1789760784.507510:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727729817113 time 727740348266 state 0x0 ref 1 00000100:00100000:0.0:1789760784.507525:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727740608682 time 727740348266 state 0x1 ref 1 00000100:00100000:0.0:1789760784.507545:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727740608682 time 727753164611 state 0x0 ref 1 00000100:00100000:0.0:1789760784.507557:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554807680:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760784.507605:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65310720, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f260 00010000:00010000:0.0:1789760784.507613:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e35 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.507630:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760784.507634:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.507641:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65310720->65314815] (req 65310720->65314815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.507656:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65310720->65314815] (req 65310720->65314815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.513307:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983254 is committed 00002000:00010000:0.0:1789760784.513581:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65310720 -> 65314816 00002000:00010000:0.0:1789760784.513587:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127584 -> 127592 00010000:00010000:0.0:1789760784.513604:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65310720->65314815] (req 65310720->65314815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.513617:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e35 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65310720->65314815] (req 65310720->65314815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.513626:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e35 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65310720->65314815] (req 65310720->65314815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.513634:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760784.513638:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e35 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65310720->65314815] (req 65310720->65314815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.513648:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e35 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65310720->65314815] (req 65310720->65314815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.513686:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e35 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65310720->65314815] (req 65310720->65314815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.513695:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e35 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65310720->65314815] (req 65310720->65314815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.513836:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554807680:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6275us (6529us total) trans 4294983254 rc 0/0 00000100:00100000:0.0:1789760784.513847:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15956 00000100:00100000:0.0:1789760784.513857:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727740608682 time 727753164611 state 0x2 ref 0 00000100:00100000:1.0:1789760784.518127:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760784.518262:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554807808 00000100:00100000:2.0:1789760784.518271:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554807808 00000100:00100000:2.0:1789760784.518324:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727740608682 time 727753164611 state 0x0 ref 1 00000100:00100000:2.0:1789760784.518341:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727753425027 time 727753164611 state 0x1 ref 1 00000100:00100000:2.0:1789760784.518373:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727753425027 time 727763991005 state 0x0 ref 1 00000100:00100000:2.0:1789760784.518417:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554807808:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760784.518625:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65314816, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f268 00010000:00010000:2.0:1789760784.518635:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5e3c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.518652:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760784.518656:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.518672:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5e3c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65314816->65318911] (req 65314816->65318911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.518687:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5e3c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65314816->65318911] (req 65314816->65318911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.523910:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983255 is committed 00002000:00010000:2.0:1789760784.525224:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65314816 -> 65318912 00002000:00010000:2.0:1789760784.525233:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127592 -> 127600 00010000:00010000:2.0:1789760784.525248:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5e3c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65314816->65318911] (req 65314816->65318911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.525261:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5e3c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65314816->65318911] (req 65314816->65318911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.525269:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5e3c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65314816->65318911] (req 65314816->65318911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.525287:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760784.525292:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5e3c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65314816->65318911] (req 65314816->65318911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.525302:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5e3c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65314816->65318911] (req 65314816->65318911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.525322:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5e3c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65314816->65318911] (req 65314816->65318911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.525335:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5e3c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65314816->65318911] (req 65314816->65318911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.525730:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554807808:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7310us (7609us total) trans 4294983255 rc 0/0 00000100:00100000:2.0:1789760784.525743:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15957 00000100:00100000:2.0:1789760784.525754:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727753425027 time 727763991005 state 0x2 ref 0 00000100:00100000:2.0:1789760784.530888:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760784.530970:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554807936 00000100:00100000:3.0:1789760784.530979:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554807936 00000100:00100000:3.0:1789760784.531051:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727753425027 time 727763991005 state 0x0 ref 1 00000100:00100000:3.0:1789760784.531067:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727764251421 time 727763991005 state 0x1 ref 1 00000100:00100000:3.0:1789760784.531088:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727764251421 time 727776706733 state 0x0 ref 1 00000100:00100000:3.0:1789760784.531100:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554807936:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760784.531145:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65318912, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f270 00010000:00010000:3.0:1789760784.531157:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5e43 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.531172:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:3.0:1789760784.531175:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760784.531180:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5e43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65318912->65323007] (req 65318912->65323007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760784.531190:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5e43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65318912->65323007] (req 65318912->65323007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.544102:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983256 is committed 00002000:00010000:1.0:1789760784.547785:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65318912 -> 65323008 00002000:00010000:1.0:1789760784.547794:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127600 -> 127608 00010000:00010000:1.0:1789760784.547810:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5e43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65318912->65323007] (req 65318912->65323007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.547824:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5e43 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65318912->65323007] (req 65318912->65323007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.547834:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5e43 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65318912->65323007] (req 65318912->65323007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.547842:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.547846:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5e43 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65318912->65323007] (req 65318912->65323007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.547855:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5e43 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65318912->65323007] (req 65318912->65323007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.547874:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5e43 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65318912->65323007] (req 65318912->65323007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.547881:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c5e43 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65318912->65323007] (req 65318912->65323007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.551692:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554807936:12345-192.168.203.22@tcp:4:dd.0 Request processed in 20564us (20783us total) trans 4294983256 rc 0/0 00000100:00100000:1.0:1789760784.551707:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15958 00000100:00100000:1.0:1789760784.551717:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727764251421 time 727776706733 state 0x2 ref 0 00000100:00100000:3.0:1789760784.557991:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760784.558072:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554808064 00000100:00100000:0.0:1789760784.558082:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554808064 00000100:00100000:0.0:1789760784.558137:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727764251421 time 727776706733 state 0x0 ref 1 00000100:00100000:0.0:1789760784.558153:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727776967149 time 727776706733 state 0x1 ref 1 00000100:00100000:0.0:1789760784.558279:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727776967149 time 727803895718 state 0x0 ref 1 00000100:00100000:0.0:1789760784.558293:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f63480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554808064:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760784.558339:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65323008, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f278 00010000:00010000:0.0:1789760784.558348:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e4a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.558364:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760784.558369:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.558375:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65323008->65327103] (req 65323008->65327103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.558423:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65323008->65327103] (req 65323008->65327103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.566128:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983257 is committed 00002000:00010000:0.0:1789760784.566229:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65323008 -> 65327104 00002000:00010000:0.0:1789760784.566234:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127608 -> 127616 00010000:00010000:0.0:1789760784.566247:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65323008->65327103] (req 65323008->65327103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.566264:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e4a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65323008->65327103] (req 65323008->65327103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.566270:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e4a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65323008->65327103] (req 65323008->65327103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.566275:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760784.566278:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e4a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65323008->65327103] (req 65323008->65327103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.566284:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e4a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65323008->65327103] (req 65323008->65327103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.566300:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e4a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65323008->65327103] (req 65323008->65327103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.566304:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e4a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65323008->65327103] (req 65323008->65327103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.566420:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f63480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554808064:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8085us (8394us total) trans 4294983257 rc 0/0 00000100:00100000:0.0:1789760784.566430:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15959 00000100:00100000:0.0:1789760784.566440:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727776967149 time 727803895718 state 0x2 ref 0 00000100:00100000:2.0:1789760784.578634:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760784.578934:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554808192 00000100:00100000:0.0:1789760784.579092:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554808192 00000100:00100000:0.0:1789760784.579153:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727776967149 time 727803895718 state 0x0 ref 1 00000100:00100000:0.0:1789760784.579166:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727804156134 time 727803895718 state 0x1 ref 1 00000100:00100000:0.0:1789760784.579255:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727804156134 time 727824872718 state 0x0 ref 1 00000100:00100000:0.0:1789760784.579268:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554808192:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760784.579318:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65327104, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f280 00010000:00010000:0.0:1789760784.579327:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e51 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.579343:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760784.579347:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.579353:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65327104->65331199] (req 65327104->65331199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.579366:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65327104->65331199] (req 65327104->65331199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.586293:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983258 is committed 00002000:00010000:0.0:1789760784.586916:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65327104 -> 65331200 00002000:00010000:0.0:1789760784.586923:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127616 -> 127624 00010000:00010000:0.0:1789760784.586935:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65327104->65331199] (req 65327104->65331199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.586947:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e51 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65327104->65331199] (req 65327104->65331199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.586962:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e51 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65327104->65331199] (req 65327104->65331199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.586970:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760784.586975:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e51 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65327104->65331199] (req 65327104->65331199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.586984:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e51 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65327104->65331199] (req 65327104->65331199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.587003:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e51 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65327104->65331199] (req 65327104->65331199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.587012:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e51 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65327104->65331199] (req 65327104->65331199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.587170:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554808192:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7895us (8539us total) trans 4294983258 rc 0/0 00000100:00100000:0.0:1789760784.587248:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15960 00000100:00100000:0.0:1789760784.587258:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727804156134 time 727824872718 state 0x2 ref 0 00000100:00100000:2.0:1789760784.593146:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760784.593290:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554808320 00000100:00100000:1.0:1789760784.593299:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554808320 00000100:00100000:1.0:1789760784.593353:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727804156134 time 727824872718 state 0x0 ref 1 00000100:00100000:1.0:1789760784.593368:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727825133134 time 727824872718 state 0x1 ref 1 00000100:00100000:1.0:1789760784.593461:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727825133134 time 727839078299 state 0x0 ref 1 00000100:00100000:1.0:1789760784.593474:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554808320:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760784.593519:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65331200, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f288 00010000:00010000:1.0:1789760784.593527:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5e58 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.593542:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760784.593546:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.593553:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5e58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65331200->65335295] (req 65331200->65335295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.593565:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5e58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65331200->65335295] (req 65331200->65335295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.600162:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983259 is committed 00002000:00010000:1.0:1789760784.600353:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65331200 -> 65335296 00002000:00010000:1.0:1789760784.600358:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127624 -> 127632 00010000:00010000:1.0:1789760784.600371:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5e58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65331200->65335295] (req 65331200->65335295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.600527:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5e58 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65331200->65335295] (req 65331200->65335295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.600537:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5e58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65331200->65335295] (req 65331200->65335295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.600549:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.600553:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5e58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65331200->65335295] (req 65331200->65335295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.600563:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5e58 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65331200->65335295] (req 65331200->65335295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.600582:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5e58 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65331200->65335295] (req 65331200->65335295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.600590:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5e58 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65331200->65335295] (req 65331200->65335295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.600887:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554808320:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7409us (7743us total) trans 4294983259 rc 0/0 00000100:00100000:1.0:1789760784.600898:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15961 00000100:00100000:1.0:1789760784.600907:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727825133134 time 727839078299 state 0x2 ref 0 00000100:00100000:3.0:1789760784.605261:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760784.605357:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554808448 00000100:00100000:1.0:1789760784.605367:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554808448 00000100:00100000:1.0:1789760784.605469:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727825133134 time 727839078299 state 0x0 ref 1 00000100:00100000:1.0:1789760784.605484:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727839338715 time 727839078299 state 0x1 ref 1 00000100:00100000:1.0:1789760784.605629:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727839338715 time 727851245942 state 0x0 ref 1 00000100:00100000:1.0:1789760784.605644:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f63b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554808448:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760784.605702:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65335296, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f290 00010000:00010000:1.0:1789760784.605711:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5e5f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.605726:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760784.605730:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.605736:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5e5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65335296->65339391] (req 65335296->65339391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.605750:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5e5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65335296->65339391] (req 65335296->65339391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.611009:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983260 is committed 00002000:00010000:1.0:1789760784.611107:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65335296 -> 65339392 00002000:00010000:1.0:1789760784.611111:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127632 -> 127640 00010000:00010000:1.0:1789760784.611122:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5e5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65335296->65339391] (req 65335296->65339391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.611131:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5e5f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65335296->65339391] (req 65335296->65339391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.611138:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5e5f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65335296->65339391] (req 65335296->65339391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.611142:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.611147:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5e5f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65335296->65339391] (req 65335296->65339391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.611153:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5e5f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65335296->65339391] (req 65335296->65339391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.611166:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5e5f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65335296->65339391] (req 65335296->65339391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.611171:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5e5f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65335296->65339391] (req 65335296->65339391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.611299:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f63b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554808448:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5654us (6044us total) trans 4294983260 rc 0/0 00000100:00100000:1.0:1789760784.611307:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15962 00000100:00100000:1.0:1789760784.611313:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727839338715 time 727851245942 state 0x2 ref 0 00000100:00100000:2.0:1789760784.615719:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760784.615758:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554808576 00000100:00100000:1.0:1789760784.615769:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554808576 00000100:00100000:1.0:1789760784.615828:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727839338715 time 727851245942 state 0x0 ref 1 00000100:00100000:1.0:1789760784.615843:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727851506358 time 727851245942 state 0x1 ref 1 00000100:00100000:1.0:1789760784.615875:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727851506358 time 727861494154 state 0x0 ref 1 00000100:00100000:1.0:1789760784.615888:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554808576:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760784.616127:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65339392, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f298 00010000:00010000:1.0:1789760784.616137:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5e66 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.616154:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760784.616158:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.616165:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5e66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65339392->65343487] (req 65339392->65343487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.616178:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5e66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65339392->65343487] (req 65339392->65343487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.622330:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983261 is committed 00002000:00010000:1.0:1789760784.622511:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65339392 -> 65343488 00002000:00010000:1.0:1789760784.622517:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127640 -> 127648 00010000:00010000:1.0:1789760784.622529:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5e66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65339392->65343487] (req 65339392->65343487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.622542:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5e66 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65339392->65343487] (req 65339392->65343487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.622551:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5e66 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65339392->65343487] (req 65339392->65343487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.622559:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.622563:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5e66 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65339392->65343487] (req 65339392->65343487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.622572:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5e66 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65339392->65343487] (req 65339392->65343487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.622591:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5e66 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65339392->65343487] (req 65339392->65343487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.622599:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5e66 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65339392->65343487] (req 65339392->65343487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.623053:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554808576:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7157us (7331us total) trans 4294983261 rc 0/0 00000100:00100000:1.0:1789760784.623064:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15963 00000100:00100000:1.0:1789760784.623074:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727851506358 time 727861494154 state 0x2 ref 0 00000100:00100000:2.0:1789760784.629925:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760784.631014:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554808704 00000100:00100000:0.0:1789760784.631024:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554808704 00000100:00100000:0.0:1789760784.631075:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727851506358 time 727861494154 state 0x0 ref 1 00000100:00100000:0.0:1789760784.631098:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727861754570 time 727861494154 state 0x1 ref 1 00000100:00100000:0.0:1789760784.631121:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727861754570 time 727876739757 state 0x0 ref 1 00000100:00100000:0.0:1789760784.631219:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554808704:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760784.631369:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65343488, mtime: 0x6aad950f, atime: 0x0, ctime: 0x6aad950f, blocks: 0x1f2a0 00010000:00010000:0.0:1789760784.631406:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e6d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.631431:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760784.631437:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.631444:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65343488->65347583] (req 65343488->65347583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.631537:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65343488->65347583] (req 65343488->65347583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.636356:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983262 is committed 00002000:00010000:2.0:1789760784.637156:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65343488 -> 65347584 00002000:00010000:2.0:1789760784.637162:0:8441:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760783 -> 1789760784 00002000:00010000:2.0:1789760784.637164:0:8441:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760783 -> 1789760784 00002000:00010000:2.0:1789760784.637166:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127648 -> 127656 00010000:00010000:2.0:1789760784.637179:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65343488->65347583] (req 65343488->65347583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.637191:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e6d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65343488->65347583] (req 65343488->65347583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.637204:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65343488->65347583] (req 65343488->65347583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.637210:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760784.637215:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65343488->65347583] (req 65343488->65347583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.637330:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e6d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65343488->65347583] (req 65343488->65347583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.637350:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e6d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65343488->65347583] (req 65343488->65347583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.637357:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c5e6d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65343488->65347583] (req 65343488->65347583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.637530:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554808704:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6318us (7608us total) trans 4294983262 rc 0/0 00000100:00100000:2.0:1789760784.637549:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15964 00000100:00100000:2.0:1789760784.637558:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727861754570 time 727876739757 state 0x2 ref 0 00000100:00100000:3.0:1789760784.644112:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760784.644272:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554808832 00000100:00100000:2.0:1789760784.644282:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554808832 00000100:00100000:2.0:1789760784.644332:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727861754570 time 727876739757 state 0x0 ref 1 00000100:00100000:2.0:1789760784.644347:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727877000173 time 727876739757 state 0x1 ref 1 00000100:00100000:2.0:1789760784.644413:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727877000173 time 727890031565 state 0x0 ref 1 00000100:00100000:2.0:1789760784.644426:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f60a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554808832:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760784.644472:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65347584, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f2a8 00010000:00010000:2.0:1789760784.644481:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5e74 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.644496:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760784.644501:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.644507:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5e74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65347584->65351679] (req 65347584->65351679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.644520:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5e74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65347584->65351679] (req 65347584->65351679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.649117:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983263 is committed 00002000:00010000:2.0:1789760784.649267:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65347584 -> 65351680 00002000:00010000:2.0:1789760784.649272:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127656 -> 127664 00010000:00010000:2.0:1789760784.649284:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5e74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65347584->65351679] (req 65347584->65351679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.649297:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5e74 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65347584->65351679] (req 65347584->65351679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.649307:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5e74 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65347584->65351679] (req 65347584->65351679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.649322:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760784.649327:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5e74 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65347584->65351679] (req 65347584->65351679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.649336:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5e74 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65347584->65351679] (req 65347584->65351679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.649356:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5e74 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65347584->65351679] (req 65347584->65351679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.649365:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5e74 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65347584->65351679] (req 65347584->65351679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.649503:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f60a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554808832:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5074us (5394us total) trans 4294983263 rc 0/0 00000100:00100000:2.0:1789760784.649513:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15965 00000100:00100000:2.0:1789760784.649522:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727877000173 time 727890031565 state 0x2 ref 0 00000100:00100000:1.0:1789760784.652553:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760784.652602:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554808960 00000100:00100000:2.0:1789760784.652614:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554808960 00000100:00100000:2.0:1789760784.652897:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727877000173 time 727890031565 state 0x0 ref 1 00000100:00100000:2.0:1789760784.652913:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727890291981 time 727890031565 state 0x1 ref 1 00000100:00100000:2.0:1789760784.652947:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727890291981 time 727898565547 state 0x0 ref 1 00000100:00100000:2.0:1789760784.652963:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554808960:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760784.653009:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65351680, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f2b0 00010000:00010000:2.0:1789760784.653018:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5e7b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.653043:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760784.653047:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.653061:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5e7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65351680->65355775] (req 65351680->65355775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.653075:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5e7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65351680->65355775] (req 65351680->65355775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.658059:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983264 is committed 00002000:00010000:2.0:1789760784.658795:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65351680 -> 65355776 00002000:00010000:2.0:1789760784.658800:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127664 -> 127672 00010000:00010000:2.0:1789760784.658812:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5e7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65351680->65355775] (req 65351680->65355775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.658826:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5e7b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65351680->65355775] (req 65351680->65355775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.658836:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5e7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65351680->65355775] (req 65351680->65355775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.658843:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760784.658848:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5e7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65351680->65355775] (req 65351680->65355775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.658857:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5e7b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65351680->65355775] (req 65351680->65355775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.658876:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5e7b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65351680->65355775] (req 65351680->65355775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.658883:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5e7b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65351680->65355775] (req 65351680->65355775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.658987:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554808960:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6021us (6437us total) trans 4294983264 rc 0/0 00000100:00100000:2.0:1789760784.658998:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15966 00000100:00100000:2.0:1789760784.659008:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727890291981 time 727898565547 state 0x2 ref 0 00000100:00100000:1.0:1789760784.662709:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760784.663907:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554809088 00000100:00100000:0.0:1789760784.663919:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554809088 00000100:00100000:0.0:1789760784.663980:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727890291981 time 727898565547 state 0x0 ref 1 00000100:00100000:0.0:1789760784.663995:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727898825963 time 727898565547 state 0x1 ref 1 00000100:00100000:0.0:1789760784.665352:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727898825963 time 727909643171 state 0x0 ref 1 00000100:00100000:0.0:1789760784.665407:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554809088:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760784.665693:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65355776, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f2b8 00010000:00010000:0.0:1789760784.665705:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e82 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.665771:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760784.665776:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.665782:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65355776->65359871] (req 65355776->65359871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.665796:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65355776->65359871] (req 65355776->65359871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.674691:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983265 is committed 00002000:00010000:1.0:1789760784.674810:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65355776 -> 65359872 00002000:00010000:1.0:1789760784.674815:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127672 -> 127680 00010000:00010000:1.0:1789760784.674829:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65355776->65359871] (req 65355776->65359871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.674842:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e82 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65355776->65359871] (req 65355776->65359871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.674851:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e82 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65355776->65359871] (req 65355776->65359871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.674859:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.674865:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e82 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65355776->65359871] (req 65355776->65359871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.674876:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e82 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65355776->65359871] (req 65355776->65359871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.674896:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e82 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65355776->65359871] (req 65355776->65359871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.674904:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c5e82 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65355776->65359871] (req 65355776->65359871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.675003:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554809088:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9628us (12297us total) trans 4294983265 rc 0/0 00000100:00100000:1.0:1789760784.675014:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15967 00000100:00100000:1.0:1789760784.675025:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727898825963 time 727909643171 state 0x2 ref 0 00000100:00100000:2.0:1789760784.678761:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760784.678898:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554809216 00000100:00100000:0.0:1789760784.678909:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554809216 00000100:00100000:0.0:1789760784.678962:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727898825963 time 727909643171 state 0x0 ref 1 00000100:00100000:0.0:1789760784.678977:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727909903587 time 727909643171 state 0x1 ref 1 00000100:00100000:0.0:1789760784.679007:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727909903587 time 727924625343 state 0x0 ref 1 00000100:00100000:0.0:1789760784.679019:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554809216:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760784.679074:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65359872, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f2c0 00010000:00010000:0.0:1789760784.679083:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5e89 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.679100:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760784.679104:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.679110:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5e89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65359872->65363967] (req 65359872->65363967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.679305:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5e89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65359872->65363967] (req 65359872->65363967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.684020:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983266 is committed 00002000:00010000:0.0:1789760784.686961:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65359872 -> 65363968 00002000:00010000:0.0:1789760784.686973:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127680 -> 127688 00010000:00010000:0.0:1789760784.686988:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5e89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65359872->65363967] (req 65359872->65363967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.687002:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5e89 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65359872->65363967] (req 65359872->65363967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.687011:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5e89 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65359872->65363967] (req 65359872->65363967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.687018:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760784.687022:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5e89 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65359872->65363967] (req 65359872->65363967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.687031:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5e89 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65359872->65363967] (req 65359872->65363967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.687069:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5e89 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65359872->65363967] (req 65359872->65363967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.687077:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5e89 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65359872->65363967] (req 65359872->65363967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.687243:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554809216:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8219us (8485us total) trans 4294983266 rc 0/0 00000100:00100000:0.0:1789760784.687267:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15968 00000100:00100000:0.0:1789760784.687277:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727909903587 time 727924625343 state 0x2 ref 0 00000100:00100000:3.0:1789760784.710632:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760784.711031:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554809344 00000100:00100000:1.0:1789760784.711053:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554809344 00000100:00100000:1.0:1789760784.711111:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727909903587 time 727924625343 state 0x0 ref 1 00000100:00100000:1.0:1789760784.711128:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727924885759 time 727924625343 state 0x1 ref 1 00000100:00100000:1.0:1789760784.711211:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727924885759 time 727956828138 state 0x0 ref 1 00000100:00100000:1.0:1789760784.711225:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe18c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554809344:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760784.711275:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65363968, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f2c8 00010000:00010000:1.0:1789760784.711284:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5e90 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.711300:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:1.0:1789760784.711305:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.711311:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5e90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65363968->65368063] (req 65363968->65368063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.711324:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5e90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65363968->65368063] (req 65363968->65368063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.716326:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983267 is committed 00002000:00010000:1.0:1789760784.716452:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65363968 -> 65368064 00002000:00010000:1.0:1789760784.716458:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127688 -> 127696 00010000:00010000:1.0:1789760784.716470:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5e90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65363968->65368063] (req 65363968->65368063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.716483:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5e90 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65363968->65368063] (req 65363968->65368063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.716492:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5e90 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65363968->65368063] (req 65363968->65368063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.716499:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.716504:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5e90 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65363968->65368063] (req 65363968->65368063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.716512:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5e90 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65363968->65368063] (req 65363968->65368063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.716533:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5e90 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65363968->65368063] (req 65363968->65368063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.716541:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5e90 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65363968->65368063] (req 65363968->65368063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.716829:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe18c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554809344:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5601us (6202us total) trans 4294983267 rc 0/0 00000100:00100000:1.0:1789760784.716839:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15969 00000100:00100000:1.0:1789760784.716849:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727924885759 time 727956828138 state 0x2 ref 0 00000100:00100000:1.0:1789760784.722435:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760784.722567:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554809472 00000100:00100000:2.0:1789760784.722577:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554809472 00000100:00100000:2.0:1789760784.722628:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727924885759 time 727956828138 state 0x0 ref 1 00000100:00100000:2.0:1789760784.722644:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727957088554 time 727956828138 state 0x1 ref 1 00000100:00100000:2.0:1789760784.722750:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727957088554 time 727968367136 state 0x0 ref 1 00000100:00100000:2.0:1789760784.722768:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554809472:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760784.722815:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65368064, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f2d0 00010000:00010000:2.0:1789760784.722824:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5e97 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.722840:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:2.0:1789760784.722844:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.722850:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5e97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65368064->65372159] (req 65368064->65372159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.722864:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5e97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65368064->65372159] (req 65368064->65372159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.726939:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983268 is committed 00002000:00010000:2.0:1789760784.727087:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65368064 -> 65372160 00002000:00010000:2.0:1789760784.727092:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127696 -> 127704 00010000:00010000:2.0:1789760784.727106:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5e97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65368064->65372159] (req 65368064->65372159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.727119:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5e97 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65368064->65372159] (req 65368064->65372159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.727259:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5e97 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65368064->65372159] (req 65368064->65372159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.727268:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760784.727273:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5e97 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65368064->65372159] (req 65368064->65372159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.727282:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5e97 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65368064->65372159] (req 65368064->65372159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.727302:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5e97 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65368064->65372159] (req 65368064->65372159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.727844:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5e97 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65368064->65372159] (req 65368064->65372159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.727984:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554809472:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5215us (5563us total) trans 4294983268 rc 0/0 00000100:00100000:2.0:1789760784.727993:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15970 00000100:00100000:2.0:1789760784.728002:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727957088554 time 727968367136 state 0x2 ref 0 00000100:00100000:2.0:1789760784.731532:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760784.731916:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554809600 00000100:00100000:0.0:1789760784.731926:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554809600 00000100:00100000:0.0:1789760784.731969:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727957088554 time 727968367136 state 0x0 ref 1 00000100:00100000:0.0:1789760784.731981:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727968627552 time 727968367136 state 0x1 ref 1 00000100:00100000:0.0:1789760784.732011:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727968627552 time 727977629302 state 0x0 ref 1 00000100:00100000:0.0:1789760784.732024:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554809600:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760784.732083:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65372160, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f2d8 00010000:00010000:0.0:1789760784.732091:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5e9e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.732111:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760784.732115:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.732121:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5e9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65372160->65376255] (req 65372160->65376255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.732143:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5e9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65372160->65376255] (req 65372160->65376255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.737823:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983269 is committed 00002000:00010000:0.0:1789760784.738180:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65372160 -> 65376256 00002000:00010000:0.0:1789760784.738185:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127704 -> 127712 00010000:00010000:0.0:1789760784.738198:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5e9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65372160->65376255] (req 65372160->65376255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.738222:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5e9e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65372160->65376255] (req 65372160->65376255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.738231:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5e9e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65372160->65376255] (req 65372160->65376255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.738238:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760784.738242:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5e9e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65372160->65376255] (req 65372160->65376255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.738252:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5e9e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65372160->65376255] (req 65372160->65376255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.738276:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5e9e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65372160->65376255] (req 65372160->65376255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.738283:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5e9e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65372160->65376255] (req 65372160->65376255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.738374:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554809600:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6348us (6849us total) trans 4294983269 rc 0/0 00000100:00100000:0.0:1789760784.738440:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15971 00000100:00100000:0.0:1789760784.738449:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727968627552 time 727977629302 state 0x2 ref 0 00000100:00100000:2.0:1789760784.744880:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760784.744975:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554809728 00000100:00100000:0.0:1789760784.744984:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554809728 00000100:00100000:0.0:1789760784.745039:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727968627552 time 727977629302 state 0x0 ref 1 00000100:00100000:0.0:1789760784.745064:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727977889718 time 727977629302 state 0x1 ref 1 00000100:00100000:0.0:1789760784.745092:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727977889718 time 727990711316 state 0x0 ref 1 00000100:00100000:0.0:1789760784.745104:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554809728:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760784.745162:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65376256, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f2e0 00010000:00010000:0.0:1789760784.745171:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5ea5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.745186:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=727s v=(0 0) 00010000:00010000:0.0:1789760784.745191:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.745197:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5ea5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65376256->65380351] (req 65376256->65380351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.745211:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5ea5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65376256->65380351] (req 65376256->65380351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.756603:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983270 is committed 00002000:00010000:1.0:1789760784.756743:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65376256 -> 65380352 00002000:00010000:1.0:1789760784.756749:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127712 -> 127720 00010000:00010000:1.0:1789760784.756762:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5ea5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65376256->65380351] (req 65376256->65380351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.756775:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5ea5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65376256->65380351] (req 65376256->65380351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.756785:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5ea5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65376256->65380351] (req 65376256->65380351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.756793:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.756798:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5ea5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65376256->65380351] (req 65376256->65380351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.756807:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5ea5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65376256->65380351] (req 65376256->65380351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.756828:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5ea5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65376256->65380351] (req 65376256->65380351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.756837:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c5ea5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65376256->65380351] (req 65376256->65380351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.756933:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554809728:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11825us (12058us total) trans 4294983270 rc 0/0 00000100:00100000:1.0:1789760784.756943:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15972 00000100:00100000:1.0:1789760784.756953:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727977889718 time 727990711316 state 0x2 ref 0 00000100:00100000:1.0:1789760784.767579:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760784.767899:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554809856 00000100:00100000:2.0:1789760784.767909:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554809856 00000100:00100000:2.0:1789760784.767964:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727977889718 time 727990711316 state 0x0 ref 1 00000100:00100000:2.0:1789760784.767980:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727990971732 time 727990711316 state 0x1 ref 1 00000100:00100000:2.0:1789760784.768061:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727990971732 time 728013673107 state 0x0 ref 1 00000100:00100000:2.0:1789760784.768075:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554809856:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760784.768121:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65380352, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f2e8 00010000:00010000:2.0:1789760784.768130:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5eac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.768146:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760784.768151:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760784.768158:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5eac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65380352->65384447] (req 65380352->65384447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.768171:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5eac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65380352->65384447] (req 65380352->65384447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.779017:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983271 is committed 00002000:00010000:2.0:1789760784.780297:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65380352 -> 65384448 00002000:00010000:2.0:1789760784.780304:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127720 -> 127728 00010000:00010000:2.0:1789760784.780317:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5eac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65380352->65384447] (req 65380352->65384447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.780330:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5eac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65380352->65384447] (req 65380352->65384447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.780339:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5eac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65380352->65384447] (req 65380352->65384447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.780346:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760784.780353:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5eac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65380352->65384447] (req 65380352->65384447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.780362:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5eac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65380352->65384447] (req 65380352->65384447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.780418:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5eac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65380352->65384447] (req 65380352->65384447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.780427:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5eac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65380352->65384447] (req 65380352->65384447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.780530:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554809856:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12452us (12955us total) trans 4294983271 rc 0/0 00000100:00100000:2.0:1789760784.780704:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15973 00000100:00100000:2.0:1789760784.780713:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727990971732 time 728013673107 state 0x2 ref 0 00000100:00100000:3.0:1789760784.785700:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789760784.785725:0:3649:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb036c6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3649:1876699573918592:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760784.785797:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760784.785877:0:8439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573918592 00000100:00100000:3.0:1789760784.785885:0:8439:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573918592 00000100:00100000:3.0:1789760784.785936:0:8439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 364 00000100:00100000:3.0:1789760784.785946:0:8439:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f60e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3649:x1876699573918592:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789760784.785982:0:8439:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484916928 pending 0 free 3858386944 avail 3625086976 00000100:00100000:3.0:1789760784.786024:0:8439:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f60e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3649:x1876699573918592:12345-0@lo:13:osp-pre-0-0.0 Request processed in 80us (226us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760784.786030:0:8439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 364 00000100:00080000:3.0:1789760784.786049:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789760784.786089:0:3649:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb036c6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3649:1876699573918592:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789760784.786545:0:3648:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789760784.786557:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb036c7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3648:1876699573918720:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1789760784.786592:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760784.786699:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573918720 00000100:00100000:2.0:1789760784.786704:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573918720 00000100:00100000:2.0:1789760784.786751:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 365 00000100:00100000:2.0:1789760784.786759:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3648:x1876699573918720:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1789760784.786771:0:8440:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484916928 pending 0 free 3858386944 avail 3625086976 00000100:00100000:2.0:1789760784.786815:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3648:x1876699573918720:12345-0@lo:13:osp-pre-0-1.0 Request processed in 58us (223us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760784.786822:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 365 00000100:00080000:0.0:1789760784.786976:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760784.787183:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb036c7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3648:1876699573918720:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1789760784.792924:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760784.792984:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554809984 00000100:00100000:1.0:1789760784.792995:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554809984 00000100:00100000:1.0:1789760784.793061:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 727990971732 time 728013673107 state 0x0 ref 1 00000100:00100000:1.0:1789760784.793076:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728013933523 time 728013673107 state 0x1 ref 1 00000100:00100000:1.0:1789760784.793106:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728013933523 time 728038724910 state 0x0 ref 1 00000100:00100000:1.0:1789760784.793118:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554809984:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760784.793165:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65384448, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f2f0 00010000:00010000:1.0:1789760784.793174:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5eb3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.793189:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760784.793193:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.793199:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5eb3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65384448->65388543] (req 65384448->65388543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.793212:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5eb3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65384448->65388543] (req 65384448->65388543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.800429:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983272 is committed 00002000:00010000:1.0:1789760784.800672:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65384448 -> 65388544 00002000:00010000:1.0:1789760784.800678:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127728 -> 127736 00010000:00010000:1.0:1789760784.800692:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5eb3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65384448->65388543] (req 65384448->65388543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.800706:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5eb3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65384448->65388543] (req 65384448->65388543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.800715:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5eb3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65384448->65388543] (req 65384448->65388543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.800726:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.800730:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5eb3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65384448->65388543] (req 65384448->65388543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.800738:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5eb3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65384448->65388543] (req 65384448->65388543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.800757:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5eb3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65384448->65388543] (req 65384448->65388543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.800765:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5eb3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65384448->65388543] (req 65384448->65388543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.800998:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554809984:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7876us (8081us total) trans 4294983272 rc 0/0 00000100:00100000:1.0:1789760784.801008:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15974 00000100:00100000:1.0:1789760784.801017:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728013933523 time 728038724910 state 0x2 ref 0 00000100:00100000:1.0:1789760784.810860:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760784.811052:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554810112 00000100:00100000:0.0:1789760784.811063:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554810112 00000100:00100000:0.0:1789760784.811121:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728013933523 time 728038724910 state 0x0 ref 1 00000100:00100000:0.0:1789760784.811138:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728038985326 time 728038724910 state 0x1 ref 1 00000100:00100000:0.0:1789760784.811173:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728038985326 time 728056791951 state 0x0 ref 1 00000100:00100000:0.0:1789760784.811186:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554810112:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760784.811230:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65388544, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f2f8 00010000:00010000:0.0:1789760784.811435:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5eba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.811451:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760784.811455:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.811461:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5eba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65388544->65392639] (req 65388544->65392639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.811474:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5eba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65388544->65392639] (req 65388544->65392639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.822569:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983273 is committed 00002000:00010000:0.0:1789760784.822694:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65388544 -> 65392640 00002000:00010000:0.0:1789760784.822699:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127736 -> 127744 00010000:00010000:0.0:1789760784.822712:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5eba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65388544->65392639] (req 65388544->65392639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.822725:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5eba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65388544->65392639] (req 65388544->65392639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.822734:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5eba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65388544->65392639] (req 65388544->65392639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.822742:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760784.822746:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5eba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65388544->65392639] (req 65388544->65392639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.822755:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5eba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65388544->65392639] (req 65388544->65392639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.822775:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5eba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65388544->65392639] (req 65388544->65392639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.822783:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5eba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65388544->65392639] (req 65388544->65392639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.823001:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554810112:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11813us (12145us total) trans 4294983273 rc 0/0 00000100:00100000:0.0:1789760784.823012:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15975 00000100:00100000:0.0:1789760784.823021:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728038985326 time 728056791951 state 0x2 ref 0 00000100:00100000:2.0:1789760784.836257:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760784.836362:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554810240 00000100:00100000:1.0:1789760784.836372:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554810240 00000100:00100000:1.0:1789760784.836472:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728038985326 time 728056791951 state 0x0 ref 1 00000100:00100000:1.0:1789760784.836489:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728057052367 time 728056791951 state 0x1 ref 1 00000100:00100000:1.0:1789760784.836519:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728057052367 time 728082137301 state 0x0 ref 1 00000100:00100000:1.0:1789760784.836531:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554810240:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760784.836578:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65392640, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f300 00010000:00010000:1.0:1789760784.836588:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5ec1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.836604:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760784.836608:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.836614:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5ec1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65392640->65396735] (req 65392640->65396735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.836628:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5ec1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65392640->65396735] (req 65392640->65396735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.849228:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983274 is committed 00002000:00010000:1.0:1789760784.849440:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65392640 -> 65396736 00002000:00010000:1.0:1789760784.849445:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127744 -> 127752 00010000:00010000:1.0:1789760784.849458:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5ec1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65392640->65396735] (req 65392640->65396735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.849470:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5ec1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65392640->65396735] (req 65392640->65396735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.849479:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5ec1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65392640->65396735] (req 65392640->65396735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.849486:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.849492:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5ec1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65392640->65396735] (req 65392640->65396735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.849500:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5ec1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65392640->65396735] (req 65392640->65396735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.849519:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5ec1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65392640->65396735] (req 65392640->65396735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.849527:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c5ec1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65392640->65396735] (req 65392640->65396735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.849621:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554810240:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13088us (13370us total) trans 4294983274 rc 0/0 00000100:00100000:1.0:1789760784.849632:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15976 00000100:00100000:1.0:1789760784.849736:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728057052367 time 728082137301 state 0x2 ref 0 00000100:00100000:2.0:1789760784.855824:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760784.855908:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554810368 00000100:00100000:1.0:1789760784.855918:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554810368 00000100:00100000:1.0:1789760784.855974:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728057052367 time 728082137301 state 0x0 ref 1 00000100:00100000:1.0:1789760784.855990:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728082397717 time 728082137301 state 0x1 ref 1 00000100:00100000:1.0:1789760784.856021:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728082397717 time 728101639719 state 0x0 ref 1 00000100:00100000:1.0:1789760784.856032:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554810368:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760784.856080:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65396736, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f308 00010000:00010000:1.0:1789760784.856090:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5ec8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.856106:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760784.856111:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.856119:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5ec8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65396736->65400831] (req 65396736->65400831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.856134:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5ec8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65396736->65400831] (req 65396736->65400831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760784.862700:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983275 is committed 00002000:00010000:2.0:1789760784.862849:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65396736 -> 65400832 00002000:00010000:2.0:1789760784.862854:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127752 -> 127760 00010000:00010000:2.0:1789760784.862867:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5ec8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65396736->65400831] (req 65396736->65400831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.863226:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5ec8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65396736->65400831] (req 65396736->65400831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.863310:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5ec8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65396736->65400831] (req 65396736->65400831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.863319:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760784.863325:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5ec8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65396736->65400831] (req 65396736->65400831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.863334:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5ec8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65396736->65400831] (req 65396736->65400831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.863354:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5ec8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65396736->65400831] (req 65396736->65400831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760784.863362:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c5ec8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65396736->65400831] (req 65396736->65400831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760784.863571:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554810368:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7535us (7749us total) trans 4294983275 rc 0/0 00000100:00100000:2.0:1789760784.863582:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15977 00000100:00100000:2.0:1789760784.863592:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728082397717 time 728101639719 state 0x2 ref 0 00000100:00100000:2.0:1789760784.872326:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760784.872511:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554810496 00000100:00100000:1.0:1789760784.872520:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554810496 00000100:00100000:1.0:1789760784.872570:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728082397717 time 728101639719 state 0x0 ref 1 00000100:00100000:1.0:1789760784.872583:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728101900135 time 728101639719 state 0x1 ref 1 00000100:00100000:1.0:1789760784.872642:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728101900135 time 728118260564 state 0x0 ref 1 00000100:00100000:1.0:1789760784.872654:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554810496:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760784.872716:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65400832, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f310 00010000:00010000:1.0:1789760784.872725:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ecf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.872741:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760784.872745:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.872752:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ecf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65400832->65404927] (req 65400832->65404927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.872765:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ecf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65400832->65404927] (req 65400832->65404927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.888842:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983276 is committed 00002000:00010000:1.0:1789760784.888946:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65400832 -> 65404928 00002000:00010000:1.0:1789760784.888951:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127760 -> 127768 00010000:00010000:1.0:1789760784.888963:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ecf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65400832->65404927] (req 65400832->65404927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.888976:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ecf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65400832->65404927] (req 65400832->65404927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.888985:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ecf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65400832->65404927] (req 65400832->65404927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.888992:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.888997:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ecf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65400832->65404927] (req 65400832->65404927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.889015:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ecf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65400832->65404927] (req 65400832->65404927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.889041:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ecf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65400832->65404927] (req 65400832->65404927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.889056:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ecf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65400832->65404927] (req 65400832->65404927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.889148:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554810496:12345-192.168.203.22@tcp:4:dd.0 Request processed in 16492us (16828us total) trans 4294983276 rc 0/0 00000100:00100000:1.0:1789760784.889158:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15978 00000100:00100000:1.0:1789760784.889168:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728101900135 time 728118260564 state 0x2 ref 0 00000100:00100000:1.0:1789760784.895410:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760784.895491:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554810624 00000100:00100000:0.0:1789760784.895500:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554810624 00000100:00100000:0.0:1789760784.895548:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728101900135 time 728118260564 state 0x0 ref 1 00000100:00100000:0.0:1789760784.895562:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728118520980 time 728118260564 state 0x1 ref 1 00000100:00100000:0.0:1789760784.895631:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728118520980 time 728141249810 state 0x0 ref 1 00000100:00100000:0.0:1789760784.895643:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554810624:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760784.895706:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65404928, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f318 00010000:00010000:0.0:1789760784.895715:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5ed6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.895730:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760784.895735:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.895740:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5ed6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65404928->65409023] (req 65404928->65409023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.895753:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5ed6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65404928->65409023] (req 65404928->65409023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760784.906862:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983277 is committed 00002000:00010000:0.0:1789760784.908000:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65404928 -> 65409024 00002000:00010000:0.0:1789760784.908008:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127768 -> 127776 00010000:00010000:0.0:1789760784.908023:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5ed6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65404928->65409023] (req 65404928->65409023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.908036:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5ed6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65404928->65409023] (req 65404928->65409023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.908047:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5ed6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65404928->65409023] (req 65404928->65409023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.908054:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760784.908059:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5ed6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65404928->65409023] (req 65404928->65409023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.908068:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5ed6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65404928->65409023] (req 65404928->65409023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.908087:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5ed6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65404928->65409023] (req 65404928->65409023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.908095:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5ed6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65404928->65409023] (req 65404928->65409023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.908195:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554810624:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12549us (12822us total) trans 4294983277 rc 0/0 00000100:00100000:0.0:1789760784.908206:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15979 00000100:00100000:0.0:1789760784.908216:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728118520980 time 728141249810 state 0x2 ref 0 00000100:00100000:2.0:1789760784.912230:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760784.912334:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554810752 00000100:00100000:1.0:1789760784.912344:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554810752 00000100:00100000:1.0:1789760784.912420:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728118520980 time 728141249810 state 0x0 ref 1 00000100:00100000:1.0:1789760784.912435:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728141510226 time 728141249810 state 0x1 ref 1 00000100:00100000:1.0:1789760784.912456:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728141510226 time 728158075817 state 0x0 ref 1 00000100:00100000:1.0:1789760784.912468:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554810752:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760784.912515:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65409024, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f320 00010000:00010000:1.0:1789760784.912524:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5edd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.913096:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760784.913101:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.913108:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5edd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65409024->65413119] (req 65409024->65413119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.913124:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5edd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65409024->65413119] (req 65409024->65413119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.926683:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983278 is committed 00002000:00010000:1.0:1789760784.926809:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65409024 -> 65413120 00002000:00010000:1.0:1789760784.926814:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127776 -> 127784 00010000:00010000:1.0:1789760784.926827:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5edd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65409024->65413119] (req 65409024->65413119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.926839:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5edd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65409024->65413119] (req 65409024->65413119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.926848:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5edd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65409024->65413119] (req 65409024->65413119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.926856:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.926860:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5edd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65409024->65413119] (req 65409024->65413119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.926869:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5edd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65409024->65413119] (req 65409024->65413119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.926889:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5edd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65409024->65413119] (req 65409024->65413119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.926898:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5edd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65409024->65413119] (req 65409024->65413119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.927033:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554810752:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14561us (14806us total) trans 4294983278 rc 0/0 00000100:00100000:1.0:1789760784.927052:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15980 00000100:00100000:1.0:1789760784.927061:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728141510226 time 728158075817 state 0x2 ref 0 00000100:00100000:2.0:1789760784.937117:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760784.937247:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554810880 00000100:00100000:1.0:1789760784.937257:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554810880 00000100:00100000:1.0:1789760784.937317:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728141510226 time 728158075817 state 0x0 ref 1 00000100:00100000:1.0:1789760784.937333:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728158336233 time 728158075817 state 0x1 ref 1 00000100:00100000:1.0:1789760784.937562:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728158336233 time 728183179498 state 0x0 ref 1 00000100:00100000:1.0:1789760784.937576:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554810880:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760784.937624:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65413120, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f328 00010000:00010000:1.0:1789760784.937634:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ee4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.937649:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760784.937654:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.937670:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ee4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65413120->65417215] (req 65413120->65417215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.937685:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ee4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65413120->65417215] (req 65413120->65417215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760784.944778:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983279 is committed 00002000:00010000:1.0:1789760784.945022:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65413120 -> 65417216 00002000:00010000:1.0:1789760784.945027:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127784 -> 127792 00010000:00010000:1.0:1789760784.945050:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ee4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65413120->65417215] (req 65413120->65417215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.945064:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ee4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65413120->65417215] (req 65413120->65417215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.945073:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ee4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65413120->65417215] (req 65413120->65417215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.945080:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.945085:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ee4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65413120->65417215] (req 65413120->65417215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.945093:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ee4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65413120->65417215] (req 65413120->65417215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.945111:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ee4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65413120->65417215] (req 65413120->65417215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.945118:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5ee4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65413120->65417215] (req 65413120->65417215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.945474:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554810880:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7893us (8366us total) trans 4294983279 rc 0/0 00000100:00100000:1.0:1789760784.945486:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15981 00000100:00100000:1.0:1789760784.945498:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728158336233 time 728183179498 state 0x2 ref 0 00000100:00100000:2.0:1789760784.952901:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760784.952974:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554811008 00000100:00100000:1.0:1789760784.952985:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554811008 00000100:00100000:1.0:1789760784.953038:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728158336233 time 728183179498 state 0x0 ref 1 00000100:00100000:1.0:1789760784.953054:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728183439914 time 728183179498 state 0x1 ref 1 00000100:00100000:1.0:1789760784.953085:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728183439914 time 728198703807 state 0x0 ref 1 00000100:00100000:1.0:1789760784.953097:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554811008:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760784.953147:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65417216, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f330 00010000:00010000:1.0:1789760784.953157:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5eeb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.953173:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760784.953177:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.953183:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5eeb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65417216->65421311] (req 65417216->65421311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.953197:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5eeb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65417216->65421311] (req 65417216->65421311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.961309:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983280 is committed 00002000:00010000:1.0:1789760784.961595:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65417216 -> 65421312 00002000:00010000:1.0:1789760784.961601:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127792 -> 127800 00010000:00010000:1.0:1789760784.961614:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5eeb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65417216->65421311] (req 65417216->65421311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.961632:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5eeb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65417216->65421311] (req 65417216->65421311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.961643:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5eeb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65417216->65421311] (req 65417216->65421311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.961650:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.961656:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5eeb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65417216->65421311] (req 65417216->65421311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.961677:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5eeb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65417216->65421311] (req 65417216->65421311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.961697:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5eeb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65417216->65421311] (req 65417216->65421311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.961706:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5eeb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65417216->65421311] (req 65417216->65421311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.962111:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554811008:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9010us (9215us total) trans 4294983280 rc 0/0 00000100:00100000:1.0:1789760784.962123:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15982 00000100:00100000:1.0:1789760784.962133:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728183439914 time 728198703807 state 0x2 ref 0 00000100:00100000:2.0:1789760784.976119:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760784.976316:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554811136 00000100:00100000:0.0:1789760784.976326:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554811136 00000100:00100000:0.0:1789760784.976398:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728183439914 time 728198703807 state 0x0 ref 1 00000100:00100000:0.0:1789760784.976414:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728198964223 time 728198703807 state 0x1 ref 1 00000100:00100000:0.0:1789760784.976492:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728198964223 time 728222110552 state 0x0 ref 1 00000100:00100000:0.0:1789760784.976506:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554811136:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760784.976555:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65421312, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f338 00010000:00010000:0.0:1789760784.976564:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5ef2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.976580:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760784.976585:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760784.976591:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5ef2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65421312->65425407] (req 65421312->65425407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.976605:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5ef2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65421312->65425407] (req 65421312->65425407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.978255:0:6509:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760784.982763:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983281 is committed 00002000:00010000:0.0:1789760784.983057:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65421312 -> 65425408 00002000:00010000:0.0:1789760784.983063:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127800 -> 127808 00010000:00010000:0.0:1789760784.983079:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5ef2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65421312->65425407] (req 65421312->65425407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.983093:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5ef2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65421312->65425407] (req 65421312->65425407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.983103:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5ef2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65421312->65425407] (req 65421312->65425407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.983110:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760784.983115:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5ef2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65421312->65425407] (req 65421312->65425407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.983124:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5ef2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65421312->65425407] (req 65421312->65425407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.983153:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5ef2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65421312->65425407] (req 65421312->65425407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760784.983161:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5ef2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65421312->65425407] (req 65421312->65425407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760784.983491:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554811136:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6981us (7376us total) trans 4294983281 rc 0/0 00000100:00100000:0.0:1789760784.983505:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15983 00000100:00100000:0.0:1789760784.983519:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728198964223 time 728222110552 state 0x2 ref 0 00000100:00100000:2.0:1789760784.990733:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760784.991225:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554811264 00000100:00100000:1.0:1789760784.991236:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554811264 00000100:00100000:1.0:1789760784.991293:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728198964223 time 728222110552 state 0x0 ref 1 00000100:00100000:1.0:1789760784.991327:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728222370968 time 728222110552 state 0x1 ref 1 00000100:00100000:1.0:1789760784.991359:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728222370968 time 728236977258 state 0x0 ref 1 00000100:00100000:1.0:1789760784.991372:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554811264:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760784.991469:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65425408, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f340 00010000:00010000:1.0:1789760784.991480:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5ef9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.991498:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760784.991503:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760784.991511:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5ef9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65425408->65429503] (req 65425408->65429503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.991526:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5ef9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65425408->65429503] (req 65425408->65429503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760784.998616:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983282 is committed 00002000:00010000:1.0:1789760784.998882:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65425408 -> 65429504 00002000:00010000:1.0:1789760784.998888:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127808 -> 127816 00010000:00010000:1.0:1789760784.998902:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5ef9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65425408->65429503] (req 65425408->65429503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.998919:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5ef9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65425408->65429503] (req 65425408->65429503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.998931:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5ef9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65425408->65429503] (req 65425408->65429503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.998937:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760784.998941:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5ef9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65425408->65429503] (req 65425408->65429503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.998950:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5ef9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65425408->65429503] (req 65425408->65429503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.998969:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5ef9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65425408->65429503] (req 65425408->65429503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760784.998976:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5ef9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65425408->65429503] (req 65425408->65429503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760784.999294:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554811264:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7919us (8567us total) trans 4294983282 rc 0/0 00000100:00100000:1.0:1789760784.999304:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15984 00000100:00100000:1.0:1789760784.999313:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728222370968 time 728236977258 state 0x2 ref 0 00000100:00100000:2.0:1789760785.016751:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760785.016849:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554811392 00000100:00100000:1.0:1789760785.016859:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554811392 00000100:00100000:1.0:1789760785.016929:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728222370968 time 728236977258 state 0x0 ref 1 00000100:00100000:1.0:1789760785.016943:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728237237674 time 728236977258 state 0x1 ref 1 00000100:00100000:1.0:1789760785.016977:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728237237674 time 728262595626 state 0x0 ref 1 00000100:00100000:1.0:1789760785.016989:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554811392:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760785.017035:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65429504, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f348 00010000:00010000:1.0:1789760785.017044:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f00 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.017060:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760785.017064:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.017070:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65429504->65433599] (req 65429504->65433599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.017083:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65429504->65433599] (req 65429504->65433599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760785.017503:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789760785.022972:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983283 is committed 00002000:00010000:1.0:1789760785.023285:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65429504 -> 65433600 00002000:00010000:1.0:1789760785.023440:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127816 -> 127824 00010000:00010000:1.0:1789760785.023458:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65429504->65433599] (req 65429504->65433599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.023470:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f00 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65429504->65433599] (req 65429504->65433599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.023486:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f00 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65429504->65433599] (req 65429504->65433599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.023493:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760785.023499:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f00 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65429504->65433599] (req 65429504->65433599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.023509:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f00 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65429504->65433599] (req 65429504->65433599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.023530:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f00 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65429504->65433599] (req 65429504->65433599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.023538:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f00 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65429504->65433599] (req 65429504->65433599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.023635:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554811392:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6642us (6887us total) trans 4294983283 rc 0/0 00000100:00100000:1.0:1789760785.023645:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15985 00000100:00100000:1.0:1789760785.023654:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728237237674 time 728262595626 state 0x2 ref 0 00000100:00100000:2.0:1789760785.031283:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760785.031456:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554811520 00000100:00100000:1.0:1789760785.031467:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554811520 00000100:00100000:1.0:1789760785.031520:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728237237674 time 728262595626 state 0x0 ref 1 00000100:00100000:1.0:1789760785.031535:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728262856042 time 728262595626 state 0x1 ref 1 00000100:00100000:1.0:1789760785.031569:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728262856042 time 728277186505 state 0x0 ref 1 00000100:00100000:1.0:1789760785.031582:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554811520:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760785.031629:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65433600, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f350 00010000:00010000:1.0:1789760785.031639:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f07 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.031653:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760785.031667:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.031673:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65433600->65437695] (req 65433600->65437695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.031687:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65433600->65437695] (req 65433600->65437695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.038453:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983284 is committed 00002000:00010000:1.0:1789760785.044283:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65433600 -> 65437696 00002000:00010000:1.0:1789760785.044293:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127824 -> 127832 00010000:00010000:1.0:1789760785.044307:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65433600->65437695] (req 65433600->65437695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.044320:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f07 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65433600->65437695] (req 65433600->65437695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.044329:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f07 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65433600->65437695] (req 65433600->65437695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.044341:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760785.044347:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f07 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65433600->65437695] (req 65433600->65437695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.044356:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f07 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65433600->65437695] (req 65433600->65437695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.044392:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f07 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65433600->65437695] (req 65433600->65437695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.044401:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f07 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65433600->65437695] (req 65433600->65437695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.044607:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554811520:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13023us (13329us total) trans 4294983284 rc 0/0 00000100:00100000:1.0:1789760785.044619:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15986 00000100:00100000:1.0:1789760785.044628:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728262856042 time 728277186505 state 0x2 ref 0 00000100:00100000:2.0:1789760785.061622:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760785.061899:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554811648 00000100:00100000:0.0:1789760785.061908:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554811648 00000100:00100000:0.0:1789760785.061963:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728262856042 time 728277186505 state 0x0 ref 1 00000100:00100000:0.0:1789760785.061978:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728277446921 time 728277186505 state 0x1 ref 1 00000100:00100000:0.0:1789760785.062072:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728277446921 time 728307689382 state 0x0 ref 1 00000100:00100000:0.0:1789760785.062086:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554811648:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760785.062134:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65437696, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f358 00010000:00010000:0.0:1789760785.062144:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5f0e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.062160:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760785.062165:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.062171:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5f0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65437696->65441791] (req 65437696->65441791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.062185:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5f0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65437696->65441791] (req 65437696->65441791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.073573:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983285 is committed 00002000:00010000:0.0:1789760785.073772:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65437696 -> 65441792 00002000:00010000:0.0:1789760785.073778:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127832 -> 127840 00010000:00010000:0.0:1789760785.073791:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5f0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65437696->65441791] (req 65437696->65441791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.073804:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5f0e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65437696->65441791] (req 65437696->65441791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.073813:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5f0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65437696->65441791] (req 65437696->65441791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.073821:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760785.073826:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5f0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65437696->65441791] (req 65437696->65441791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.073836:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5f0e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65437696->65441791] (req 65437696->65441791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.073856:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5f0e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65437696->65441791] (req 65437696->65441791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.073863:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5f0e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65437696->65441791] (req 65437696->65441791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.073955:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554811648:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11866us (12338us total) trans 4294983285 rc 0/0 00000100:00100000:0.0:1789760785.073966:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15987 00000100:00100000:0.0:1789760785.073976:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728277446921 time 728307689382 state 0x2 ref 0 00000100:00100000:2.0:1789760785.077832:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760785.077931:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554811776 00000100:00100000:1.0:1789760785.077941:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554811776 00000100:00100000:1.0:1789760785.077994:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728277446921 time 728307689382 state 0x0 ref 1 00000100:00100000:1.0:1789760785.078009:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728307949798 time 728307689382 state 0x1 ref 1 00000100:00100000:1.0:1789760785.078041:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728307949798 time 728323659634 state 0x0 ref 1 00000100:00100000:1.0:1789760785.078058:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554811776:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760785.078103:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65441792, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f360 00010000:00010000:1.0:1789760785.078113:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f15 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.078129:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760785.078134:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.078141:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65441792->65445887] (req 65441792->65445887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.078155:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65441792->65445887] (req 65441792->65445887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.086163:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983286 is committed 00002000:00010000:1.0:1789760785.086342:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65441792 -> 65445888 00002000:00010000:1.0:1789760785.086348:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127840 -> 127848 00010000:00010000:1.0:1789760785.086360:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65441792->65445887] (req 65441792->65445887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.086374:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f15 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65441792->65445887] (req 65441792->65445887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.086423:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65441792->65445887] (req 65441792->65445887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.086431:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760785.086436:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65441792->65445887] (req 65441792->65445887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.086445:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f15 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65441792->65445887] (req 65441792->65445887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.086464:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f15 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65441792->65445887] (req 65441792->65445887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.086474:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f15 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65441792->65445887] (req 65441792->65445887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.086568:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554811776:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8506us (8737us total) trans 4294983286 rc 0/0 00000100:00100000:1.0:1789760785.086578:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15988 00000100:00100000:1.0:1789760785.086588:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728307949798 time 728323659634 state 0x2 ref 0 00000100:00100000:2.0:1789760785.097598:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760785.097750:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554811904 00000100:00100000:1.0:1789760785.097761:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554811904 00000100:00100000:1.0:1789760785.097818:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728307949798 time 728323659634 state 0x0 ref 1 00000100:00100000:1.0:1789760785.097834:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728323920050 time 728323659634 state 0x1 ref 1 00000100:00100000:1.0:1789760785.097864:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728323920050 time 728343482849 state 0x0 ref 1 00000100:00100000:1.0:1789760785.097877:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554811904:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760785.097928:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65445888, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f368 00010000:00010000:1.0:1789760785.097938:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f1c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.097955:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760785.097960:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.097967:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65445888->65449983] (req 65445888->65449983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.097983:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65445888->65449983] (req 65445888->65449983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.105938:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983287 is committed 00002000:00010000:0.0:1789760785.106101:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65445888 -> 65449984 00002000:00010000:0.0:1789760785.106107:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127848 -> 127856 00010000:00010000:0.0:1789760785.106119:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65445888->65449983] (req 65445888->65449983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.106132:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f1c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65445888->65449983] (req 65445888->65449983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.106142:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f1c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65445888->65449983] (req 65445888->65449983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.106149:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760785.106155:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f1c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65445888->65449983] (req 65445888->65449983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.106164:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f1c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65445888->65449983] (req 65445888->65449983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.106192:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f1c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65445888->65449983] (req 65445888->65449983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.106200:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c5f1c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65445888->65449983] (req 65445888->65449983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.106284:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554811904:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8406us (8691us total) trans 4294983287 rc 0/0 00000100:00100000:0.0:1789760785.106293:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15989 00000100:00100000:0.0:1789760785.106302:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728323920050 time 728343482849 state 0x2 ref 0 00000100:00100000:2.0:1789760785.109828:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760785.110019:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554812032 00000100:00100000:1.0:1789760785.110030:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554812032 00000100:00100000:1.0:1789760785.110134:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728323920050 time 728343482849 state 0x0 ref 1 00000100:00100000:1.0:1789760785.110149:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728343743265 time 728343482849 state 0x1 ref 1 00000100:00100000:1.0:1789760785.110270:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728343743265 time 728355887855 state 0x0 ref 1 00000100:00100000:1.0:1789760785.110283:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554812032:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760785.110335:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65449984, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f370 00010000:00010000:1.0:1789760785.110345:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5f23 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.110364:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760785.110368:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.110375:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5f23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65449984->65454079] (req 65449984->65454079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.110520:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5f23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65449984->65454079] (req 65449984->65454079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.119958:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983288 is committed 00002000:00010000:1.0:1789760785.120152:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65449984 -> 65454080 00002000:00010000:1.0:1789760785.120157:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127856 -> 127864 00010000:00010000:1.0:1789760785.120170:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5f23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65449984->65454079] (req 65449984->65454079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.120182:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5f23 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65449984->65454079] (req 65449984->65454079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.120191:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5f23 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65449984->65454079] (req 65449984->65454079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.120199:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760785.120204:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5f23 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65449984->65454079] (req 65449984->65454079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.120213:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5f23 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65449984->65454079] (req 65449984->65454079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.120232:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5f23 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65449984->65454079] (req 65449984->65454079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.120241:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c5f23 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65449984->65454079] (req 65449984->65454079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.120339:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554812032:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10053us (10517us total) trans 4294983288 rc 0/0 00000100:00100000:1.0:1789760785.120350:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15990 00000100:00100000:1.0:1789760785.120360:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728343743265 time 728355887855 state 0x2 ref 0 00000100:00100000:2.0:1789760785.130030:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760785.130156:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554812160 00000100:00100000:0.0:1789760785.130167:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554812160 00000100:00100000:0.0:1789760785.130222:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728343743265 time 728355887855 state 0x0 ref 1 00000100:00100000:0.0:1789760785.130239:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728356148271 time 728355887855 state 0x1 ref 1 00000100:00100000:0.0:1789760785.130306:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728356148271 time 728375924773 state 0x0 ref 1 00000100:00100000:0.0:1789760785.130318:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554812160:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760785.130364:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65454080, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f378 00010000:00010000:0.0:1789760785.130407:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5f2a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.130436:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760785.130441:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.130447:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5f2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65454080->65458175] (req 65454080->65458175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.130468:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5f2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65454080->65458175] (req 65454080->65458175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.136168:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983289 is committed 00002000:00010000:2.0:1789760785.136278:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65454080 -> 65458176 00002000:00010000:2.0:1789760785.136283:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127864 -> 127872 00010000:00010000:2.0:1789760785.136296:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5f2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65454080->65458175] (req 65454080->65458175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.136309:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5f2a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65454080->65458175] (req 65454080->65458175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.136318:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5f2a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65454080->65458175] (req 65454080->65458175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.136331:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760785.136335:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5f2a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65454080->65458175] (req 65454080->65458175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.136344:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5f2a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65454080->65458175] (req 65454080->65458175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.136362:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5f2a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65454080->65458175] (req 65454080->65458175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.136370:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c5f2a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65454080->65458175] (req 65454080->65458175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.136634:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554812160:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6313us (6608us total) trans 4294983289 rc 0/0 00000100:00100000:2.0:1789760785.136644:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15991 00000100:00100000:2.0:1789760785.136653:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728356148271 time 728375924773 state 0x2 ref 0 00000100:00100000:2.0:1789760785.139737:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760785.139824:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554812288 00000100:00100000:1.0:1789760785.139847:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554812288 00000100:00100000:1.0:1789760785.139901:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728356148271 time 728375924773 state 0x0 ref 1 00000100:00100000:1.0:1789760785.139916:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728376185189 time 728375924773 state 0x1 ref 1 00000100:00100000:1.0:1789760785.139948:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728376185189 time 728385566370 state 0x0 ref 1 00000100:00100000:1.0:1789760785.139959:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554812288:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760785.140005:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65458176, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f380 00010000:00010000:1.0:1789760785.140013:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f31 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.140029:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760785.140034:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.140040:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65458176->65462271] (req 65458176->65462271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.140063:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65458176->65462271] (req 65458176->65462271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.153938:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983290 is committed 00002000:00010000:1.0:1789760785.154183:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65458176 -> 65462272 00002000:00010000:1.0:1789760785.154189:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127872 -> 127880 00010000:00010000:1.0:1789760785.154204:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65458176->65462271] (req 65458176->65462271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.154217:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f31 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65458176->65462271] (req 65458176->65462271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.154227:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f31 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65458176->65462271] (req 65458176->65462271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.154235:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760785.154241:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f31 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65458176->65462271] (req 65458176->65462271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.154250:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f31 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65458176->65462271] (req 65458176->65462271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.154269:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f31 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65458176->65462271] (req 65458176->65462271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.154277:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f31 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65458176->65462271] (req 65458176->65462271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.154373:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554812288:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14410us (14640us total) trans 4294983290 rc 0/0 00000100:00100000:1.0:1789760785.154429:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15992 00000100:00100000:1.0:1789760785.154439:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728376185189 time 728385566370 state 0x2 ref 0 00000100:00100000:1.0:1789760785.161876:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760785.162161:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554812416 00000100:00100000:2.0:1789760785.162172:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554812416 00000100:00100000:2.0:1789760785.162225:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728376185189 time 728385566370 state 0x0 ref 1 00000100:00100000:2.0:1789760785.162240:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728385826786 time 728385566370 state 0x1 ref 1 00000100:00100000:2.0:1789760785.162274:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728385826786 time 728407892849 state 0x0 ref 1 00000100:00100000:2.0:1789760785.162289:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554812416:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760785.162339:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65462272, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f388 00010000:00010000:2.0:1789760785.162348:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5f38 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.162363:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760785.162367:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.162374:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5f38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65462272->65466367] (req 65462272->65466367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.162437:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5f38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65462272->65466367] (req 65462272->65466367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.168186:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983291 is committed 00002000:00010000:2.0:1789760785.168296:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65462272 -> 65466368 00002000:00010000:2.0:1789760785.168301:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127880 -> 127888 00010000:00010000:2.0:1789760785.168314:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5f38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65462272->65466367] (req 65462272->65466367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.168328:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5f38 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65462272->65466367] (req 65462272->65466367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.168338:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5f38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65462272->65466367] (req 65462272->65466367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.168345:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760785.168352:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5f38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65462272->65466367] (req 65462272->65466367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.168362:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5f38 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65462272->65466367] (req 65462272->65466367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.169175:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5f38 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65462272->65466367] (req 65462272->65466367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.169189:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c5f38 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65462272->65466367] (req 65462272->65466367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.169292:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554812416:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7000us (7420us total) trans 4294983291 rc 0/0 00000100:00100000:2.0:1789760785.169304:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15993 00000100:00100000:2.0:1789760785.169315:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728385826786 time 728407892849 state 0x2 ref 0 00000100:00100000:2.0:1789760785.185075:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760785.192726:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554812544 00000100:00100000:1.0:1789760785.192741:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554812544 00000100:00100000:1.0:1789760785.192804:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728385826786 time 728407892849 state 0x0 ref 1 00000100:00100000:1.0:1789760785.192819:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728408153265 time 728407892849 state 0x1 ref 1 00000100:00100000:1.0:1789760785.192923:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728408153265 time 728438541014 state 0x0 ref 1 00000100:00100000:1.0:1789760785.192936:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554812544:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760785.192983:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65466368, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f390 00010000:00010000:1.0:1789760785.192993:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f3f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.193015:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760785.193020:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.193027:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65466368->65470463] (req 65466368->65470463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.193050:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65466368->65470463] (req 65466368->65470463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.206805:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983292 is committed 00002000:00010000:1.0:1789760785.207084:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65466368 -> 65470464 00002000:00010000:1.0:1789760785.207089:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127888 -> 127896 00010000:00010000:1.0:1789760785.207102:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65466368->65470463] (req 65466368->65470463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.207115:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f3f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65466368->65470463] (req 65466368->65470463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.207124:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f3f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65466368->65470463] (req 65466368->65470463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.207131:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760785.207136:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f3f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65466368->65470463] (req 65466368->65470463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.207146:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f3f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65466368->65470463] (req 65466368->65470463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.207166:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f3f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65466368->65470463] (req 65466368->65470463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.207174:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f3f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65466368->65470463] (req 65466368->65470463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.207278:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554812544:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14339us (22207us total) trans 4294983292 rc 0/0 00000100:00100000:1.0:1789760785.207288:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15994 00000100:00100000:1.0:1789760785.207298:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728408153265 time 728438541014 state 0x2 ref 0 00000100:00100000:3.0:1789760785.217886:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760785.218634:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554812672 00000100:00100000:0.0:1789760785.218645:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554812672 00000100:00100000:0.0:1789760785.218713:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728408153265 time 728438541014 state 0x0 ref 1 00000100:00100000:0.0:1789760785.218729:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728438801430 time 728438541014 state 0x1 ref 1 00000100:00100000:0.0:1789760785.218764:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728438801430 time 728464381967 state 0x0 ref 1 00000100:00100000:0.0:1789760785.218776:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554812672:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760785.218823:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65470464, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f398 00010000:00010000:0.0:1789760785.218833:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f46 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.218850:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760785.218854:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.218860:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65470464->65474559] (req 65470464->65474559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.218874:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65470464->65474559] (req 65470464->65474559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.225814:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983293 is committed 00002000:00010000:0.0:1789760785.225940:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65470464 -> 65474560 00002000:00010000:0.0:1789760785.225946:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127896 -> 127904 00010000:00010000:0.0:1789760785.225959:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65470464->65474559] (req 65470464->65474559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.225971:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f46 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65470464->65474559] (req 65470464->65474559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.225981:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f46 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65470464->65474559] (req 65470464->65474559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.225994:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760785.225999:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f46 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65470464->65474559] (req 65470464->65474559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.226007:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f46 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65470464->65474559] (req 65470464->65474559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.226027:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f46 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65470464->65474559] (req 65470464->65474559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.226044:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f46 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65470464->65474559] (req 65470464->65474559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.226135:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f60700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554812672:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7356us (8255us total) trans 4294983293 rc 0/0 00000100:00100000:0.0:1789760785.226146:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15995 00000100:00100000:0.0:1789760785.226156:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728438801430 time 728464381967 state 0x2 ref 0 00000100:00100000:2.0:1789760785.234886:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760785.235246:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554812800 00000100:00100000:1.0:1789760785.235257:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554812800 00000100:00100000:1.0:1789760785.235311:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728438801430 time 728464381967 state 0x0 ref 1 00000100:00100000:1.0:1789760785.235327:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728464642383 time 728464381967 state 0x1 ref 1 00000100:00100000:1.0:1789760785.235447:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728464642383 time 728481063169 state 0x0 ref 1 00000100:00100000:1.0:1789760785.235461:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554812800:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760785.235510:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65474560, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f3a0 00010000:00010000:1.0:1789760785.235520:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f4d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.235537:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760785.235541:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.235547:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65474560->65478655] (req 65474560->65478655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.235560:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65474560->65478655] (req 65474560->65478655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.243200:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983294 is committed 00002000:00010000:1.0:1789760785.243528:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65474560 -> 65478656 00002000:00010000:1.0:1789760785.243533:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127904 -> 127912 00010000:00010000:1.0:1789760785.243546:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65474560->65478655] (req 65474560->65478655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.243564:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f4d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65474560->65478655] (req 65474560->65478655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.243574:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f4d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65474560->65478655] (req 65474560->65478655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.243581:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760785.243585:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f4d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65474560->65478655] (req 65474560->65478655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.243593:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f4d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65474560->65478655] (req 65474560->65478655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.243612:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f4d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65474560->65478655] (req 65474560->65478655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.243619:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f4d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65474560->65478655] (req 65474560->65478655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.243757:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554812800:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8292us (8875us total) trans 4294983294 rc 0/0 00000100:00100000:1.0:1789760785.243776:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15996 00000100:00100000:1.0:1789760785.243786:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728464642383 time 728481063169 state 0x2 ref 0 00000100:00100000:3.0:1789760785.250637:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760785.250753:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554812928 00000100:00100000:2.0:1789760785.250762:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554812928 00000100:00100000:2.0:1789760785.250818:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728464642383 time 728481063169 state 0x0 ref 1 00000100:00100000:2.0:1789760785.250834:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728481323585 time 728481063169 state 0x1 ref 1 00000100:00100000:2.0:1789760785.250901:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728481323585 time 728496518541 state 0x0 ref 1 00000100:00100000:2.0:1789760785.250914:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554812928:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760785.250964:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65478656, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f3a8 00010000:00010000:2.0:1789760785.250974:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5f54 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.250996:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760785.251001:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.251008:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5f54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65478656->65482751] (req 65478656->65482751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.251023:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5f54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65478656->65482751] (req 65478656->65482751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.256707:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983295 is committed 00002000:00010000:2.0:1789760785.257348:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65478656 -> 65482752 00002000:00010000:2.0:1789760785.257355:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127912 -> 127920 00010000:00010000:2.0:1789760785.257368:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5f54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65478656->65482751] (req 65478656->65482751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.257410:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5f54 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65478656->65482751] (req 65478656->65482751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.257526:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5f54 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65478656->65482751] (req 65478656->65482751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.257535:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760785.257541:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5f54 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65478656->65482751] (req 65478656->65482751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.257560:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5f54 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65478656->65482751] (req 65478656->65482751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.257583:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5f54 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65478656->65482751] (req 65478656->65482751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.257591:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c5f54 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65478656->65482751] (req 65478656->65482751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.257754:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f62a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554812928:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6837us (7370us total) trans 4294983295 rc 0/0 00000100:00100000:2.0:1789760785.257766:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15997 00000100:00100000:2.0:1789760785.257778:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728481323585 time 728496518541 state 0x2 ref 0 00000100:00100000:2.0:1789760785.265400:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760785.265748:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554813056 00000100:00100000:1.0:1789760785.265759:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554813056 00000100:00100000:1.0:1789760785.265813:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728481323585 time 728496518541 state 0x0 ref 1 00000100:00100000:1.0:1789760785.265829:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728496778957 time 728496518541 state 0x1 ref 1 00000100:00100000:1.0:1789760785.265899:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728496778957 time 728511517233 state 0x0 ref 1 00000100:00100000:1.0:1789760785.265912:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554813056:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760785.265960:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65482752, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f3b0 00010000:00010000:1.0:1789760785.265969:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f5b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.265984:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760785.265989:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.265995:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65482752->65486847] (req 65482752->65486847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.266008:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65482752->65486847] (req 65482752->65486847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.271603:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983296 is committed 00002000:00010000:1.0:1789760785.271748:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65482752 -> 65486848 00002000:00010000:1.0:1789760785.271754:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127920 -> 127928 00010000:00010000:1.0:1789760785.271770:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65482752->65486847] (req 65482752->65486847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.271785:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f5b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65482752->65486847] (req 65482752->65486847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.271799:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f5b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65482752->65486847] (req 65482752->65486847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.271810:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760785.271816:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f5b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65482752->65486847] (req 65482752->65486847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.271828:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f5b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65482752->65486847] (req 65482752->65486847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.271850:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f5b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65482752->65486847] (req 65482752->65486847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.271862:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c5f5b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65482752->65486847] (req 65482752->65486847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.271969:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554813056:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6054us (6597us total) trans 4294983296 rc 0/0 00000100:00100000:1.0:1789760785.271984:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15998 00000100:00100000:1.0:1789760785.271993:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728496778957 time 728511517233 state 0x2 ref 0 00000100:00100000:1.0:1789760785.275795:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760785.275917:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554813184 00000100:00100000:0.0:1789760785.275927:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554813184 00000100:00100000:0.0:1789760785.275979:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728496778957 time 728511517233 state 0x0 ref 1 00000100:00100000:0.0:1789760785.275993:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728511777649 time 728511517233 state 0x1 ref 1 00000100:00100000:0.0:1789760785.276021:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728511777649 time 728521639506 state 0x0 ref 1 00000100:00100000:0.0:1789760785.276036:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554813184:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760785.276080:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65486848, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f3b8 00010000:00010000:0.0:1789760785.276100:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f62 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.276115:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760785.276119:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.276125:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65486848->65490943] (req 65486848->65490943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.276138:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65486848->65490943] (req 65486848->65490943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.282007:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983297 is committed 00002000:00010000:0.0:1789760785.286495:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65486848 -> 65490944 00002000:00010000:0.0:1789760785.286505:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127928 -> 127936 00010000:00010000:0.0:1789760785.286519:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65486848->65490943] (req 65486848->65490943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.286532:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f62 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65486848->65490943] (req 65486848->65490943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.286542:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65486848->65490943] (req 65486848->65490943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.286549:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760785.286555:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65486848->65490943] (req 65486848->65490943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.286567:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f62 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65486848->65490943] (req 65486848->65490943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.286589:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f62 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65486848->65490943] (req 65486848->65490943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.286596:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f62 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65486848->65490943] (req 65486848->65490943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.286707:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554813184:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10669us (10918us total) trans 4294983297 rc 0/0 00000100:00100000:0.0:1789760785.286727:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 15999 00000100:00100000:0.0:1789760785.286736:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728511777649 time 728521639506 state 0x2 ref 0 00000100:00100000:3.0:1789760785.306683:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760785.308514:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554813312 00000100:00100000:2.0:1789760785.308530:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554813312 00000100:00100000:2.0:1789760785.308590:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728511777649 time 728521639506 state 0x0 ref 1 00000100:00100000:2.0:1789760785.308607:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728521899922 time 728521639506 state 0x1 ref 1 00000100:00100000:2.0:1789760785.309738:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728521899922 time 728555349380 state 0x0 ref 1 00000100:00100000:2.0:1789760785.309762:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f61f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554813312:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760785.309824:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65490944, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f3c0 00010000:00010000:2.0:1789760785.309834:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5f69 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.309849:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760785.309854:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.309860:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5f69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65490944->65495039] (req 65490944->65495039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.309874:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5f69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65490944->65495039] (req 65490944->65495039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.317243:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983298 is committed 00002000:00010000:2.0:1789760785.317368:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65490944 -> 65495040 00002000:00010000:2.0:1789760785.317374:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127936 -> 127944 00010000:00010000:2.0:1789760785.317405:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5f69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65490944->65495039] (req 65490944->65495039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.317419:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5f69 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65490944->65495039] (req 65490944->65495039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.317428:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5f69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65490944->65495039] (req 65490944->65495039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.317435:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760785.317441:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5f69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65490944->65495039] (req 65490944->65495039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.317449:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5f69 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65490944->65495039] (req 65490944->65495039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.317471:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5f69 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65490944->65495039] (req 65490944->65495039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.317479:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c5f69 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65490944->65495039] (req 65490944->65495039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.318304:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f61f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554813312:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8544us (11640us total) trans 4294983298 rc 0/0 00000100:00100000:2.0:1789760785.318317:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16000 00000100:00100000:2.0:1789760785.318327:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728521899922 time 728555349380 state 0x2 ref 0 00000100:00100000:2.0:1789760785.323622:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760785.324244:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554813440 00000100:00100000:3.0:1789760785.324255:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554813440 00000100:00100000:3.0:1789760785.324322:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728521899922 time 728555349380 state 0x0 ref 1 00000100:00100000:3.0:1789760785.324339:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728555609796 time 728555349380 state 0x1 ref 1 00000100:00100000:3.0:1789760785.324404:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728555609796 time 728569995251 state 0x0 ref 1 00000100:00100000:3.0:1789760785.324653:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554813440:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760785.324716:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65495040, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f3c8 00010000:00010000:3.0:1789760785.324726:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5f70 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.324742:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760785.324746:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.324753:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5f70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65495040->65499135] (req 65495040->65499135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.324766:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5f70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65495040->65499135] (req 65495040->65499135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.333112:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983299 is committed 00002000:00010000:3.0:1789760785.333283:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65495040 -> 65499136 00002000:00010000:3.0:1789760785.333289:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127944 -> 127952 00010000:00010000:3.0:1789760785.334360:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5f70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65495040->65499135] (req 65495040->65499135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.334408:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5f70 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65495040->65499135] (req 65495040->65499135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.334494:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5f70 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65495040->65499135] (req 65495040->65499135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.334502:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760785.334508:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5f70 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65495040->65499135] (req 65495040->65499135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.334518:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5f70 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65495040->65499135] (req 65495040->65499135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.334541:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5f70 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65495040->65499135] (req 65495040->65499135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.334549:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5f70 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65495040->65499135] (req 65495040->65499135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.334736:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554813440:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10084us (11191us total) trans 4294983299 rc 0/0 00000100:00100000:3.0:1789760785.334746:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16001 00000100:00100000:3.0:1789760785.334755:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728555609796 time 728569995251 state 0x2 ref 0 00000100:00100000:1.0:1789760785.338096:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760785.338404:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554813568 00000100:00100000:2.0:1789760785.338416:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554813568 00000100:00100000:2.0:1789760785.338475:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728555609796 time 728569995251 state 0x0 ref 1 00000100:00100000:2.0:1789760785.338710:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728570255667 time 728569995251 state 0x1 ref 1 00000100:00100000:2.0:1789760785.338775:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728570255667 time 728584393662 state 0x0 ref 1 00000100:00100000:2.0:1789760785.338788:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554813568:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760785.338832:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65499136, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f3d0 00010000:00010000:2.0:1789760785.338840:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5f77 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.338861:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760785.338865:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.338871:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5f77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65499136->65503231] (req 65499136->65503231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.338885:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5f77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65499136->65503231] (req 65499136->65503231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.344981:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983300 is committed 00002000:00010000:2.0:1789760785.345096:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65499136 -> 65503232 00002000:00010000:2.0:1789760785.345101:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127952 -> 127960 00010000:00010000:2.0:1789760785.345114:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5f77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65499136->65503231] (req 65499136->65503231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.345126:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5f77 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65499136->65503231] (req 65499136->65503231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.345136:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5f77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65499136->65503231] (req 65499136->65503231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.345143:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760785.345148:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5f77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65499136->65503231] (req 65499136->65503231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.345156:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5f77 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65499136->65503231] (req 65499136->65503231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.345180:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5f77 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65499136->65503231] (req 65499136->65503231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.345188:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5f77 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65499136->65503231] (req 65499136->65503231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.345284:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554813568:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6493us (7194us total) trans 4294983300 rc 0/0 00000100:00100000:2.0:1789760785.345649:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16002 00000100:00100000:2.0:1789760785.345679:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728570255667 time 728584393662 state 0x2 ref 0 00000100:00100000:3.0:1789760785.350738:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760785.350877:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554813696 00000100:00100000:0.0:1789760785.350887:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554813696 00000100:00100000:0.0:1789760785.350944:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728570255667 time 728584393662 state 0x0 ref 1 00000100:00100000:0.0:1789760785.350960:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728584654078 time 728584393662 state 0x1 ref 1 00000100:00100000:0.0:1789760785.350990:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728584654078 time 728596608456 state 0x0 ref 1 00000100:00100000:0.0:1789760785.351003:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f61880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554813696:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760785.351058:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65503232, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f3d8 00010000:00010000:0.0:1789760785.351068:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f7e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.351084:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760785.351088:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.351094:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65503232->65507327] (req 65503232->65507327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.351109:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65503232->65507327] (req 65503232->65507327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.360328:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983301 is committed 00002000:00010000:2.0:1789760785.360492:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65503232 -> 65507328 00002000:00010000:2.0:1789760785.360497:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127960 -> 127968 00010000:00010000:2.0:1789760785.360511:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65503232->65507327] (req 65503232->65507327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.360525:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f7e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65503232->65507327] (req 65503232->65507327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.360535:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65503232->65507327] (req 65503232->65507327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.360543:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760785.360675:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65503232->65507327] (req 65503232->65507327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.360692:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f7e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65503232->65507327] (req 65503232->65507327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.360712:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f7e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65503232->65507327] (req 65503232->65507327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.360720:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c5f7e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65503232->65507327] (req 65503232->65507327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.361551:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f61880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554813696:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9808us (10080us total) trans 4294983301 rc 0/0 00000100:00100000:2.0:1789760785.361629:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16003 00000100:00100000:2.0:1789760785.361639:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728584654078 time 728596608456 state 0x2 ref 0 00000100:00100000:3.0:1789760785.363765:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760785.363917:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554813824 00000100:00100000:1.0:1789760785.363928:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554813824 00000100:00100000:1.0:1789760785.363978:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728584654078 time 728596608456 state 0x0 ref 1 00000100:00100000:1.0:1789760785.363994:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728596868872 time 728596608456 state 0x1 ref 1 00000100:00100000:1.0:1789760785.364022:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728596868872 time 728609640679 state 0x0 ref 1 00000100:00100000:1.0:1789760785.364035:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f63480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554813824:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760785.364093:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65507328, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f3e0 00010000:00010000:1.0:1789760785.364103:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5f85 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.364118:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760785.364123:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.364130:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5f85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65507328->65511423] (req 65507328->65511423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.364145:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5f85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65507328->65511423] (req 65507328->65511423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.369920:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983302 is committed 00002000:00010000:1.0:1789760785.370021:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65507328 -> 65511424 00002000:00010000:1.0:1789760785.370026:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127968 -> 127976 00010000:00010000:1.0:1789760785.370046:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5f85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65507328->65511423] (req 65507328->65511423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.370059:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5f85 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65507328->65511423] (req 65507328->65511423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.370074:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5f85 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65507328->65511423] (req 65507328->65511423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.370081:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760785.370086:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5f85 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65507328->65511423] (req 65507328->65511423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.370095:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5f85 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65507328->65511423] (req 65507328->65511423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.370112:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5f85 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65507328->65511423] (req 65507328->65511423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.370119:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c5f85 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65507328->65511423] (req 65507328->65511423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.370200:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f63480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554813824:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6163us (6439us total) trans 4294983302 rc 0/0 00000100:00100000:1.0:1789760785.370210:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16004 00000100:00100000:1.0:1789760785.370219:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728596868872 time 728609640679 state 0x2 ref 0 00000100:00100000:1.0:1789760785.374809:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760785.375325:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554813952 00000100:00100000:2.0:1789760785.375335:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554813952 00000100:00100000:2.0:1789760785.375413:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728596868872 time 728609640679 state 0x0 ref 1 00000100:00100000:2.0:1789760785.375429:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728609901095 time 728609640679 state 0x1 ref 1 00000100:00100000:2.0:1789760785.375456:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728609901095 time 728621074610 state 0x0 ref 1 00000100:00100000:2.0:1789760785.375469:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554813952:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760785.375515:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65511424, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f3e8 00010000:00010000:2.0:1789760785.375524:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5f8c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.375542:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760785.375546:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.375553:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5f8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65511424->65515519] (req 65511424->65515519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.375567:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5f8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65511424->65515519] (req 65511424->65515519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.382105:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983303 is committed 00002000:00010000:2.0:1789760785.382292:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65511424 -> 65515520 00002000:00010000:2.0:1789760785.382297:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127976 -> 127984 00010000:00010000:2.0:1789760785.382313:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5f8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65511424->65515519] (req 65511424->65515519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.382328:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5f8c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65511424->65515519] (req 65511424->65515519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.382337:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5f8c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65511424->65515519] (req 65511424->65515519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.382346:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760785.382351:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5f8c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65511424->65515519] (req 65511424->65515519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.382361:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5f8c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65511424->65515519] (req 65511424->65515519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.382455:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5f8c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65511424->65515519] (req 65511424->65515519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.382464:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5f8c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65511424->65515519] (req 65511424->65515519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.382554:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554813952:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7083us (7749us total) trans 4294983303 rc 0/0 00000100:00100000:2.0:1789760785.382566:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16005 00000100:00100000:2.0:1789760785.382575:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728609901095 time 728621074610 state 0x2 ref 0 00000100:00100000:2.0:1789760785.390128:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760785.392081:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554814080 00000100:00100000:3.0:1789760785.392096:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554814080 00000100:00100000:0.0:1789760785.392257:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728609901095 time 728621074610 state 0x0 ref 1 00000100:00100000:0.0:1789760785.392274:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728621335026 time 728621074610 state 0x1 ref 1 00000100:00100000:0.0:1789760785.392309:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728621335026 time 728637927363 state 0x0 ref 1 00000100:00100000:0.0:1789760785.392323:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554814080:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760785.392369:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65515520, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f3f0 00010000:00010000:0.0:1789760785.392397:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5f93 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.392414:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760785.392419:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.392425:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5f93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65515520->65519615] (req 65515520->65519615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.392442:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5f93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65515520->65519615] (req 65515520->65519615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.400594:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983304 is committed 00002000:00010000:0.0:1789760785.401068:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65515520 -> 65519616 00002000:00010000:0.0:1789760785.401074:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127984 -> 127992 00010000:00010000:0.0:1789760785.401087:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5f93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65515520->65519615] (req 65515520->65519615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.401099:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5f93 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65515520->65519615] (req 65515520->65519615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.401109:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5f93 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65515520->65519615] (req 65515520->65519615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.401116:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760785.401121:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5f93 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65515520->65519615] (req 65515520->65519615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.401130:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5f93 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65515520->65519615] (req 65515520->65519615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.401149:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5f93 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65515520->65519615] (req 65515520->65519615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.401158:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5f93 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65515520->65519615] (req 65515520->65519615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.401254:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554814080:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8930us (11131us total) trans 4294983304 rc 0/0 00000100:00100000:0.0:1789760785.401269:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16006 00000100:00100000:0.0:1789760785.401279:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728621335026 time 728637927363 state 0x2 ref 0 00000100:00100000:3.0:1789760785.408923:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760785.409550:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554814208 00000100:00100000:0.0:1789760785.409561:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554814208 00000100:00100000:0.0:1789760785.409619:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728621335026 time 728637927363 state 0x0 ref 1 00000100:00100000:0.0:1789760785.409633:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728638187779 time 728637927363 state 0x1 ref 1 00000100:00100000:0.0:1789760785.410019:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728638187779 time 728655634904 state 0x0 ref 1 00000100:00100000:0.0:1789760785.410034:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f63b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554814208:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760785.410083:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65519616, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f3f8 00010000:00010000:0.0:1789760785.410093:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5f9a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.410108:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760785.410113:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.410119:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5f9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65519616->65523711] (req 65519616->65523711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.410133:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5f9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65519616->65523711] (req 65519616->65523711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.416949:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983305 is committed 00002000:00010000:0.0:1789760785.417190:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65519616 -> 65523712 00002000:00010000:0.0:1789760785.417196:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 127992 -> 128000 00010000:00010000:0.0:1789760785.417210:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5f9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65519616->65523711] (req 65519616->65523711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.417223:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5f9a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65519616->65523711] (req 65519616->65523711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.417231:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5f9a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65519616->65523711] (req 65519616->65523711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.417238:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760785.417242:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5f9a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65519616->65523711] (req 65519616->65523711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.417251:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5f9a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65519616->65523711] (req 65519616->65523711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.417270:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5f9a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65519616->65523711] (req 65519616->65523711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.417277:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5f9a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65519616->65523711] (req 65519616->65523711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.417535:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f63b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554814208:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7343us (8460us total) trans 4294983305 rc 0/0 00000100:00100000:0.0:1789760785.417550:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16007 00000100:00100000:0.0:1789760785.417561:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728638187779 time 728655634904 state 0x2 ref 0 00000100:00100000:1.0:1789760785.422783:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760785.422981:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554814336 00000100:00100000:2.0:1789760785.422992:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554814336 00000100:00100000:2.0:1789760785.423105:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728638187779 time 728655634904 state 0x0 ref 1 00000100:00100000:2.0:1789760785.423121:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728655895320 time 728655634904 state 0x1 ref 1 00000100:00100000:2.0:1789760785.423193:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728655895320 time 728668810382 state 0x0 ref 1 00000100:00100000:2.0:1789760785.423206:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554814336:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760785.423251:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65523712, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f400 00010000:00010000:2.0:1789760785.423261:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5fa1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.423283:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760785.423288:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.423296:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5fa1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65523712->65527807] (req 65523712->65527807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.423310:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5fa1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65523712->65527807] (req 65523712->65527807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.431444:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983306 is committed 00002000:00010000:2.0:1789760785.431578:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65523712 -> 65527808 00002000:00010000:2.0:1789760785.431583:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128000 -> 128008 00010000:00010000:2.0:1789760785.431597:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5fa1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65523712->65527807] (req 65523712->65527807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.431610:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5fa1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65523712->65527807] (req 65523712->65527807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.431620:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5fa1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65523712->65527807] (req 65523712->65527807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.431628:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760785.431633:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5fa1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65523712->65527807] (req 65523712->65527807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.431643:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5fa1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65523712->65527807] (req 65523712->65527807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.431681:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5fa1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65523712->65527807] (req 65523712->65527807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.431689:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5fa1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65523712->65527807] (req 65523712->65527807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.431785:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554814336:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8576us (9005us total) trans 4294983306 rc 0/0 00000100:00100000:2.0:1789760785.431795:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16008 00000100:00100000:2.0:1789760785.431805:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728655895320 time 728668810382 state 0x2 ref 0 00000100:00100000:2.0:1789760785.435447:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760785.435533:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554814464 00000100:00100000:3.0:1789760785.435543:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554814464 00000100:00100000:3.0:1789760785.435599:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728655895320 time 728668810382 state 0x0 ref 1 00000100:00100000:3.0:1789760785.435615:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728669070798 time 728668810382 state 0x1 ref 1 00000100:00100000:3.0:1789760785.435641:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728669070798 time 728681259841 state 0x0 ref 1 00000100:00100000:3.0:1789760785.435653:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03cc8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554814464:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760785.435709:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65527808, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f408 00010000:00010000:3.0:1789760785.435718:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5fa8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.435733:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760785.435738:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.435744:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5fa8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65527808->65531903] (req 65527808->65531903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.435758:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5fa8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65527808->65531903] (req 65527808->65531903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.442766:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983307 is committed 00002000:00010000:2.0:1789760785.442873:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65527808 -> 65531904 00002000:00010000:2.0:1789760785.442878:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128008 -> 128016 00010000:00010000:2.0:1789760785.442900:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5fa8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65527808->65531903] (req 65527808->65531903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.442914:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5fa8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65527808->65531903] (req 65527808->65531903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.442924:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5fa8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65527808->65531903] (req 65527808->65531903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.442932:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760785.442937:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5fa8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65527808->65531903] (req 65527808->65531903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.442945:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5fa8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65527808->65531903] (req 65527808->65531903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.442964:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5fa8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65527808->65531903] (req 65527808->65531903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.442972:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c5fa8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65527808->65531903] (req 65527808->65531903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.443066:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03cc8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554814464:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7409us (7624us total) trans 4294983307 rc 0/0 00000100:00100000:2.0:1789760785.443076:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16009 00000100:00100000:2.0:1789760785.443086:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728669070798 time 728681259841 state 0x2 ref 0 00000100:00100000:3.0:1789760785.451593:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760785.451704:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554814592 00000100:00100000:0.0:1789760785.451713:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554814592 00000100:00100000:0.0:1789760785.451768:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728669070798 time 728681259841 state 0x0 ref 1 00000100:00100000:0.0:1789760785.451784:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728681520257 time 728681259841 state 0x1 ref 1 00000100:00100000:0.0:1789760785.451812:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728681520257 time 728697430517 state 0x0 ref 1 00000100:00100000:0.0:1789760785.451826:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03f60a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554814592:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760785.451874:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65531904, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f410 00010000:00010000:0.0:1789760785.451882:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5faf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.451898:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760785.451903:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.451909:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5faf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65531904->65535999] (req 65531904->65535999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.451933:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5faf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65531904->65535999] (req 65531904->65535999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.461612:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983308 is committed 00002000:00010000:0.0:1789760785.461756:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65531904 -> 65536000 00002000:00010000:0.0:1789760785.461763:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128016 -> 128024 00010000:00010000:0.0:1789760785.461778:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5faf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65531904->65535999] (req 65531904->65535999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.461792:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5faf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65531904->65535999] (req 65531904->65535999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.461801:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5faf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65531904->65535999] (req 65531904->65535999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.461810:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760785.461815:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5faf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65531904->65535999] (req 65531904->65535999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.461825:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5faf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65531904->65535999] (req 65531904->65535999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.461844:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5faf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65531904->65535999] (req 65531904->65535999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.461853:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5faf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65531904->65535999] (req 65531904->65535999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.462089:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03f60a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554814592:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10261us (10499us total) trans 4294983308 rc 0/0 00000100:00100000:0.0:1789760785.462099:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16010 00000100:00100000:0.0:1789760785.462108:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728681520257 time 728697430517 state 0x2 ref 0 00000100:00100000:1.0:1789760785.472283:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760785.472613:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554814720 00000100:00100000:0.0:1789760785.472625:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554814720 00000100:00100000:0.0:1789760785.472692:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728681520257 time 728697430517 state 0x0 ref 1 00000100:00100000:0.0:1789760785.472708:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728697690933 time 728697430517 state 0x1 ref 1 00000100:00100000:0.0:1789760785.472742:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728697690933 time 728718360123 state 0x0 ref 1 00000100:00100000:0.0:1789760785.472755:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554814720:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760785.472801:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65536000, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f418 00010000:00010000:0.0:1789760785.472810:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5fb6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.472826:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760785.472830:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.472837:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5fb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65536000->65540095] (req 65536000->65540095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.472851:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5fb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65536000->65540095] (req 65536000->65540095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.483434:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983309 is committed 00002000:00010000:0.0:1789760785.483518:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65536000 -> 65540096 00002000:00010000:0.0:1789760785.483524:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128024 -> 128032 00010000:00010000:0.0:1789760785.483538:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5fb6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65536000->65540095] (req 65536000->65540095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.483556:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5fb6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65536000->65540095] (req 65536000->65540095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.483565:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5fb6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65536000->65540095] (req 65536000->65540095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.483573:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760785.483578:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5fb6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65536000->65540095] (req 65536000->65540095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.483586:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5fb6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65536000->65540095] (req 65536000->65540095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.483605:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5fb6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65536000->65540095] (req 65536000->65540095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.483612:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c5fb6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65536000->65540095] (req 65536000->65540095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.483715:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554814720:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10957us (11438us total) trans 4294983309 rc 0/0 00000100:00100000:0.0:1789760785.483727:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16011 00000100:00100000:0.0:1789760785.483738:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728697690933 time 728718360123 state 0x2 ref 0 00000100:00100000:2.0:1789760785.488636:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760785.488692:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554814848 00000100:00100000:1.0:1789760785.488702:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554814848 00000100:00100000:1.0:1789760785.488759:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728697690933 time 728718360123 state 0x0 ref 1 00000100:00100000:1.0:1789760785.488775:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728718620539 time 728718360123 state 0x1 ref 1 00000100:00100000:1.0:1789760785.488801:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728718620539 time 728734419705 state 0x0 ref 1 00000100:00100000:1.0:1789760785.488814:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ccad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554814848:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760785.488877:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65540096, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f420 00010000:00010000:1.0:1789760785.488887:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5fbd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.488905:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760785.488910:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.488916:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5fbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65540096->65544191] (req 65540096->65544191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.488930:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5fbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65540096->65544191] (req 65540096->65544191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.495885:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983310 is committed 00002000:00010000:1.0:1789760785.497243:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65540096 -> 65544192 00002000:00010000:1.0:1789760785.497251:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128032 -> 128040 00010000:00010000:1.0:1789760785.497264:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5fbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65540096->65544191] (req 65540096->65544191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.497278:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5fbd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65540096->65544191] (req 65540096->65544191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.497293:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5fbd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65540096->65544191] (req 65540096->65544191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.497301:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760785.497306:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5fbd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65540096->65544191] (req 65540096->65544191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.497315:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5fbd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65540096->65544191] (req 65540096->65544191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.497335:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5fbd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65540096->65544191] (req 65540096->65544191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.497344:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5fbd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65540096->65544191] (req 65540096->65544191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.497472:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ccad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554814848:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8654us (8841us total) trans 4294983310 rc 0/0 00000100:00100000:1.0:1789760785.497482:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16012 00000100:00100000:1.0:1789760785.497491:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728718620539 time 728734419705 state 0x2 ref 0 00000100:00100000:3.0:1789760785.502696:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760785.503235:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554814976 00000100:00100000:2.0:1789760785.503246:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554814976 00000100:00100000:2.0:1789760785.503301:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728718620539 time 728734419705 state 0x0 ref 1 00000100:00100000:2.0:1789760785.503317:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728734680121 time 728734419705 state 0x1 ref 1 00000100:00100000:2.0:1789760785.503354:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728734680121 time 728748972518 state 0x0 ref 1 00000100:00100000:2.0:1789760785.503416:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554814976:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760785.503462:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65544192, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f428 00010000:00010000:2.0:1789760785.503472:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5fc4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.503488:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760785.503493:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.503499:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5fc4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65544192->65548287] (req 65544192->65548287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.503513:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5fc4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65544192->65548287] (req 65544192->65548287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.511848:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983311 is committed 00002000:00010000:2.0:1789760785.511961:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65544192 -> 65548288 00002000:00010000:2.0:1789760785.511965:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128040 -> 128048 00010000:00010000:2.0:1789760785.511977:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5fc4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65544192->65548287] (req 65544192->65548287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.511987:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5fc4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65544192->65548287] (req 65544192->65548287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.511993:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5fc4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65544192->65548287] (req 65544192->65548287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.511998:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760785.512001:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5fc4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65544192->65548287] (req 65544192->65548287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.512011:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5fc4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65544192->65548287] (req 65544192->65548287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.512026:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5fc4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65544192->65548287] (req 65544192->65548287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.512041:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c5fc4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65544192->65548287] (req 65544192->65548287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.512203:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554814976:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8822us (9510us total) trans 4294983311 rc 0/0 00000100:00100000:2.0:1789760785.512214:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16013 00000100:00100000:2.0:1789760785.512223:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728734680121 time 728748972518 state 0x2 ref 0 00000100:00100000:2.0:1789760785.515812:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760785.515877:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554815104 00000100:00100000:0.0:1789760785.515889:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554815104 00000100:00100000:0.0:1789760785.515943:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728734680121 time 728748972518 state 0x0 ref 1 00000100:00100000:0.0:1789760785.515958:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728749232934 time 728748972518 state 0x1 ref 1 00000100:00100000:0.0:1789760785.515988:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728749232934 time 728761605933 state 0x0 ref 1 00000100:00100000:0.0:1789760785.516002:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03cca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554815104:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760785.516085:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65548288, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f430 00010000:00010000:1.0:1789760785.516093:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5fcb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.516109:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760785.516113:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.516117:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5fcb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65548288->65552383] (req 65548288->65552383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.516128:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5fcb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65548288->65552383] (req 65548288->65552383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.521993:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983312 is committed 00002000:00010000:1.0:1789760785.522194:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65548288 -> 65552384 00002000:00010000:1.0:1789760785.522200:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128048 -> 128056 00010000:00010000:1.0:1789760785.522213:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5fcb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65548288->65552383] (req 65548288->65552383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.522227:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5fcb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65548288->65552383] (req 65548288->65552383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.522236:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5fcb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65548288->65552383] (req 65548288->65552383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.522244:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760785.522250:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5fcb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65548288->65552383] (req 65548288->65552383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.522259:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5fcb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65548288->65552383] (req 65548288->65552383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.522285:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5fcb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65548288->65552383] (req 65548288->65552383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.522294:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c5fcb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65548288->65552383] (req 65548288->65552383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.522440:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03cca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554815104:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6435us (6630us total) trans 4294983312 rc 0/0 00000100:00100000:1.0:1789760785.522450:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16014 00000100:00100000:1.0:1789760785.522459:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728749232934 time 728761605933 state 0x2 ref 0 00000100:00100000:3.0:1789760785.527598:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760785.527941:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554815232 00000100:00100000:0.0:1789760785.527951:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554815232 00000100:00100000:0.0:1789760785.528004:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728749232934 time 728761605933 state 0x0 ref 1 00000100:00100000:0.0:1789760785.528021:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728761866349 time 728761605933 state 0x1 ref 1 00000100:00100000:2.0:1789760785.528423:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728761866349 time 728774038994 state 0x0 ref 1 00000100:00100000:2.0:1789760785.528438:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554815232:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760785.528483:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65552384, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f438 00010000:00010000:2.0:1789760785.528493:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5fd2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.528509:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760785.528513:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.528521:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5fd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65552384->65556479] (req 65552384->65556479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.528531:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5fd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65552384->65556479] (req 65552384->65556479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.535806:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983313 is committed 00002000:00010000:2.0:1789760785.535962:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65552384 -> 65556480 00002000:00010000:2.0:1789760785.535967:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128056 -> 128064 00010000:00010000:2.0:1789760785.535979:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5fd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65552384->65556479] (req 65552384->65556479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.535993:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5fd2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65552384->65556479] (req 65552384->65556479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.536001:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5fd2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65552384->65556479] (req 65552384->65556479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.536009:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760785.536013:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5fd2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65552384->65556479] (req 65552384->65556479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.536022:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5fd2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65552384->65556479] (req 65552384->65556479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.536052:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5fd2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65552384->65556479] (req 65552384->65556479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.536066:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c5fd2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65552384->65556479] (req 65552384->65556479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.536168:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554815232:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7727us (8578us total) trans 4294983313 rc 0/0 00000100:00100000:2.0:1789760785.536177:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16015 00000100:00100000:2.0:1789760785.536187:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728761866349 time 728774038994 state 0x2 ref 0 00000100:00100000:2.0:1789760785.540580:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760785.540703:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554815360 00000100:00100000:1.0:1789760785.540714:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554815360 00000100:00100000:1.0:1789760785.540767:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728761866349 time 728774038994 state 0x0 ref 1 00000100:00100000:1.0:1789760785.540785:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728774299410 time 728774038994 state 0x1 ref 1 00000100:00100000:1.0:1789760785.540848:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728774299410 time 728786466079 state 0x0 ref 1 00000100:00100000:1.0:1789760785.540861:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03cc9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554815360:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760785.540911:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65556480, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f440 00010000:00010000:1.0:1789760785.540920:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5fd9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.540936:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760785.540942:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.540948:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5fd9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65556480->65560575] (req 65556480->65560575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.540963:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5fd9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65556480->65560575] (req 65556480->65560575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.548410:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983314 is committed 00002000:00010000:1.0:1789760785.549438:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65556480 -> 65560576 00002000:00010000:1.0:1789760785.549444:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128064 -> 128072 00010000:00010000:1.0:1789760785.549459:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5fd9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65556480->65560575] (req 65556480->65560575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.549471:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5fd9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65556480->65560575] (req 65556480->65560575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.549481:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5fd9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65556480->65560575] (req 65556480->65560575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.549488:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760785.549493:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5fd9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65556480->65560575] (req 65556480->65560575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.549501:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5fd9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65556480->65560575] (req 65556480->65560575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.549521:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5fd9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65556480->65560575] (req 65556480->65560575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.549534:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5fd9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65556480->65560575] (req 65556480->65560575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.549635:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03cc9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554815360:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8770us (9060us total) trans 4294983314 rc 0/0 00000100:00100000:1.0:1789760785.549645:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16016 00000100:00100000:1.0:1789760785.549664:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728774299410 time 728786466079 state 0x2 ref 0 00000100:00100000:1.0:1789760785.555470:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760785.558217:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554815488 00000100:00100000:0.0:1789760785.558233:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554815488 00000100:00100000:1.0:1789760785.559019:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728774299410 time 728786466079 state 0x0 ref 1 00000100:00100000:1.0:1789760785.559037:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728786726495 time 728786466079 state 0x1 ref 1 00000100:00100000:1.0:1789760785.559077:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728786726495 time 728804695309 state 0x0 ref 1 00000100:00100000:1.0:1789760785.559092:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554815488:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760785.559152:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65560576, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f448 00010000:00010000:1.0:1789760785.559162:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5fe0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.559179:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760785.559183:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.559189:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5fe0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65560576->65564671] (req 65560576->65564671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.559218:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5fe0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65560576->65564671] (req 65560576->65564671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.569088:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983315 is committed 00002000:00010000:1.0:1789760785.575653:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65560576 -> 65564672 00002000:00010000:1.0:1789760785.575675:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128072 -> 128080 00010000:00010000:1.0:1789760785.575687:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5fe0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65560576->65564671] (req 65560576->65564671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.575700:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5fe0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65560576->65564671] (req 65560576->65564671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.575709:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5fe0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65560576->65564671] (req 65560576->65564671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.575716:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760785.575721:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5fe0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65560576->65564671] (req 65560576->65564671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.575730:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5fe0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65560576->65564671] (req 65560576->65564671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.575750:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5fe0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65560576->65564671] (req 65560576->65564671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.575758:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5fe0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65560576->65564671] (req 65560576->65564671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.575934:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554815488:12345-192.168.203.22@tcp:4:dd.0 Request processed in 16829us (20457us total) trans 4294983315 rc 0/0 00000100:00100000:1.0:1789760785.575945:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16017 00000100:00100000:1.0:1789760785.575954:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728786726495 time 728804695309 state 0x2 ref 0 00000100:00100000:3.0:1789760785.586747:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760785.586903:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554815616 00000100:00100000:2.0:1789760785.586913:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554815616 00000100:00100000:2.0:1789760785.586978:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728786726495 time 728804695309 state 0x0 ref 1 00000100:00100000:2.0:1789760785.586995:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728804955725 time 728804695309 state 0x1 ref 1 00000100:00100000:2.0:1789760785.587026:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728804955725 time 728832644546 state 0x0 ref 1 00000100:00100000:2.0:1789760785.587105:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554815616:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760785.587152:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65564672, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f450 00010000:00010000:2.0:1789760785.587160:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5fe7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.587175:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760785.587180:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.587186:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5fe7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65564672->65568767] (req 65564672->65568767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.587199:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5fe7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65564672->65568767] (req 65564672->65568767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.597224:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983316 is committed 00002000:00010000:1.0:1789760785.597351:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65564672 -> 65568768 00002000:00010000:1.0:1789760785.597356:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128080 -> 128088 00010000:00010000:1.0:1789760785.597368:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5fe7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65564672->65568767] (req 65564672->65568767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.597393:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5fe7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65564672->65568767] (req 65564672->65568767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.597403:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5fe7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65564672->65568767] (req 65564672->65568767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.597472:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760785.597478:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5fe7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65564672->65568767] (req 65564672->65568767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.597488:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5fe7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65564672->65568767] (req 65564672->65568767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.597507:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5fe7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65564672->65568767] (req 65564672->65568767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.597520:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c5fe7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65564672->65568767] (req 65564672->65568767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.597653:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554815616:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10547us (10911us total) trans 4294983316 rc 0/0 00000100:00100000:1.0:1789760785.597673:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16018 00000100:00100000:1.0:1789760785.597682:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728804955725 time 728832644546 state 0x2 ref 0 00000100:00100000:2.0:1789760785.610418:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760785.610556:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554815744 00000100:00100000:3.0:1789760785.610567:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554815744 00000100:00100000:3.0:1789760785.610628:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728804955725 time 728832644546 state 0x0 ref 1 00000100:00100000:3.0:1789760785.610645:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728832904962 time 728832644546 state 0x1 ref 1 00000100:00100000:3.0:1789760785.610688:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728832904962 time 728856306533 state 0x0 ref 1 00000100:00100000:3.0:1789760785.610703:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ccb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554815744:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760785.610749:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65568768, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f458 00010000:00010000:3.0:1789760785.610760:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5fee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.610776:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760785.610781:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.610787:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5fee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65568768->65572863] (req 65568768->65572863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.610801:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5fee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65568768->65572863] (req 65568768->65572863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.618922:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983317 is committed 00002000:00010000:3.0:1789760785.619047:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65568768 -> 65572864 00002000:00010000:3.0:1789760785.619053:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128088 -> 128096 00010000:00010000:3.0:1789760785.619066:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5fee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65568768->65572863] (req 65568768->65572863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.619083:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5fee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65568768->65572863] (req 65568768->65572863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.619092:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5fee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65568768->65572863] (req 65568768->65572863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.619100:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760785.619105:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5fee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65568768->65572863] (req 65568768->65572863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.619113:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5fee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65568768->65572863] (req 65568768->65572863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.619133:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5fee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65568768->65572863] (req 65568768->65572863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.619142:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c5fee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65568768->65572863] (req 65568768->65572863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.619240:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ccb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554815744:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8535us (8862us total) trans 4294983317 rc 0/0 00000100:00100000:3.0:1789760785.619250:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16019 00000100:00100000:3.0:1789760785.619260:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728832904962 time 728856306533 state 0x2 ref 0 00000100:00100000:3.0:1789760785.626556:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760785.626806:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554815872 00000100:00100000:1.0:1789760785.626817:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554815872 00000100:00100000:1.0:1789760785.626881:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728832904962 time 728856306533 state 0x0 ref 1 00000100:00100000:1.0:1789760785.626898:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728856566949 time 728856306533 state 0x1 ref 1 00000100:00100000:1.0:1789760785.626931:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728856566949 time 728872549665 state 0x0 ref 1 00000100:00100000:1.0:1789760785.626944:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554815872:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760785.626988:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65572864, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f460 00010000:00010000:1.0:1789760785.626998:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5ff5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.627013:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760785.627018:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.627024:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5ff5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65572864->65576959] (req 65572864->65576959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.627038:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5ff5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65572864->65576959] (req 65572864->65576959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.634274:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983318 is committed 00002000:00010000:1.0:1789760785.634546:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65572864 -> 65576960 00002000:00010000:1.0:1789760785.634552:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128096 -> 128104 00010000:00010000:1.0:1789760785.634566:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5ff5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65572864->65576959] (req 65572864->65576959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.634578:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5ff5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65572864->65576959] (req 65572864->65576959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.634586:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5ff5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65572864->65576959] (req 65572864->65576959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.634594:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760785.634599:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5ff5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65572864->65576959] (req 65572864->65576959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.634608:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5ff5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65572864->65576959] (req 65572864->65576959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.634827:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5ff5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65572864->65576959] (req 65572864->65576959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.634836:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c5ff5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65572864->65576959] (req 65572864->65576959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.635209:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554815872:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8261us (8655us total) trans 4294983318 rc 0/0 00000100:00100000:1.0:1789760785.635220:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16020 00000100:00100000:1.0:1789760785.635229:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728856566949 time 728872549665 state 0x2 ref 0 00000100:00100000:2.0:1789760785.639779:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760785.640082:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554816000 00000100:00100000:1.0:1789760785.640152:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554816000 00000100:00100000:1.0:1789760785.640275:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728856566949 time 728872549665 state 0x0 ref 1 00000100:00100000:1.0:1789760785.640293:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728872810081 time 728872549665 state 0x1 ref 1 00000100:00100000:1.0:1789760785.640322:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728872810081 time 728885941269 state 0x0 ref 1 00000100:00100000:1.0:1789760785.640335:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ccbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554816000:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760785.640802:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65576960, mtime: 0x6aad9510, atime: 0x0, ctime: 0x6aad9510, blocks: 0x1f468 00010000:00010000:1.0:1789760785.640813:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5ffc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.640829:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760785.640833:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.640839:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5ffc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65576960->65581055] (req 65576960->65581055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.640852:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5ffc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65576960->65581055] (req 65576960->65581055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.646982:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983319 is committed 00002000:00010000:0.0:1789760785.647255:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65576960 -> 65581056 00002000:00010000:0.0:1789760785.647260:0:8442:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760784 -> 1789760785 00002000:00010000:0.0:1789760785.647263:0:8442:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760784 -> 1789760785 00002000:00010000:0.0:1789760785.647265:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128104 -> 128112 00010000:00010000:0.0:1789760785.647278:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5ffc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65576960->65581055] (req 65576960->65581055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.647290:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5ffc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65576960->65581055] (req 65576960->65581055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.647305:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5ffc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65576960->65581055] (req 65576960->65581055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.647312:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760785.647317:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5ffc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65576960->65581055] (req 65576960->65581055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.647325:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5ffc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65576960->65581055] (req 65576960->65581055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.647344:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5ffc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65576960->65581055] (req 65576960->65581055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.647352:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c5ffc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65576960->65581055] (req 65576960->65581055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.647948:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ccbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554816000:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7609us (8173us total) trans 4294983319 rc 0/0 00000100:00100000:0.0:1789760785.647960:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16021 00000100:00100000:0.0:1789760785.647969:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728872810081 time 728885941269 state 0x2 ref 0 00000100:00100000:2.0:1789760785.652275:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760785.652542:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554816128 00000100:00100000:3.0:1789760785.652552:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554816128 00000100:00100000:3.0:1789760785.652674:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728872810081 time 728885941269 state 0x0 ref 1 00000100:00100000:3.0:1789760785.652692:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728886201685 time 728885941269 state 0x1 ref 1 00000100:00100000:3.0:1789760785.652787:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728886201685 time 728898404684 state 0x0 ref 1 00000100:00100000:3.0:1789760785.652801:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03cc9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554816128:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760785.652846:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65581056, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f470 00010000:00010000:3.0:1789760785.652855:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6003 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.652870:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:3.0:1789760785.652884:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.652890:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6003 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65581056->65585151] (req 65581056->65585151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.652903:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6003 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65581056->65585151] (req 65581056->65585151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.660891:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983320 is committed 00002000:00010000:3.0:1789760785.661080:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65581056 -> 65585152 00002000:00010000:3.0:1789760785.661085:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128112 -> 128120 00010000:00010000:3.0:1789760785.661098:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6003 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65581056->65585151] (req 65581056->65585151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.661116:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6003 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65581056->65585151] (req 65581056->65585151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.661125:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6003 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65581056->65585151] (req 65581056->65585151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.661242:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760785.661247:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6003 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65581056->65585151] (req 65581056->65585151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.661257:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6003 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65581056->65585151] (req 65581056->65585151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.661277:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6003 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65581056->65585151] (req 65581056->65585151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.661286:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6003 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65581056->65585151] (req 65581056->65585151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.661638:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03cc9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554816128:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8577us (9110us total) trans 4294983320 rc 0/0 00000100:00100000:3.0:1789760785.661666:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16022 00000100:00100000:3.0:1789760785.661675:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728886201685 time 728898404684 state 0x2 ref 0 00000100:00100000:3.0:1789760785.668816:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760785.669081:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554816256 00000100:00100000:2.0:1789760785.669091:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554816256 00000100:00100000:2.0:1789760785.669144:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728886201685 time 728898404684 state 0x0 ref 1 00000100:00100000:2.0:1789760785.669158:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728898665100 time 728898404684 state 0x1 ref 1 00000100:00100000:2.0:1789760785.669238:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728898665100 time 728914855907 state 0x0 ref 1 00000100:00100000:2.0:1789760785.669250:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554816256:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760785.669296:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65585152, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f478 00010000:00010000:2.0:1789760785.669305:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c600a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.669320:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760785.669324:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.669331:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c600a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65585152->65589247] (req 65585152->65589247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.669350:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c600a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65585152->65589247] (req 65585152->65589247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.676283:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983321 is committed 00002000:00010000:2.0:1789760785.680002:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65585152 -> 65589248 00002000:00010000:2.0:1789760785.680010:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128120 -> 128128 00010000:00010000:2.0:1789760785.680023:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c600a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65585152->65589247] (req 65585152->65589247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.680036:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c600a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65585152->65589247] (req 65585152->65589247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.680046:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c600a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65585152->65589247] (req 65585152->65589247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.680053:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760785.680057:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c600a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65585152->65589247] (req 65585152->65589247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.680065:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c600a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65585152->65589247] (req 65585152->65589247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.681028:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c600a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65585152->65589247] (req 65585152->65589247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.681041:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c600a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65585152->65589247] (req 65585152->65589247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.681681:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554816256:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12426us (12868us total) trans 4294983321 rc 0/0 00000100:00100000:2.0:1789760785.681693:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16023 00000100:00100000:2.0:1789760785.681702:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728898665100 time 728914855907 state 0x2 ref 0 00000100:00100000:2.0:1789760785.688197:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760785.688271:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554816384 00000100:00100000:1.0:1789760785.688282:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554816384 00000100:00100000:1.0:1789760785.688336:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728898665100 time 728914855907 state 0x0 ref 1 00000100:00100000:1.0:1789760785.688437:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728915116323 time 728914855907 state 0x1 ref 1 00000100:00100000:1.0:1789760785.688470:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728915116323 time 728934088020 state 0x0 ref 1 00000100:00100000:1.0:1789760785.688482:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03cca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554816384:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760785.688529:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65589248, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f480 00010000:00010000:1.0:1789760785.688538:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6011 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.688554:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760785.688559:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.688566:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6011 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65589248->65593343] (req 65589248->65593343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.688580:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6011 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65589248->65593343] (req 65589248->65593343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.695085:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983322 is committed 00002000:00010000:1.0:1789760785.695253:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65589248 -> 65593344 00002000:00010000:1.0:1789760785.695259:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128128 -> 128136 00010000:00010000:1.0:1789760785.695273:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6011 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65589248->65593343] (req 65589248->65593343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.695286:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6011 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65589248->65593343] (req 65589248->65593343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.695296:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6011 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65589248->65593343] (req 65589248->65593343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.695303:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760785.695308:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6011 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65589248->65593343] (req 65589248->65593343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.695317:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6011 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65589248->65593343] (req 65589248->65593343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.695334:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6011 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65589248->65593343] (req 65589248->65593343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.695354:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6011 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65589248->65593343] (req 65589248->65593343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.695490:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03cca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554816384:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7005us (7297us total) trans 4294983322 rc 0/0 00000100:00100000:1.0:1789760785.695500:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16024 00000100:00100000:1.0:1789760785.695510:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728915116323 time 728934088020 state 0x2 ref 0 00000100:00100000:3.0:1789760785.704129:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760785.705040:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554816512 00000100:00100000:0.0:1789760785.705051:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554816512 00000100:00100000:0.0:1789760785.705107:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728915116323 time 728934088020 state 0x0 ref 1 00000100:00100000:0.0:1789760785.705123:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728934348436 time 728934088020 state 0x1 ref 1 00000100:00100000:0.0:1789760785.705156:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728934348436 time 728950774883 state 0x0 ref 1 00000100:00100000:0.0:1789760785.705169:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554816512:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760785.705337:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65593344, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f488 00010000:00010000:0.0:1789760785.705348:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6018 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.705363:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:0.0:1789760785.705368:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.705374:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6018 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65593344->65597439] (req 65593344->65597439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.705406:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6018 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65593344->65597439] (req 65593344->65597439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.711826:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983323 is committed 00002000:00010000:2.0:1789760785.712106:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65593344 -> 65597440 00002000:00010000:2.0:1789760785.712111:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128136 -> 128144 00010000:00010000:2.0:1789760785.712125:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6018 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65593344->65597439] (req 65593344->65597439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.712139:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6018 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65593344->65597439] (req 65593344->65597439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.712148:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6018 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65593344->65597439] (req 65593344->65597439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.712155:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760785.712161:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6018 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65593344->65597439] (req 65593344->65597439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.712170:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6018 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65593344->65597439] (req 65593344->65597439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.712188:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6018 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65593344->65597439] (req 65593344->65597439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.712198:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6018 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65593344->65597439] (req 65593344->65597439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.712289:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554816512:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7117us (8165us total) trans 4294983323 rc 0/0 00000100:00100000:2.0:1789760785.712301:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16025 00000100:00100000:2.0:1789760785.712310:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728934348436 time 728950774883 state 0x2 ref 0 00000100:00100000:3.0:1789760785.718599:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760785.722081:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554816640 00000100:00100000:1.0:1789760785.722098:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554816640 00000100:00100000:1.0:1789760785.722164:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728934348436 time 728950774883 state 0x0 ref 1 00000100:00100000:1.0:1789760785.722181:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728951035299 time 728950774883 state 0x1 ref 1 00000100:00100000:1.0:1789760785.723094:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728951035299 time 728968706741 state 0x0 ref 1 00000100:00100000:1.0:1789760785.723114:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe19c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554816640:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760785.723166:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65597440, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f490 00010000:00010000:1.0:1789760785.723175:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c601f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.723191:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:1.0:1789760785.723196:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.723202:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c601f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65597440->65601535] (req 65597440->65601535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.723217:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c601f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65597440->65601535] (req 65597440->65601535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.727466:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983324 is committed 00002000:00010000:1.0:1789760785.727608:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65597440 -> 65601536 00002000:00010000:1.0:1789760785.727614:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128144 -> 128152 00010000:00010000:1.0:1789760785.727627:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c601f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65597440->65601535] (req 65597440->65601535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.727645:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c601f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65597440->65601535] (req 65597440->65601535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.727664:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c601f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65597440->65601535] (req 65597440->65601535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.727672:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760785.727677:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c601f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65597440->65601535] (req 65597440->65601535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.727686:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c601f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65597440->65601535] (req 65597440->65601535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.727708:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c601f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65597440->65601535] (req 65597440->65601535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.727715:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c601f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65597440->65601535] (req 65597440->65601535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.727859:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe19c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554816640:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4747us (9265us total) trans 4294983324 rc 0/0 00000100:00100000:1.0:1789760785.727870:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16026 00000100:00100000:1.0:1789760785.727879:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728951035299 time 728968706741 state 0x2 ref 0 00000100:00100000:3.0:1789760785.736711:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760785.737231:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554816768 00000100:00100000:2.0:1789760785.737241:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554816768 00000100:00100000:2.0:1789760785.737292:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728951035299 time 728968706741 state 0x0 ref 1 00000100:00100000:2.0:1789760785.737307:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728968967157 time 728968706741 state 0x1 ref 1 00000100:00100000:2.0:1789760785.737407:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728968967157 time 728982994176 state 0x0 ref 1 00000100:00100000:2.0:1789760785.737421:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554816768:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760785.737465:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65601536, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f498 00010000:00010000:2.0:1789760785.737474:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6026 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.737490:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=728s v=(0 0) 00010000:00010000:2.0:1789760785.737494:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.737501:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6026 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65601536->65605631] (req 65601536->65605631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.737521:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6026 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65601536->65605631] (req 65601536->65605631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.742104:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983325 is committed 00002000:00010000:2.0:1789760785.748614:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65601536 -> 65605632 00002000:00010000:2.0:1789760785.748623:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128152 -> 128160 00010000:00010000:2.0:1789760785.748636:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6026 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65601536->65605631] (req 65601536->65605631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.748649:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6026 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65601536->65605631] (req 65601536->65605631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.748669:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6026 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65601536->65605631] (req 65601536->65605631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.748677:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760785.748682:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6026 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65601536->65605631] (req 65601536->65605631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.748690:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6026 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65601536->65605631] (req 65601536->65605631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.748709:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6026 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65601536->65605631] (req 65601536->65605631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.748725:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6026 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65601536->65605631] (req 65601536->65605631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.748828:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554816768:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11405us (12121us total) trans 4294983325 rc 0/0 00000100:00100000:2.0:1789760785.748840:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16027 00000100:00100000:2.0:1789760785.748850:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728968967157 time 728982994176 state 0x2 ref 0 00000100:00100000:3.0:1789760785.754129:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760785.756167:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554816896 00000100:00100000:2.0:1789760785.756182:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554816896 00000100:00100000:2.0:1789760785.756238:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728968967157 time 728982994176 state 0x0 ref 1 00000100:00100000:2.0:1789760785.756254:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728983254592 time 728982994176 state 0x1 ref 1 00000100:00100000:2.0:1789760785.756289:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728983254592 time 729001907254 state 0x0 ref 1 00000100:00100000:2.0:1789760785.756303:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554816896:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760785.756349:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65605632, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f4a0 00010000:00010000:2.0:1789760785.756357:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c602d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.756373:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760785.756396:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.756403:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c602d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65605632->65609727] (req 65605632->65609727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.756431:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c602d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65605632->65609727] (req 65605632->65609727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760785.764751:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983326 is committed 00002000:00010000:2.0:1789760785.764938:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65605632 -> 65609728 00002000:00010000:2.0:1789760785.764942:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128160 -> 128168 00010000:00010000:2.0:1789760785.764953:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c602d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65605632->65609727] (req 65605632->65609727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.764961:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c602d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65605632->65609727] (req 65605632->65609727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.764967:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c602d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65605632->65609727] (req 65605632->65609727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.764971:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760785.764974:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c602d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65605632->65609727] (req 65605632->65609727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.764980:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c602d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65605632->65609727] (req 65605632->65609727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.764994:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c602d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65605632->65609727] (req 65605632->65609727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.764999:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c602d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65605632->65609727] (req 65605632->65609727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.765079:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554816896:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8774us (10956us total) trans 4294983326 rc 0/0 00000100:00100000:2.0:1789760785.765089:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16028 00000100:00100000:2.0:1789760785.765098:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728983254592 time 729001907254 state 0x2 ref 0 00000100:00100000:3.0:1789760785.769318:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760785.769944:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554817024 00000100:00100000:0.0:1789760785.769955:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554817024 00000100:00100000:0.0:1789760785.770014:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 728983254592 time 729001907254 state 0x0 ref 1 00000100:00100000:0.0:1789760785.770039:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729002167670 time 729001907254 state 0x1 ref 1 00000100:00100000:0.0:1789760785.770178:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729002167670 time 729015794899 state 0x0 ref 1 00000100:00100000:0.0:1789760785.770192:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554817024:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760785.770239:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65609728, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f4a8 00010000:00010000:0.0:1789760785.770247:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6034 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.770262:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760785.770266:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.770273:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6034 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65609728->65613823] (req 65609728->65613823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.770285:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6034 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65609728->65613823] (req 65609728->65613823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.781581:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983327 is committed 00002000:00010000:1.0:1789760785.781902:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65609728 -> 65613824 00002000:00010000:1.0:1789760785.781907:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128168 -> 128176 00010000:00010000:1.0:1789760785.781921:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6034 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65609728->65613823] (req 65609728->65613823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.781935:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6034 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65609728->65613823] (req 65609728->65613823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.781944:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6034 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65609728->65613823] (req 65609728->65613823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.781951:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760785.781956:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6034 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65609728->65613823] (req 65609728->65613823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.781965:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6034 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65609728->65613823] (req 65609728->65613823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.781985:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6034 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65609728->65613823] (req 65609728->65613823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.781993:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6034 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65609728->65613823] (req 65609728->65613823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.782193:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554817024:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11998us (12880us total) trans 4294983327 rc 0/0 00000100:00100000:1.0:1789760785.782204:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16029 00000100:00100000:1.0:1789760785.782214:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729002167670 time 729015794899 state 0x2 ref 0 00000100:00100000:3.0:1789760785.788309:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760785.789218:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554817152 00000100:00100000:1.0:1789760785.789232:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554817152 00000100:00100000:1.0:1789760785.789298:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729002167670 time 729015794899 state 0x0 ref 1 00000100:00100000:1.0:1789760785.789316:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729016055315 time 729015794899 state 0x1 ref 1 00000100:00100000:1.0:1789760785.789349:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729016055315 time 729034968067 state 0x0 ref 1 00000100:00100000:1.0:1789760785.789362:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554817152:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760785.789669:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65613824, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f4b0 00010000:00010000:1.0:1789760785.789679:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c603b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.791769:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760785.791775:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.791783:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c603b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65613824->65617919] (req 65613824->65617919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.791802:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c603b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65613824->65617919] (req 65613824->65617919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.800427:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983328 is committed 00002000:00010000:3.0:1789760785.800631:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65613824 -> 65617920 00002000:00010000:3.0:1789760785.800636:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128176 -> 128184 00010000:00010000:3.0:1789760785.800648:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c603b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65613824->65617919] (req 65613824->65617919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.800670:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c603b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65613824->65617919] (req 65613824->65617919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.800678:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c603b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65613824->65617919] (req 65613824->65617919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.800687:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760785.800692:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c603b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65613824->65617919] (req 65613824->65617919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.800700:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c603b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65613824->65617919] (req 65613824->65617919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.800717:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c603b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65613824->65617919] (req 65613824->65617919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.800728:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c603b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65613824->65617919] (req 65613824->65617919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.800821:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554817152:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11455us (12516us total) trans 4294983328 rc 0/0 00000100:00100000:3.0:1789760785.800831:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16030 00000100:00100000:3.0:1789760785.800840:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729016055315 time 729034968067 state 0x2 ref 0 00000100:00100000:3.0:1789760785.803786:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760785.805219:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554817280 00000100:00100000:2.0:1789760785.805231:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554817280 00000100:00100000:2.0:1789760785.805290:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729016055315 time 729034968067 state 0x0 ref 1 00000100:00100000:2.0:1789760785.805307:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729035228483 time 729034968067 state 0x1 ref 1 00000100:00100000:2.0:1789760785.805344:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729035228483 time 729050962754 state 0x0 ref 1 00000100:00100000:2.0:1789760785.805357:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554817280:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760785.805449:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65617920, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f4b8 00010000:00010000:2.0:1789760785.805459:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6042 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.805475:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760785.805480:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.805486:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6042 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65617920->65622015] (req 65617920->65622015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.805501:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6042 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65617920->65622015] (req 65617920->65622015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.810979:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573918848:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1789760785.811042:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760785.811095:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573918848 00000100:00100000:2.0:1789760785.811104:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573918848 00000100:00100000:2.0:1789760785.811152:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 389 00000100:00100000:2.0:1789760785.811161:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07b36450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573918848:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789760785.811301:0:6525:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446383616 avail 1300144128 00000100:00100000:2.0:1789760785.811357:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07b36450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573918848:12345-0@lo:41:osp-pre-1-0.0 Request processed in 197us (314us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760785.811365:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 389 00000100:00100000:3.0:1789760785.811431:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573918848:0@lo:41:osp-pre-1-0.0 00000001:00080000:1.0:1789760785.814269:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983329 is committed 00002000:00010000:3.0:1789760785.814517:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65617920 -> 65622016 00002000:00010000:3.0:1789760785.814524:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128184 -> 128192 00010000:00010000:3.0:1789760785.814537:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6042 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65617920->65622015] (req 65617920->65622015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.814550:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6042 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65617920->65622015] (req 65617920->65622015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.814560:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6042 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65617920->65622015] (req 65617920->65622015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.814567:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760785.814571:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6042 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65617920->65622015] (req 65617920->65622015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.814726:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6042 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65617920->65622015] (req 65617920->65622015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.814755:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6042 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65617920->65622015] (req 65617920->65622015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.814764:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6042 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65617920->65622015] (req 65617920->65622015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.814862:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554817280:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9503us (11080us total) trans 4294983329 rc 0/0 00000100:00100000:3.0:1789760785.814874:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16031 00000100:00100000:3.0:1789760785.814883:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729035228483 time 729050962754 state 0x2 ref 0 00000100:00100000:1.0:1789760785.818022:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760785.818150:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554817408 00000100:00100000:2.0:1789760785.818159:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554817408 00000100:00100000:2.0:1789760785.818270:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729035228483 time 729050962754 state 0x0 ref 1 00000100:00100000:2.0:1789760785.818287:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729051223170 time 729050962754 state 0x1 ref 1 00000100:00100000:2.0:1789760785.818316:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729051223170 time 729063935370 state 0x0 ref 1 00000100:00100000:2.0:1789760785.818329:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554817408:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760785.818405:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65622016, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f4c0 00010000:00010000:2.0:1789760785.818640:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6049 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.818665:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760785.818670:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.818677:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6049 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65622016->65626111] (req 65622016->65626111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.818691:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6049 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65622016->65626111] (req 65622016->65626111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.824486:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983330 is committed 00002000:00010000:1.0:1789760785.824594:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65622016 -> 65626112 00002000:00010000:1.0:1789760785.824600:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128192 -> 128200 00010000:00010000:1.0:1789760785.824614:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6049 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65622016->65626111] (req 65622016->65626111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.824627:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6049 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65622016->65626111] (req 65622016->65626111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.824636:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6049 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65622016->65626111] (req 65622016->65626111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.824643:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760785.824648:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6049 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65622016->65626111] (req 65622016->65626111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.824669:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6049 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65622016->65626111] (req 65622016->65626111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.824687:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6049 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65622016->65626111] (req 65622016->65626111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.824696:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6049 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65622016->65626111] (req 65622016->65626111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.825074:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554817408:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6741us (7061us total) trans 4294983330 rc 0/0 00000100:00100000:1.0:1789760785.825085:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16032 00000100:00100000:1.0:1789760785.825094:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729051223170 time 729063935370 state 0x2 ref 0 00000100:00100000:2.0:1789760785.835310:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760785.835410:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554817536 00000100:00100000:1.0:1789760785.835420:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554817536 00000100:00100000:1.0:1789760785.835490:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729051223170 time 729063935370 state 0x0 ref 1 00000100:00100000:1.0:1789760785.835506:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729064195786 time 729063935370 state 0x1 ref 1 00000100:00100000:1.0:1789760785.835542:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729064195786 time 729081160762 state 0x0 ref 1 00000100:00100000:1.0:1789760785.835555:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ccb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554817536:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760785.835602:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65626112, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f4c8 00010000:00010000:1.0:1789760785.835612:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6050 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.835627:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760785.835632:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.835638:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6050 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65626112->65630207] (req 65626112->65630207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.835650:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6050 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65626112->65630207] (req 65626112->65630207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.845101:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983331 is committed 00002000:00010000:2.0:1789760785.845328:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65626112 -> 65630208 00002000:00010000:2.0:1789760785.845333:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128200 -> 128208 00010000:00010000:2.0:1789760785.845347:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6050 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65626112->65630207] (req 65626112->65630207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.845359:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6050 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65626112->65630207] (req 65626112->65630207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.845367:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6050 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65626112->65630207] (req 65626112->65630207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.845373:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760785.845786:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6050 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65626112->65630207] (req 65626112->65630207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.845798:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6050 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65626112->65630207] (req 65626112->65630207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.845820:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6050 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65626112->65630207] (req 65626112->65630207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.845829:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6050 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65626112->65630207] (req 65626112->65630207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.846040:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ccb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554817536:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10475us (10726us total) trans 4294983331 rc 0/0 00000100:00100000:2.0:1789760785.846050:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16033 00000100:00100000:2.0:1789760785.846059:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729064195786 time 729081160762 state 0x2 ref 0 00000100:00100000:3.0:1789760785.849974:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760785.850398:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554817664 00000100:00100000:1.0:1789760785.850408:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554817664 00000100:00100000:1.0:1789760785.850456:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729064195786 time 729081160762 state 0x0 ref 1 00000100:00100000:1.0:1789760785.850472:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729081421178 time 729081160762 state 0x1 ref 1 00000100:00100000:1.0:1789760785.850537:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729081421178 time 729096154776 state 0x0 ref 1 00000100:00100000:1.0:1789760785.850550:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554817664:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760785.850598:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65630208, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f4d0 00010000:00010000:1.0:1789760785.850616:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6057 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.850632:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760785.850637:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.850643:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6057 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65630208->65634303] (req 65630208->65634303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.850663:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6057 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65630208->65634303] (req 65630208->65634303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.857271:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983332 is committed 00002000:00010000:1.0:1789760785.857448:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65630208 -> 65634304 00002000:00010000:1.0:1789760785.857453:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128208 -> 128216 00010000:00010000:1.0:1789760785.857466:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6057 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65630208->65634303] (req 65630208->65634303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.857479:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6057 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65630208->65634303] (req 65630208->65634303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.857489:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6057 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65630208->65634303] (req 65630208->65634303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.857503:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760785.857508:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6057 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65630208->65634303] (req 65630208->65634303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.857518:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6057 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65630208->65634303] (req 65630208->65634303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.857539:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6057 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65630208->65634303] (req 65630208->65634303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.857547:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6057 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65630208->65634303] (req 65630208->65634303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.857648:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554817664:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7095us (7678us total) trans 4294983332 rc 0/0 00000100:00100000:1.0:1789760785.857669:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16034 00000100:00100000:1.0:1789760785.858333:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729081421178 time 729096154776 state 0x2 ref 0 00000100:00100000:1.0:1789760785.863312:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760785.867921:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554817792 00000100:00100000:3.0:1789760785.867937:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554817792 00000100:00100000:3.0:1789760785.868013:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729081421178 time 729096154776 state 0x0 ref 1 00000100:00100000:3.0:1789760785.868030:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729096415192 time 729096154776 state 0x1 ref 1 00000100:00100000:3.0:1789760785.868064:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729096415192 time 729113681888 state 0x0 ref 1 00000100:00100000:3.0:1789760785.868077:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554817792:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760785.868140:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65634304, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f4d8 00010000:00010000:3.0:1789760785.868151:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c605e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.868167:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760785.868172:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.868178:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c605e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65634304->65638399] (req 65634304->65638399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.868192:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c605e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65634304->65638399] (req 65634304->65638399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760785.876999:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983333 is committed 00002000:00010000:2.0:1789760785.877751:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65634304 -> 65638400 00002000:00010000:2.0:1789760785.877757:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128216 -> 128224 00010000:00010000:2.0:1789760785.877771:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c605e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65634304->65638399] (req 65634304->65638399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.877783:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c605e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65634304->65638399] (req 65634304->65638399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.877793:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c605e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65634304->65638399] (req 65634304->65638399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.877800:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760785.877806:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c605e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65634304->65638399] (req 65634304->65638399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.878101:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c605e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65634304->65638399] (req 65634304->65638399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.878322:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c605e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65634304->65638399] (req 65634304->65638399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.878337:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c605e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65634304->65638399] (req 65634304->65638399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.878558:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554817792:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10478us (15248us total) trans 4294983333 rc 0/0 00000100:00100000:2.0:1789760785.878569:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16035 00000100:00100000:2.0:1789760785.878579:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729096415192 time 729113681888 state 0x2 ref 0 00000100:00100000:2.0:1789760785.887460:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760785.887564:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554817920 00000100:00100000:1.0:1789760785.887573:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554817920 00000100:00100000:1.0:1789760785.887627:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729096415192 time 729113681888 state 0x0 ref 1 00000100:00100000:1.0:1789760785.887643:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729113942304 time 729113681888 state 0x1 ref 1 00000100:00100000:1.0:1789760785.887687:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729113942304 time 729133304948 state 0x0 ref 1 00000100:00100000:1.0:1789760785.887701:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03cc8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554817920:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760785.887745:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65638400, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f4e0 00010000:00010000:1.0:1789760785.887753:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6065 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.887769:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760785.887773:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.887780:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6065 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65638400->65642495] (req 65638400->65642495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.887793:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6065 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65638400->65642495] (req 65638400->65642495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.893311:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983334 is committed 00002000:00010000:1.0:1789760785.893444:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65638400 -> 65642496 00002000:00010000:1.0:1789760785.893449:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128224 -> 128232 00010000:00010000:1.0:1789760785.893462:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6065 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65638400->65642495] (req 65638400->65642495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.893476:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6065 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65638400->65642495] (req 65638400->65642495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.893485:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6065 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65638400->65642495] (req 65638400->65642495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.893494:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760785.893499:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6065 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65638400->65642495] (req 65638400->65642495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.893509:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6065 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65638400->65642495] (req 65638400->65642495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.893529:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6065 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65638400->65642495] (req 65638400->65642495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.893537:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6065 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65638400->65642495] (req 65638400->65642495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760785.893686:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03cc8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554817920:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5976us (6224us total) trans 4294983334 rc 0/0 00000100:00100000:1.0:1789760785.893698:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16036 00000100:00100000:1.0:1789760785.893709:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729113942304 time 729133304948 state 0x2 ref 0 00000100:00100000:3.0:1789760785.899228:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760785.899340:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554818048 00000100:00100000:2.0:1789760785.899350:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554818048 00000100:00100000:2.0:1789760785.899450:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729113942304 time 729133304948 state 0x0 ref 1 00000100:00100000:2.0:1789760785.899465:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729133565364 time 729133304948 state 0x1 ref 1 00000100:00100000:2.0:1789760785.899496:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729133565364 time 729145114178 state 0x0 ref 1 00000100:00100000:2.0:1789760785.899508:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554818048:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760785.899556:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65642496, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f4e8 00010000:00010000:2.0:1789760785.899566:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c606c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.899583:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760785.899587:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.899594:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c606c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65642496->65646591] (req 65642496->65646591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.899609:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c606c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65642496->65646591] (req 65642496->65646591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.908828:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983335 is committed 00002000:00010000:2.0:1789760785.910985:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65642496 -> 65646592 00002000:00010000:2.0:1789760785.910993:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128232 -> 128240 00010000:00010000:2.0:1789760785.911008:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c606c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65642496->65646591] (req 65642496->65646591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.911020:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c606c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65642496->65646591] (req 65642496->65646591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.911199:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c606c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65642496->65646591] (req 65642496->65646591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.911209:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760785.911214:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c606c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65642496->65646591] (req 65642496->65646591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.911224:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c606c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65642496->65646591] (req 65642496->65646591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.911244:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c606c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65642496->65646591] (req 65642496->65646591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.911256:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c606c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65642496->65646591] (req 65642496->65646591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.911572:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554818048:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12059us (12347us total) trans 4294983335 rc 0/0 00000100:00100000:2.0:1789760785.911584:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16037 00000100:00100000:2.0:1789760785.911593:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729133565364 time 729145114178 state 0x2 ref 0 00000100:00100000:3.0:1789760785.915633:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760785.915736:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554818176 00000100:00100000:0.0:1789760785.915747:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554818176 00000100:00100000:0.0:1789760785.915800:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729133565364 time 729145114178 state 0x0 ref 1 00000100:00100000:0.0:1789760785.915814:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729145374594 time 729145114178 state 0x1 ref 1 00000100:00100000:0.0:1789760785.915840:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729145374594 time 729161459040 state 0x0 ref 1 00000100:00100000:0.0:1789760785.915852:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554818176:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760785.915898:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65646592, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f4f0 00010000:00010000:0.0:1789760785.915907:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6073 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.915922:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760785.915927:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.915933:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6073 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65646592->65650687] (req 65646592->65650687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.915946:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6073 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65646592->65650687] (req 65646592->65650687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.922076:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983336 is committed 00002000:00010000:0.0:1789760785.922374:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65646592 -> 65650688 00002000:00010000:0.0:1789760785.922510:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128240 -> 128248 00010000:00010000:0.0:1789760785.922524:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6073 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65646592->65650687] (req 65646592->65650687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.922537:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6073 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65646592->65650687] (req 65646592->65650687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.922547:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6073 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65646592->65650687] (req 65646592->65650687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.922555:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760785.922559:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6073 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65646592->65650687] (req 65646592->65650687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.922568:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6073 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65646592->65650687] (req 65646592->65650687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.922588:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6073 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65646592->65650687] (req 65646592->65650687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.922596:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6073 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65646592->65650687] (req 65646592->65650687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.922762:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554818176:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6906us (7133us total) trans 4294983336 rc 0/0 00000100:00100000:0.0:1789760785.922774:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16038 00000100:00100000:0.0:1789760785.922784:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729145374594 time 729161459040 state 0x2 ref 0 00000100:00100000:1.0:1789760785.926869:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760785.927526:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554818304 00000100:00100000:3.0:1789760785.927537:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554818304 00000100:00100000:3.0:1789760785.927593:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729145374594 time 729161459040 state 0x0 ref 1 00000100:00100000:3.0:1789760785.927609:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729161719456 time 729161459040 state 0x1 ref 1 00000100:00100000:3.0:1789760785.927645:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729161719456 time 729173262747 state 0x0 ref 1 00000100:00100000:3.0:1789760785.927668:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554818304:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760785.927722:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65650688, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f4f8 00010000:00010000:3.0:1789760785.927732:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c607a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.927748:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760785.927753:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.927759:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c607a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65650688->65654783] (req 65650688->65654783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.927774:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c607a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65650688->65654783] (req 65650688->65654783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.935597:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983337 is committed 00002000:00010000:3.0:1789760785.935841:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65650688 -> 65654784 00002000:00010000:3.0:1789760785.935847:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128248 -> 128256 00010000:00010000:3.0:1789760785.935860:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c607a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65650688->65654783] (req 65650688->65654783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.935873:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c607a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65650688->65654783] (req 65650688->65654783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.935883:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c607a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65650688->65654783] (req 65650688->65654783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.935891:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760785.935896:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c607a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65650688->65654783] (req 65650688->65654783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.935905:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c607a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65650688->65654783] (req 65650688->65654783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.935929:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c607a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65650688->65654783] (req 65650688->65654783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.935937:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c607a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65650688->65654783] (req 65650688->65654783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.936073:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554818304:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8413us (9209us total) trans 4294983337 rc 0/0 00000100:00100000:3.0:1789760785.936084:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16039 00000100:00100000:3.0:1789760785.936093:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729161719456 time 729173262747 state 0x2 ref 0 00000100:00100000:2.0:1789760785.940228:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760785.940302:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554818432 00000100:00100000:0.0:1789760785.940313:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554818432 00000100:00100000:0.0:1789760785.940367:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729161719456 time 729173262747 state 0x0 ref 1 00000100:00100000:0.0:1789760785.940406:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729173523163 time 729173262747 state 0x1 ref 1 00000100:00100000:0.0:1789760785.940488:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729173523163 time 729186106898 state 0x0 ref 1 00000100:00100000:0.0:1789760785.940500:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ccb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554818432:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760785.940545:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65654784, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f500 00010000:00010000:0.0:1789760785.940554:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6081 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.940568:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760785.940573:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.940579:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6081 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65654784->65658879] (req 65654784->65658879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.940591:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6081 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65654784->65658879] (req 65654784->65658879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.947572:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983338 is committed 00002000:00010000:0.0:1789760785.949797:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65654784 -> 65658880 00002000:00010000:0.0:1789760785.949808:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128256 -> 128264 00010000:00010000:0.0:1789760785.949822:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6081 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65654784->65658879] (req 65654784->65658879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.949836:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6081 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65654784->65658879] (req 65654784->65658879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.949846:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6081 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65654784->65658879] (req 65654784->65658879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.949853:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760785.949858:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6081 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65654784->65658879] (req 65654784->65658879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.949869:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6081 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65654784->65658879] (req 65654784->65658879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.949889:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6081 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65654784->65658879] (req 65654784->65658879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.949902:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6081 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65654784->65658879] (req 65654784->65658879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760785.950058:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ccb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554818432:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9552us (9831us total) trans 4294983338 rc 0/0 00000100:00100000:0.0:1789760785.950069:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16040 00000100:00100000:0.0:1789760785.950078:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729173523163 time 729186106898 state 0x2 ref 0 00000100:00100000:2.0:1789760785.954526:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760785.954617:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554818560 00000100:00100000:1.0:1789760785.954631:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554818560 00000100:00100000:1.0:1789760785.954709:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729173523163 time 729186106898 state 0x0 ref 1 00000100:00100000:1.0:1789760785.954726:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729186367314 time 729186106898 state 0x1 ref 1 00000100:00100000:1.0:1789760785.954756:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729186367314 time 729200374497 state 0x0 ref 1 00000100:00100000:1.0:1789760785.954770:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03cc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554818560:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760785.954816:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65658880, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f508 00010000:00010000:1.0:1789760785.954825:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6088 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.954843:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760785.954847:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760785.954855:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6088 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65658880->65662975] (req 65658880->65662975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760785.954869:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6088 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65658880->65662975] (req 65658880->65662975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.960052:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983339 is committed 00002000:00010000:2.0:1789760785.960309:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65658880 -> 65662976 00002000:00010000:2.0:1789760785.960315:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128264 -> 128272 00010000:00010000:2.0:1789760785.960327:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6088 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65658880->65662975] (req 65658880->65662975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.960340:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6088 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65658880->65662975] (req 65658880->65662975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.960349:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6088 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65658880->65662975] (req 65658880->65662975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.960357:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760785.960361:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6088 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65658880->65662975] (req 65658880->65662975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.960371:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6088 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65658880->65662975] (req 65658880->65662975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.960526:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6088 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65658880->65662975] (req 65658880->65662975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.960535:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6088 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65658880->65662975] (req 65658880->65662975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760785.960630:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03cc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554818560:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5856us (6108us total) trans 4294983339 rc 0/0 00000100:00100000:2.0:1789760785.960641:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16041 00000100:00100000:2.0:1789760785.960651:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729186367314 time 729200374497 state 0x2 ref 0 00000100:00100000:1.0:1789760785.968752:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760785.969798:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554818688 00000100:00100000:3.0:1789760785.969808:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554818688 00000100:00100000:3.0:1789760785.969867:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729186367314 time 729200374497 state 0x0 ref 1 00000100:00100000:3.0:1789760785.969882:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729200634913 time 729200374497 state 0x1 ref 1 00000100:00100000:3.0:1789760785.969903:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729200634913 time 729215522088 state 0x0 ref 1 00000100:00100000:3.0:1789760785.969915:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554818688:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760785.969957:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65662976, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f510 00010000:00010000:3.0:1789760785.969966:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c608f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.969981:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760785.969986:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760785.969992:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c608f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65662976->65667071] (req 65662976->65667071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.970005:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c608f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65662976->65667071] (req 65662976->65667071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760785.976633:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983340 is committed 00002000:00010000:3.0:1789760785.976837:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65662976 -> 65667072 00002000:00010000:3.0:1789760785.976908:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128272 -> 128280 00010000:00010000:3.0:1789760785.976924:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c608f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65662976->65667071] (req 65662976->65667071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.976937:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c608f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65662976->65667071] (req 65662976->65667071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.976948:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c608f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65662976->65667071] (req 65662976->65667071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.976955:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760785.976960:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c608f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65662976->65667071] (req 65662976->65667071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.976969:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c608f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65662976->65667071] (req 65662976->65667071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.976990:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c608f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65662976->65667071] (req 65662976->65667071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.977004:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c608f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65662976->65667071] (req 65662976->65667071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.977222:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554818688:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7295us (8466us total) trans 4294983340 rc 0/0 00000100:00100000:3.0:1789760785.977233:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16042 00000100:00100000:3.0:1789760785.977243:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729200634913 time 729215522088 state 0x2 ref 0 00000100:00100000:3.0:1789760785.983073:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760785.983315:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554818816 00000100:00100000:2.0:1789760785.983326:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554818816 00000100:00100000:2.0:1789760785.983431:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729200634913 time 729215522088 state 0x0 ref 1 00000100:00100000:2.0:1789760785.983447:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729215782504 time 729215522088 state 0x1 ref 1 00000100:00100000:2.0:1789760785.983482:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729215782504 time 729229100786 state 0x0 ref 1 00000100:00100000:2.0:1789760785.983496:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554818816:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760785.983543:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65667072, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f518 00010000:00010000:2.0:1789760785.983553:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6096 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.983569:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760785.983585:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760785.983591:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6096 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65667072->65671167] (req 65667072->65671167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760785.983604:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6096 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65667072->65671167] (req 65667072->65671167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760785.989591:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983341 is committed 00002000:00010000:3.0:1789760785.989836:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65667072 -> 65671168 00002000:00010000:3.0:1789760785.989842:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128280 -> 128288 00010000:00010000:3.0:1789760785.989855:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6096 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65667072->65671167] (req 65667072->65671167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.989868:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6096 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65667072->65671167] (req 65667072->65671167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.989877:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6096 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65667072->65671167] (req 65667072->65671167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.989895:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760785.989900:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6096 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65667072->65671167] (req 65667072->65671167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.989910:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6096 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65667072->65671167] (req 65667072->65671167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.989929:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6096 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65667072->65671167] (req 65667072->65671167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760785.989937:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6096 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65667072->65671167] (req 65667072->65671167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760785.990093:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554818816:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6594us (7023us total) trans 4294983341 rc 0/0 00000100:00100000:3.0:1789760785.990106:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16043 00000100:00100000:3.0:1789760785.990115:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729215782504 time 729229100786 state 0x2 ref 0 00000100:00100000:2.0:1789760785.997788:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760785.997950:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554818944 00000100:00100000:0.0:1789760785.997961:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554818944 00000100:00100000:0.0:1789760785.998021:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729215782504 time 729229100786 state 0x0 ref 1 00000100:00100000:0.0:1789760785.998101:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729229361202 time 729229100786 state 0x1 ref 1 00000100:00100000:0.0:1789760785.998171:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729229361202 time 729243788798 state 0x0 ref 1 00000100:00100000:0.0:1789760785.998185:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03cc8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554818944:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760785.998241:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65671168, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f520 00010000:00010000:0.0:1789760785.998251:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c609d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.998266:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760785.998271:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760785.998278:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c609d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65671168->65675263] (req 65671168->65675263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760785.998296:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c609d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65671168->65675263] (req 65671168->65675263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.001882:0:6508:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760786.003595:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983342 is committed 00002000:00010000:0.0:1789760786.003725:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65671168 -> 65675264 00002000:00010000:0.0:1789760786.003731:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128288 -> 128296 00010000:00010000:0.0:1789760786.003744:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c609d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65671168->65675263] (req 65671168->65675263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.003832:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c609d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65671168->65675263] (req 65671168->65675263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.003843:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c609d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65671168->65675263] (req 65671168->65675263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.003850:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760786.003855:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c609d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65671168->65675263] (req 65671168->65675263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.003865:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c609d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65671168->65675263] (req 65671168->65675263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.003956:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c609d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65671168->65675263] (req 65671168->65675263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.003965:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c609d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65671168->65675263] (req 65671168->65675263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.004159:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03cc8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554818944:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5972us (6443us total) trans 4294983342 rc 0/0 00000100:00100000:0.0:1789760786.004172:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16044 00000100:00100000:0.0:1789760786.004182:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729229361202 time 729243788798 state 0x2 ref 0 00000100:00100000:1.0:1789760786.010941:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760786.011109:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554819072 00000100:00100000:2.0:1789760786.011120:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554819072 00000100:00100000:2.0:1789760786.011177:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729229361202 time 729243788798 state 0x0 ref 1 00000100:00100000:2.0:1789760786.011192:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729244049214 time 729243788798 state 0x1 ref 1 00000100:00100000:2.0:1789760786.011270:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729244049214 time 729256887895 state 0x0 ref 1 00000100:00100000:2.0:1789760786.011284:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554819072:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760786.011341:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65675264, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f528 00010000:00010000:2.0:1789760786.011351:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c60a4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.011366:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760786.011370:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.011425:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c60a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65675264->65679359] (req 65675264->65679359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.011440:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c60a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65675264->65679359] (req 65675264->65679359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.015998:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983343 is committed 00002000:00010000:2.0:1789760786.017138:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65675264 -> 65679360 00002000:00010000:2.0:1789760786.017147:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128296 -> 128304 00010000:00010000:2.0:1789760786.017161:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c60a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65675264->65679359] (req 65675264->65679359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.017174:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c60a4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65675264->65679359] (req 65675264->65679359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.017185:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c60a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65675264->65679359] (req 65675264->65679359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.017193:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760786.017198:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c60a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65675264->65679359] (req 65675264->65679359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.017207:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c60a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65675264->65679359] (req 65675264->65679359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.017230:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c60a4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65675264->65679359] (req 65675264->65679359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.017241:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c60a4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65675264->65679359] (req 65675264->65679359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.017347:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554819072:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6060us (6411us total) trans 4294983343 rc 0/0 00000100:00100000:2.0:1789760786.017358:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16045 00000100:00100000:2.0:1789760786.017368:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729244049214 time 729256887895 state 0x2 ref 0 00000100:00100000:3.0:1789760786.021070:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760786.021196:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554819200 00000100:00100000:0.0:1789760786.021205:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554819200 00000100:00100000:0.0:1789760786.021258:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729244049214 time 729256887895 state 0x0 ref 1 00000100:00100000:0.0:1789760786.021273:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729257148311 time 729256887895 state 0x1 ref 1 00000100:00100000:0.0:1789760786.021304:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729257148311 time 729266921934 state 0x0 ref 1 00000100:00100000:0.0:1789760786.021316:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554819200:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760786.021365:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65679360, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f530 00010000:00010000:0.0:1789760786.021374:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c60ab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.021438:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760786.021443:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.021450:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c60ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65679360->65683455] (req 65679360->65683455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.021463:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c60ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65679360->65683455] (req 65679360->65683455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.025790:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983344 is committed 00002000:00010000:0.0:1789760786.025904:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65679360 -> 65683456 00002000:00010000:0.0:1789760786.025909:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128304 -> 128312 00010000:00010000:0.0:1789760786.025921:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c60ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65679360->65683455] (req 65679360->65683455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.025934:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c60ab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65679360->65683455] (req 65679360->65683455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.025942:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c60ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65679360->65683455] (req 65679360->65683455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.025948:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760786.025952:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c60ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65679360->65683455] (req 65679360->65683455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.025960:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c60ab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65679360->65683455] (req 65679360->65683455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.025978:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c60ab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65679360->65683455] (req 65679360->65683455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.025984:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c60ab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65679360->65683455] (req 65679360->65683455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.026081:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554819200:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4761us (5016us total) trans 4294983344 rc 0/0 00000100:00100000:0.0:1789760786.026090:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16046 00000100:00100000:0.0:1789760786.026099:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729257148311 time 729266921934 state 0x2 ref 0 00000100:00100000:3.0:1789760786.031426:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760786.031548:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554819328 00000100:00100000:2.0:1789760786.031557:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554819328 00000100:00100000:2.0:1789760786.031608:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729257148311 time 729266921934 state 0x0 ref 1 00000100:00100000:2.0:1789760786.031623:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729267182350 time 729266921934 state 0x1 ref 1 00000100:00100000:2.0:1789760786.031666:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729267182350 time 729277283416 state 0x0 ref 1 00000100:00100000:2.0:1789760786.031678:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554819328:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760786.031721:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65683456, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f538 00010000:00010000:2.0:1789760786.031729:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c60b2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.031746:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760786.031750:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.031756:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c60b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65683456->65687551] (req 65683456->65687551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.031777:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c60b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65683456->65687551] (req 65683456->65687551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.037932:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983345 is committed 00002000:00010000:3.0:1789760786.038062:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65683456 -> 65687552 00002000:00010000:3.0:1789760786.038067:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128312 -> 128320 00010000:00010000:3.0:1789760786.038080:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c60b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65683456->65687551] (req 65683456->65687551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.038093:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c60b2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65683456->65687551] (req 65683456->65687551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.038103:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c60b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65683456->65687551] (req 65683456->65687551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.038111:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760786.038117:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c60b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65683456->65687551] (req 65683456->65687551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.038127:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c60b2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65683456->65687551] (req 65683456->65687551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.038146:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c60b2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65683456->65687551] (req 65683456->65687551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.038155:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c60b2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65683456->65687551] (req 65683456->65687551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.038245:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554819328:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6564us (6823us total) trans 4294983345 rc 0/0 00000100:00100000:3.0:1789760786.038256:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16047 00000100:00100000:3.0:1789760786.038265:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729267182350 time 729277283416 state 0x2 ref 0 00000100:00100000:1.0:1789760786.041767:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760786.041937:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554819456 00000100:00100000:0.0:1789760786.041949:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554819456 00000100:00100000:0.0:1789760786.042005:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729267182350 time 729277283416 state 0x0 ref 1 00000100:00100000:0.0:1789760786.042021:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729277543832 time 729277283416 state 0x1 ref 1 00000100:00100000:0.0:1789760786.042054:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729277543832 time 729287672646 state 0x0 ref 1 00000100:00100000:0.0:1789760786.042066:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554819456:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760786.042115:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65687552, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f540 00010000:00010000:0.0:1789760786.042125:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c60b9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.042140:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760786.042144:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.042151:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c60b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65687552->65691647] (req 65687552->65691647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.042164:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c60b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65687552->65691647] (req 65687552->65691647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.049960:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983346 is committed 00002000:00010000:0.0:1789760786.052440:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65687552 -> 65691648 00002000:00010000:0.0:1789760786.052451:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128320 -> 128328 00010000:00010000:0.0:1789760786.052465:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c60b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65687552->65691647] (req 65687552->65691647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.052479:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c60b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65687552->65691647] (req 65687552->65691647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.052488:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c60b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65687552->65691647] (req 65687552->65691647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.052496:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760786.052501:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c60b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65687552->65691647] (req 65687552->65691647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.052510:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c60b9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65687552->65691647] (req 65687552->65691647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.052531:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c60b9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65687552->65691647] (req 65687552->65691647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.052539:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c60b9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65687552->65691647] (req 65687552->65691647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.052641:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554819456:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10572us (10888us total) trans 4294983346 rc 0/0 00000100:00100000:0.0:1789760786.052661:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16048 00000100:00100000:0.0:1789760786.052671:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729277543832 time 729287672646 state 0x2 ref 0 00000100:00100000:2.0:1789760786.056154:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760786.056945:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554819584 00000100:00100000:1.0:1789760786.056957:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554819584 00000100:00100000:1.0:1789760786.057013:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729277543832 time 729287672646 state 0x0 ref 1 00000100:00100000:1.0:1789760786.057084:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729287933062 time 729287672646 state 0x1 ref 1 00000100:00100000:1.0:1789760786.057162:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729287933062 time 729302779362 state 0x0 ref 1 00000100:00100000:1.0:1789760786.057175:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ccaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554819584:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760786.057219:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65691648, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f548 00010000:00010000:1.0:1789760786.057228:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c60c0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.057244:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760786.057248:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.057254:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c60c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65691648->65695743] (req 65691648->65695743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.057267:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c60c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65691648->65695743] (req 65691648->65695743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.064442:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983347 is committed 00002000:00010000:1.0:1789760786.064549:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65691648 -> 65695744 00002000:00010000:1.0:1789760786.064555:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128328 -> 128336 00010000:00010000:1.0:1789760786.064568:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c60c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65691648->65695743] (req 65691648->65695743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.064581:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c60c0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65691648->65695743] (req 65691648->65695743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.064590:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c60c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65691648->65695743] (req 65691648->65695743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.064603:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655de00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760786.064608:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c60c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65691648->65695743] (req 65691648->65695743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.064625:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c60c0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65691648->65695743] (req 65691648->65695743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.064643:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c60c0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65691648->65695743] (req 65691648->65695743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.064658:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c60c0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65691648->65695743] (req 65691648->65695743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.064757:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ccaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554819584:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7580us (8606us total) trans 4294983347 rc 0/0 00000100:00100000:1.0:1789760786.064768:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16049 00000100:00100000:1.0:1789760786.064778:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729287933062 time 729302779362 state 0x2 ref 0 00000100:00100000:2.0:1789760786.069495:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760786.069557:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554819712 00000100:00100000:3.0:1789760786.069567:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554819712 00000100:00100000:3.0:1789760786.069622:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729287933062 time 729302779362 state 0x0 ref 1 00000100:00100000:3.0:1789760786.069637:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729303039778 time 729302779362 state 0x1 ref 1 00000100:00100000:3.0:1789760786.069702:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729303039778 time 729315319334 state 0x0 ref 1 00000100:00100000:3.0:1789760786.069720:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03cc8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554819712:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760786.069765:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65695744, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f550 00010000:00010000:3.0:1789760786.069774:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c60c7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.069788:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760786.069793:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.069800:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c60c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65695744->65699839] (req 65695744->65699839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.069815:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c60c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65695744->65699839] (req 65695744->65699839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.074840:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983348 is committed 00002000:00010000:3.0:1789760786.075243:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65695744 -> 65699840 00002000:00010000:3.0:1789760786.075249:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128336 -> 128344 00010000:00010000:3.0:1789760786.075262:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c60c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65695744->65699839] (req 65695744->65699839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.075275:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c60c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65695744->65699839] (req 65695744->65699839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.075283:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c60c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65695744->65699839] (req 65695744->65699839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.075290:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760786.075298:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c60c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65695744->65699839] (req 65695744->65699839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.075307:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c60c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65695744->65699839] (req 65695744->65699839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.075327:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c60c7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65695744->65699839] (req 65695744->65699839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.075335:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c60c7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65695744->65699839] (req 65695744->65699839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.075596:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03cc8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554819712:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5874us (6217us total) trans 4294983348 rc 0/0 00000100:00100000:3.0:1789760786.075607:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16050 00000100:00100000:3.0:1789760786.075617:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729303039778 time 729315319334 state 0x2 ref 0 00000100:00100000:3.0:1789760786.079978:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760786.080075:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554819840 00000100:00100000:2.0:1789760786.080085:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554819840 00000100:00100000:2.0:1789760786.080139:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729303039778 time 729315319334 state 0x0 ref 1 00000100:00100000:2.0:1789760786.080154:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729315579750 time 729315319334 state 0x1 ref 1 00000100:00100000:2.0:1789760786.080183:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729315579750 time 729325802167 state 0x0 ref 1 00000100:00100000:2.0:1789760786.080199:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc410d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554819840:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760786.080244:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65699840, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f558 00010000:00010000:2.0:1789760786.080252:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c60ce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.080268:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760786.080272:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.080278:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c60ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65699840->65703935] (req 65699840->65703935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.080292:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c60ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65699840->65703935] (req 65699840->65703935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.088692:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983349 is committed 00002000:00010000:2.0:1789760786.088916:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65699840 -> 65703936 00002000:00010000:2.0:1789760786.088921:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128344 -> 128352 00010000:00010000:2.0:1789760786.088934:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c60ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65699840->65703935] (req 65699840->65703935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.088947:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c60ce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65699840->65703935] (req 65699840->65703935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.088955:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c60ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65699840->65703935] (req 65699840->65703935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.088961:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760786.088965:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c60ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65699840->65703935] (req 65699840->65703935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.088974:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c60ce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65699840->65703935] (req 65699840->65703935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.088994:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c60ce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65699840->65703935] (req 65699840->65703935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.089001:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c60ce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65699840->65703935] (req 65699840->65703935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.089190:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc410d1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554819840:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8988us (9216us total) trans 4294983349 rc 0/0 00000100:00100000:2.0:1789760786.089201:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16051 00000100:00100000:2.0:1789760786.089211:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729315579750 time 729325802167 state 0x2 ref 0 00000100:00100000:2.0:1789760786.094034:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760786.094202:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554819968 00000100:00100000:0.0:1789760786.094212:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554819968 00000100:00100000:0.0:1789760786.094268:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729315579750 time 729325802167 state 0x0 ref 1 00000100:00100000:0.0:1789760786.094284:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729326062583 time 729325802167 state 0x1 ref 1 00000100:00100000:0.0:1789760786.094357:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729326062583 time 729339975374 state 0x0 ref 1 00000100:00100000:0.0:1789760786.094370:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ccad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554819968:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760786.094596:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65703936, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f560 00010000:00010000:0.0:1789760786.094606:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c60d5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.094622:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760786.094626:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.094632:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c60d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65703936->65708031] (req 65703936->65708031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.094647:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c60d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65703936->65708031] (req 65703936->65708031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.100252:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983350 is committed 00002000:00010000:0.0:1789760786.100641:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65703936 -> 65708032 00002000:00010000:0.0:1789760786.100646:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128352 -> 128360 00010000:00010000:0.0:1789760786.100673:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c60d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65703936->65708031] (req 65703936->65708031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.100690:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c60d5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65703936->65708031] (req 65703936->65708031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.100698:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c60d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65703936->65708031] (req 65703936->65708031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.100704:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760786.100708:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c60d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65703936->65708031] (req 65703936->65708031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.100718:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c60d5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65703936->65708031] (req 65703936->65708031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.100739:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c60d5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65703936->65708031] (req 65703936->65708031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.100747:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c60d5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65703936->65708031] (req 65703936->65708031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.100842:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ccad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554819968:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6468us (6813us total) trans 4294983350 rc 0/0 00000100:00100000:0.0:1789760786.100853:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16052 00000100:00100000:0.0:1789760786.100862:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729326062583 time 729339975374 state 0x2 ref 0 00000100:00100000:0.0:1789760786.118485:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760786.118634:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554820096 00000100:00100000:1.0:1789760786.118644:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554820096 00000100:00100000:1.0:1789760786.118706:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729326062583 time 729339975374 state 0x0 ref 1 00000100:00100000:1.0:1789760786.118720:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729340235790 time 729339975374 state 0x1 ref 1 00000100:00100000:1.0:1789760786.118755:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729340235790 time 729364374095 state 0x0 ref 1 00000100:00100000:1.0:1789760786.118768:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554820096:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760786.118811:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65708032, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f568 00010000:00010000:1.0:1789760786.118820:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c60dc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.118835:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760786.118840:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.118846:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c60dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65708032->65712127] (req 65708032->65712127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.118861:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c60dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65708032->65712127] (req 65708032->65712127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.125441:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983351 is committed 00002000:00010000:1.0:1789760786.125661:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65708032 -> 65712128 00002000:00010000:1.0:1789760786.125666:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128360 -> 128368 00010000:00010000:1.0:1789760786.125679:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c60dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65708032->65712127] (req 65708032->65712127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.125693:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c60dc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65708032->65712127] (req 65708032->65712127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.125702:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c60dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65708032->65712127] (req 65708032->65712127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.125719:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655de00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760786.125724:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c60dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65708032->65712127] (req 65708032->65712127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.125733:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c60dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65708032->65712127] (req 65708032->65712127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.125752:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c60dc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65708032->65712127] (req 65708032->65712127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.125759:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c60dc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65708032->65712127] (req 65708032->65712127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.126124:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554820096:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7352us (7643us total) trans 4294983351 rc 0/0 00000100:00100000:1.0:1789760786.126133:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16053 00000100:00100000:1.0:1789760786.126143:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729340235790 time 729364374095 state 0x2 ref 0 00000100:00100000:3.0:1789760786.132184:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760786.132310:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554820224 00000100:00100000:0.0:1789760786.132325:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554820224 00000100:00100000:0.0:1789760786.132412:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729340235790 time 729364374095 state 0x0 ref 1 00000100:00100000:0.0:1789760786.132428:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729364634511 time 729364374095 state 0x1 ref 1 00000100:00100000:0.0:1789760786.132456:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729364634511 time 729378074618 state 0x0 ref 1 00000100:00100000:0.0:1789760786.132468:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc410d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554820224:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760786.132513:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65712128, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f570 00010000:00010000:0.0:1789760786.132522:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c60e3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.132537:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760786.132541:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.132547:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c60e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65712128->65716223] (req 65712128->65716223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.132561:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c60e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65712128->65716223] (req 65712128->65716223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.145371:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983352 is committed 00002000:00010000:0.0:1789760786.145680:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65712128 -> 65716224 00002000:00010000:0.0:1789760786.145687:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128368 -> 128376 00010000:00010000:0.0:1789760786.145700:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c60e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65712128->65716223] (req 65712128->65716223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.145712:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c60e3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65712128->65716223] (req 65712128->65716223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.145722:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c60e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65712128->65716223] (req 65712128->65716223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.145728:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760786.145733:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c60e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65712128->65716223] (req 65712128->65716223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.145742:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c60e3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65712128->65716223] (req 65712128->65716223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.145759:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c60e3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65712128->65716223] (req 65712128->65716223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.145768:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c60e3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65712128->65716223] (req 65712128->65716223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.145863:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc410d1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554820224:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13392us (13684us total) trans 4294983352 rc 0/0 00000100:00100000:0.0:1789760786.145874:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16054 00000100:00100000:0.0:1789760786.145884:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729364634511 time 729378074618 state 0x2 ref 0 00000100:00100000:2.0:1789760786.150633:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760786.150730:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554820352 00000100:00100000:3.0:1789760786.150740:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554820352 00000100:00100000:3.0:1789760786.150792:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729364634511 time 729378074618 state 0x0 ref 1 00000100:00100000:3.0:1789760786.150805:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729378335034 time 729378074618 state 0x1 ref 1 00000100:00100000:3.0:1789760786.150836:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729378335034 time 729396455363 state 0x0 ref 1 00000100:00100000:3.0:1789760786.150848:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03cca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554820352:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760786.150893:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65716224, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f578 00010000:00010000:3.0:1789760786.150902:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c60ea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.150921:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760786.150925:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.150932:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c60ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65716224->65720319] (req 65716224->65720319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.150944:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c60ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65716224->65720319] (req 65716224->65720319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.156313:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983353 is committed 00002000:00010000:3.0:1789760786.156444:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65716224 -> 65720320 00002000:00010000:3.0:1789760786.156450:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128376 -> 128384 00010000:00010000:3.0:1789760786.156463:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c60ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65716224->65720319] (req 65716224->65720319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.156476:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c60ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65716224->65720319] (req 65716224->65720319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.156486:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c60ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65716224->65720319] (req 65716224->65720319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.156493:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760786.156509:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c60ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65716224->65720319] (req 65716224->65720319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.156519:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c60ea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65716224->65720319] (req 65716224->65720319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.156544:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c60ea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65716224->65720319] (req 65716224->65720319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.156551:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c60ea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65716224->65720319] (req 65716224->65720319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.156649:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03cca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554820352:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5799us (6021us total) trans 4294983353 rc 0/0 00000100:00100000:3.0:1789760786.156668:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16055 00000100:00100000:3.0:1789760786.156679:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729378335034 time 729396455363 state 0x2 ref 0 00000100:00100000:2.0:1789760786.162946:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760786.163475:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554820480 00000100:00100000:1.0:1789760786.163487:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554820480 00000100:00100000:1.0:1789760786.163541:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729378335034 time 729396455363 state 0x0 ref 1 00000100:00100000:1.0:1789760786.163557:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729396715779 time 729396455363 state 0x1 ref 1 00000100:00100000:1.0:1789760786.163585:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729396715779 time 729409203842 state 0x0 ref 1 00000100:00100000:1.0:1789760786.163600:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03cc9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554820480:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760786.163644:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65720320, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f580 00010000:00010000:1.0:1789760786.163665:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c60f1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.163681:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760786.163686:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.163693:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c60f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65720320->65724415] (req 65720320->65724415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.163708:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c60f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65720320->65724415] (req 65720320->65724415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.169620:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983354 is committed 00002000:00010000:1.0:1789760786.177892:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65720320 -> 65724416 00002000:00010000:1.0:1789760786.177903:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128384 -> 128392 00010000:00010000:1.0:1789760786.177916:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c60f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65720320->65724415] (req 65720320->65724415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.177929:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c60f1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65720320->65724415] (req 65720320->65724415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.177939:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c60f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65720320->65724415] (req 65720320->65724415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.177947:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760786.177952:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c60f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65720320->65724415] (req 65720320->65724415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.177962:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c60f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65720320->65724415] (req 65720320->65724415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.177980:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c60f1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65720320->65724415] (req 65720320->65724415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.177988:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c60f1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65720320->65724415] (req 65720320->65724415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.178259:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03cc9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554820480:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14654us (15315us total) trans 4294983354 rc 0/0 00000100:00100000:1.0:1789760786.178271:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16056 00000100:00100000:1.0:1789760786.178280:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729396715779 time 729409203842 state 0x2 ref 0 00000100:00100000:1.0:1789760786.184578:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760786.184930:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554820608 00000100:00100000:2.0:1789760786.184939:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554820608 00000100:00100000:2.0:1789760786.184994:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729396715779 time 729409203842 state 0x0 ref 1 00000100:00100000:2.0:1789760786.185009:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729409464258 time 729409203842 state 0x1 ref 1 00000100:00100000:2.0:1789760786.185043:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729409464258 time 729430661570 state 0x0 ref 1 00000100:00100000:2.0:1789760786.185054:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554820608:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760786.185105:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65724416, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f588 00010000:00010000:2.0:1789760786.185115:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c60f8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.185134:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760786.185138:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.185145:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c60f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65724416->65728511] (req 65724416->65728511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.185157:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c60f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65724416->65728511] (req 65724416->65728511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.192485:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983355 is committed 00002000:00010000:2.0:1789760786.192621:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65724416 -> 65728512 00002000:00010000:2.0:1789760786.192626:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128392 -> 128400 00010000:00010000:2.0:1789760786.192639:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c60f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65724416->65728511] (req 65724416->65728511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.192663:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c60f8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65724416->65728511] (req 65724416->65728511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.192672:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c60f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65724416->65728511] (req 65724416->65728511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.192679:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760786.192684:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c60f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65724416->65728511] (req 65724416->65728511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.192693:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c60f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65724416->65728511] (req 65724416->65728511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.192719:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c60f8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65724416->65728511] (req 65724416->65728511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.192727:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c60f8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65724416->65728511] (req 65724416->65728511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.192832:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554820608:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7774us (8259us total) trans 4294983355 rc 0/0 00000100:00100000:2.0:1789760786.192843:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16057 00000100:00100000:2.0:1789760786.192852:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729409464258 time 729430661570 state 0x2 ref 0 00000100:00100000:2.0:1789760786.197166:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760786.200020:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554820736 00000100:00100000:0.0:1789760786.200036:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554820736 00000100:00100000:0.0:1789760786.200194:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729409464258 time 729430661570 state 0x0 ref 1 00000100:00100000:0.0:1789760786.200225:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729430921986 time 729430661570 state 0x1 ref 1 00000100:00100000:1.0:1789760786.200481:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729430921986 time 729446096846 state 0x0 ref 1 00000100:00100000:1.0:1789760786.200500:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ccb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554820736:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760786.200561:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65728512, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f590 00010000:00010000:1.0:1789760786.200570:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c60ff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.200589:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760786.200593:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.200599:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c60ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65728512->65732607] (req 65728512->65732607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.200612:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c60ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65728512->65732607] (req 65728512->65732607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.210566:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983356 is committed 00002000:00010000:1.0:1789760786.210752:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65728512 -> 65732608 00002000:00010000:1.0:1789760786.210757:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128400 -> 128408 00010000:00010000:1.0:1789760786.210771:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c60ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65728512->65732607] (req 65728512->65732607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.210785:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c60ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65728512->65732607] (req 65728512->65732607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.210795:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c60ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65728512->65732607] (req 65728512->65732607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.210803:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760786.210808:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c60ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65728512->65732607] (req 65728512->65732607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.210818:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c60ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65728512->65732607] (req 65728512->65732607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.210842:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c60ff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65728512->65732607] (req 65728512->65732607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.210850:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c60ff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65728512->65732607] (req 65728512->65732607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.214863:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ccb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554820736:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14355us (17694us total) trans 4294983356 rc 0/0 00000100:00100000:1.0:1789760786.214879:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16058 00000100:00100000:1.0:1789760786.214889:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729430921986 time 729446096846 state 0x2 ref 0 00000100:00100000:0.0:1789760786.221282:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760786.221365:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554820864 00000100:00100000:3.0:1789760786.221416:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554820864 00000100:00100000:3.0:1789760786.221472:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729430921986 time 729446096846 state 0x0 ref 1 00000100:00100000:3.0:1789760786.221488:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729446357262 time 729446096846 state 0x1 ref 1 00000100:00100000:3.0:1789760786.221521:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729446357262 time 729467139610 state 0x0 ref 1 00000100:00100000:3.0:1789760786.221533:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554820864:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760786.221579:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65732608, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f598 00010000:00010000:3.0:1789760786.221589:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6106 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.221604:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760786.221609:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.221614:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6106 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65732608->65736703] (req 65732608->65736703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.221628:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6106 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65732608->65736703] (req 65732608->65736703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.236945:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983357 is committed 00002000:00010000:2.0:1789760786.237185:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65732608 -> 65736704 00002000:00010000:2.0:1789760786.237190:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128408 -> 128416 00010000:00010000:2.0:1789760786.237203:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6106 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65732608->65736703] (req 65732608->65736703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.237216:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6106 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65732608->65736703] (req 65732608->65736703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.237225:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6106 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65732608->65736703] (req 65732608->65736703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.237233:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760786.237239:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6106 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65732608->65736703] (req 65732608->65736703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.237248:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6106 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65732608->65736703] (req 65732608->65736703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.237266:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6106 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65732608->65736703] (req 65732608->65736703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.237274:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6106 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65732608->65736703] (req 65732608->65736703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.237372:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554820864:12345-192.168.203.22@tcp:4:dd.0 Request processed in 15836us (16094us total) trans 4294983357 rc 0/0 00000100:00100000:2.0:1789760786.237430:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16059 00000100:00100000:2.0:1789760786.237439:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729446357262 time 729467139610 state 0x2 ref 0 00000100:00100000:3.0:1789760786.241500:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760786.241854:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554820992 00000100:00100000:0.0:1789760786.241864:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554820992 00000100:00100000:0.0:1789760786.241919:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729446357262 time 729467139610 state 0x0 ref 1 00000100:00100000:0.0:1789760786.241934:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729467400026 time 729467139610 state 0x1 ref 1 00000100:00100000:0.0:1789760786.241969:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729467400026 time 729487587674 state 0x0 ref 1 00000100:00100000:0.0:1789760786.241981:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc410d1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554820992:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760786.242037:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65736704, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f5a0 00010000:00010000:0.0:1789760786.242046:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c610d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.242063:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760786.242067:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.242074:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c610d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65736704->65740799] (req 65736704->65740799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.242088:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c610d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65736704->65740799] (req 65736704->65740799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.256528:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983358 is committed 00002000:00010000:0.0:1789760786.256670:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65736704 -> 65740800 00002000:00010000:0.0:1789760786.256675:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128416 -> 128424 00010000:00010000:0.0:1789760786.256688:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c610d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65736704->65740799] (req 65736704->65740799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.256715:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c610d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65736704->65740799] (req 65736704->65740799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.256724:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c610d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65736704->65740799] (req 65736704->65740799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.256731:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760786.256737:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c610d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65736704->65740799] (req 65736704->65740799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.256745:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c610d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65736704->65740799] (req 65736704->65740799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.256765:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c610d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65736704->65740799] (req 65736704->65740799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.256772:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c610d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65736704->65740799] (req 65736704->65740799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.256910:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc410d1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554820992:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14926us (15414us total) trans 4294983358 rc 0/0 00000100:00100000:0.0:1789760786.256920:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16060 00000100:00100000:0.0:1789760786.256930:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729467400026 time 729487587674 state 0x2 ref 0 00000100:00100000:3.0:1789760786.267763:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760786.267909:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554821120 00000100:00100000:2.0:1789760786.267919:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554821120 00000100:00100000:2.0:1789760786.267974:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729467400026 time 729487587674 state 0x0 ref 1 00000100:00100000:2.0:1789760786.267990:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729487848090 time 729487587674 state 0x1 ref 1 00000100:00100000:2.0:1789760786.268020:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729487848090 time 729513639259 state 0x0 ref 1 00000100:00100000:2.0:1789760786.268106:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc410d3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554821120:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760786.268217:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65740800, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f5a8 00010000:00010000:2.0:1789760786.268228:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6114 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.268245:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760786.268249:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.268256:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6114 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65740800->65744895] (req 65740800->65744895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.268270:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6114 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65740800->65744895] (req 65740800->65744895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.278070:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983359 is committed 00002000:00010000:1.0:1789760786.278359:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65740800 -> 65744896 00002000:00010000:1.0:1789760786.278364:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128424 -> 128432 00010000:00010000:1.0:1789760786.278517:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6114 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65740800->65744895] (req 65740800->65744895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.278530:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6114 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65740800->65744895] (req 65740800->65744895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.278539:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6114 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65740800->65744895] (req 65740800->65744895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.278547:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760786.278552:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6114 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65740800->65744895] (req 65740800->65744895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.278562:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6114 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65740800->65744895] (req 65740800->65744895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.278580:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6114 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65740800->65744895] (req 65740800->65744895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.278588:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6114 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65740800->65744895] (req 65740800->65744895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.278834:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc410d3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554821120:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10728us (11075us total) trans 4294983359 rc 0/0 00000100:00100000:1.0:1789760786.278845:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16061 00000100:00100000:1.0:1789760786.278854:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729487848090 time 729513639259 state 0x2 ref 0 00000100:00100000:2.0:1789760786.283086:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760786.283370:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554821248 00000100:00100000:0.0:1789760786.283416:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554821248 00000100:00100000:0.0:1789760786.283464:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729487848090 time 729513639259 state 0x0 ref 1 00000100:00100000:0.0:1789760786.283479:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729513899675 time 729513639259 state 0x1 ref 1 00000100:00100000:0.0:1789760786.283509:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729513899675 time 729529127336 state 0x0 ref 1 00000100:00100000:0.0:1789760786.283520:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ccbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554821248:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760786.283565:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65744896, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f5b0 00010000:00010000:0.0:1789760786.283574:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c611b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.283589:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760786.283594:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.283600:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c611b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65744896->65748991] (req 65744896->65748991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.283614:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c611b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65744896->65748991] (req 65744896->65748991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.294451:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983360 is committed 00002000:00010000:0.0:1789760786.294555:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65744896 -> 65748992 00002000:00010000:0.0:1789760786.294561:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128432 -> 128440 00010000:00010000:0.0:1789760786.294573:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c611b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65744896->65748991] (req 65744896->65748991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.294586:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c611b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65744896->65748991] (req 65744896->65748991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.294600:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c611b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65744896->65748991] (req 65744896->65748991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.294607:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760786.294611:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c611b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65744896->65748991] (req 65744896->65748991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.294619:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c611b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65744896->65748991] (req 65744896->65748991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.294637:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c611b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65744896->65748991] (req 65744896->65748991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.294645:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c611b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65744896->65748991] (req 65744896->65748991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.294752:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ccbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554821248:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11228us (11670us total) trans 4294983360 rc 0/0 00000100:00100000:0.0:1789760786.294764:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16062 00000100:00100000:0.0:1789760786.294773:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729513899675 time 729529127336 state 0x2 ref 0 00000100:00100000:3.0:1789760786.301205:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760786.301271:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554821376 00000100:00100000:2.0:1789760786.301282:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554821376 00000100:00100000:2.0:1789760786.301464:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729513899675 time 729529127336 state 0x0 ref 1 00000100:00100000:2.0:1789760786.301480:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729529387752 time 729529127336 state 0x1 ref 1 00000100:00100000:2.0:1789760786.301551:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729529387752 time 729547168084 state 0x0 ref 1 00000100:00100000:2.0:1789760786.301564:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc410d2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554821376:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760786.301612:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65748992, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f5b8 00010000:00010000:2.0:1789760786.301627:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6122 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.301644:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760786.301660:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.301667:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6122 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65748992->65753087] (req 65748992->65753087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.301692:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6122 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65748992->65753087] (req 65748992->65753087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.308465:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983361 is committed 00002000:00010000:2.0:1789760786.308636:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65748992 -> 65753088 00002000:00010000:2.0:1789760786.308641:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128440 -> 128448 00010000:00010000:2.0:1789760786.308666:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6122 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65748992->65753087] (req 65748992->65753087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.308678:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6122 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65748992->65753087] (req 65748992->65753087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.308686:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6122 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65748992->65753087] (req 65748992->65753087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.308701:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760786.308706:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6122 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65748992->65753087] (req 65748992->65753087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.308716:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6122 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65748992->65753087] (req 65748992->65753087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.308736:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6122 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65748992->65753087] (req 65748992->65753087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.308747:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6122 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65748992->65753087] (req 65748992->65753087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.308964:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc410d2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554821376:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7398us (7764us total) trans 4294983361 rc 0/0 00000100:00100000:2.0:1789760786.308974:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16063 00000100:00100000:2.0:1789760786.308983:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729529387752 time 729547168084 state 0x2 ref 0 00000100:00100000:3.0:1789760786.316477:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760786.316592:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554821504 00000100:00100000:0.0:1789760786.316602:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554821504 00000100:00100000:0.0:1789760786.316664:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729529387752 time 729547168084 state 0x0 ref 1 00000100:00100000:0.0:1789760786.316679:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729547428500 time 729547168084 state 0x1 ref 1 00000100:00100000:0.0:1789760786.316772:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729547428500 time 729562390254 state 0x0 ref 1 00000100:00100000:0.0:1789760786.316786:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc410d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554821504:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760786.316840:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65753088, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f5c0 00010000:00010000:0.0:1789760786.316850:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6129 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.316865:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760786.316870:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.316877:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6129 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65753088->65757183] (req 65753088->65757183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.316891:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6129 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65753088->65757183] (req 65753088->65757183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.329017:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983362 is committed 00002000:00010000:0.0:1789760786.329673:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65753088 -> 65757184 00002000:00010000:0.0:1789760786.329678:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128448 -> 128456 00010000:00010000:0.0:1789760786.332076:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6129 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65753088->65757183] (req 65753088->65757183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.332097:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6129 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65753088->65757183] (req 65753088->65757183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.332107:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6129 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65753088->65757183] (req 65753088->65757183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.332130:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760786.332134:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6129 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65753088->65757183] (req 65753088->65757183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.332144:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6129 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65753088->65757183] (req 65753088->65757183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.332164:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6129 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65753088->65757183] (req 65753088->65757183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.332173:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6129 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65753088->65757183] (req 65753088->65757183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.332277:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc410d3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554821504:12345-192.168.203.22@tcp:4:dd.0 Request processed in 15490us (15806us total) trans 4294983362 rc 0/0 00000100:00100000:0.0:1789760786.332289:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16064 00000100:00100000:0.0:1789760786.332299:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729547428500 time 729562390254 state 0x2 ref 0 00000100:00100000:0.0:1789760786.341367:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760786.341501:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554821632 00000100:00100000:1.0:1789760786.341511:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554821632 00000100:00100000:1.0:1789760786.341566:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729547428500 time 729562390254 state 0x0 ref 1 00000100:00100000:1.0:1789760786.341581:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729562650670 time 729562390254 state 0x1 ref 1 00000100:00100000:1.0:1789760786.341602:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729562650670 time 729587220988 state 0x0 ref 1 00000100:00100000:1.0:1789760786.341613:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554821632:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760786.341669:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65757184, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f5c8 00010000:00010000:1.0:1789760786.341677:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6130 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.341818:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760786.341824:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.341831:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6130 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65757184->65761279] (req 65757184->65761279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.341845:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6130 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65757184->65761279] (req 65757184->65761279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.349892:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983363 is committed 00002000:00010000:1.0:1789760786.350274:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65757184 -> 65761280 00002000:00010000:1.0:1789760786.350280:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128456 -> 128464 00010000:00010000:1.0:1789760786.350292:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6130 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65757184->65761279] (req 65757184->65761279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.350305:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6130 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65757184->65761279] (req 65757184->65761279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.350315:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6130 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65757184->65761279] (req 65757184->65761279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.350322:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760786.350327:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6130 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65757184->65761279] (req 65757184->65761279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.350338:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6130 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65757184->65761279] (req 65757184->65761279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.350363:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6130 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65757184->65761279] (req 65757184->65761279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.350371:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6130 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65757184->65761279] (req 65757184->65761279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.350546:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554821632:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8929us (9183us total) trans 4294983363 rc 0/0 00000100:00100000:1.0:1789760786.350557:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16065 00000100:00100000:1.0:1789760786.350567:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729562650670 time 729587220988 state 0x2 ref 0 00000100:00100000:3.0:1789760786.355019:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760786.355698:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554821760 00000100:00100000:2.0:1789760786.355710:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554821760 00000100:00100000:2.0:1789760786.355824:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729562650670 time 729587220988 state 0x0 ref 1 00000100:00100000:2.0:1789760786.355840:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729587481404 time 729587220988 state 0x1 ref 1 00000100:00100000:2.0:1789760786.355872:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729587481404 time 729601490193 state 0x0 ref 1 00000100:00100000:2.0:1789760786.355885:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc410d1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554821760:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760786.355934:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65761280, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f5d0 00010000:00010000:2.0:1789760786.355943:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6137 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.355958:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760786.355963:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.355970:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6137 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65761280->65765375] (req 65761280->65765375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.355983:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6137 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65761280->65765375] (req 65761280->65765375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.366188:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983364 is committed 00002000:00010000:2.0:1789760786.366435:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65761280 -> 65765376 00002000:00010000:2.0:1789760786.366441:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128464 -> 128472 00010000:00010000:2.0:1789760786.366455:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6137 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65761280->65765375] (req 65761280->65765375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.366467:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6137 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65761280->65765375] (req 65761280->65765375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.366476:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6137 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65761280->65765375] (req 65761280->65765375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.366483:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760786.366488:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6137 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65761280->65765375] (req 65761280->65765375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.366497:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6137 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65761280->65765375] (req 65761280->65765375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.366516:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6137 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65761280->65765375] (req 65761280->65765375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.366529:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6137 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65761280->65765375] (req 65761280->65765375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.366619:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc410d1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554821760:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10731us (11610us total) trans 4294983364 rc 0/0 00000100:00100000:2.0:1789760786.366629:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16066 00000100:00100000:2.0:1789760786.366639:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729587481404 time 729601490193 state 0x2 ref 0 00000100:00100000:3.0:1789760786.370944:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760786.371079:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554821888 00000100:00100000:2.0:1789760786.371088:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554821888 00000100:00100000:2.0:1789760786.371151:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729587481404 time 729601490193 state 0x0 ref 1 00000100:00100000:2.0:1789760786.371167:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729601750609 time 729601490193 state 0x1 ref 1 00000100:00100000:2.0:1789760786.371208:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729601750609 time 729616825882 state 0x0 ref 1 00000100:00100000:2.0:1789760786.371221:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc410d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554821888:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760786.371271:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65765376, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f5d8 00010000:00010000:2.0:1789760786.371282:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c613e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.371299:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760786.371303:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.371310:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c613e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65765376->65769471] (req 65765376->65769471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.371329:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c613e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65765376->65769471] (req 65765376->65769471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.381960:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983365 is committed 00002000:00010000:2.0:1789760786.382090:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65765376 -> 65769472 00002000:00010000:2.0:1789760786.382095:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128472 -> 128480 00010000:00010000:2.0:1789760786.382107:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c613e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65765376->65769471] (req 65765376->65769471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.382119:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c613e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65765376->65769471] (req 65765376->65769471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.382128:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c613e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65765376->65769471] (req 65765376->65769471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.382135:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760786.382140:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c613e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65765376->65769471] (req 65765376->65769471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.382148:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c613e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65765376->65769471] (req 65765376->65769471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.382177:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c613e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65765376->65769471] (req 65765376->65769471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.382186:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c613e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65765376->65769471] (req 65765376->65769471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.382281:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc410d2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554821888:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11058us (11342us total) trans 4294983365 rc 0/0 00000100:00100000:2.0:1789760786.382292:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16067 00000100:00100000:2.0:1789760786.382302:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729601750609 time 729616825882 state 0x2 ref 0 00000100:00100000:0.0:1789760786.387374:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760786.387604:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554822016 00000100:00100000:3.0:1789760786.387614:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554822016 00000100:00100000:3.0:1789760786.387678:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729601750609 time 729616825882 state 0x0 ref 1 00000100:00100000:3.0:1789760786.387693:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729617086298 time 729616825882 state 0x1 ref 1 00000100:00100000:3.0:1789760786.387725:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729617086298 time 729633343819 state 0x0 ref 1 00000100:00100000:3.0:1789760786.387737:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554822016:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760786.387796:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65769472, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f5e0 00010000:00010000:3.0:1789760786.387806:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6145 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.387823:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760786.387828:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.387835:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6145 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65769472->65773567] (req 65769472->65773567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.387848:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6145 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65769472->65773567] (req 65769472->65773567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.396081:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983366 is committed 00002000:00010000:3.0:1789760786.397950:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65769472 -> 65773568 00002000:00010000:3.0:1789760786.397960:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128480 -> 128488 00010000:00010000:3.0:1789760786.397974:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6145 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65769472->65773567] (req 65769472->65773567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.397987:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6145 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65769472->65773567] (req 65769472->65773567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.397996:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6145 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65769472->65773567] (req 65769472->65773567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.398004:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760786.398009:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6145 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65769472->65773567] (req 65769472->65773567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.398189:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6145 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65769472->65773567] (req 65769472->65773567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.398216:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6145 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65769472->65773567] (req 65769472->65773567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.398225:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6145 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65769472->65773567] (req 65769472->65773567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.398426:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554822016:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10641us (11012us total) trans 4294983366 rc 0/0 00000100:00100000:3.0:1789760786.398438:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16068 00000100:00100000:3.0:1789760786.398448:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729617086298 time 729633343819 state 0x2 ref 0 00000100:00100000:3.0:1789760786.403586:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760786.407300:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554822144 00000100:00100000:1.0:1789760786.407317:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554822144 00000100:00100000:1.0:1789760786.407521:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729617086298 time 729633343819 state 0x0 ref 1 00000100:00100000:1.0:1789760786.407538:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729633604235 time 729633343819 state 0x1 ref 1 00000100:00100000:1.0:1789760786.407639:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729633604235 time 729653255688 state 0x0 ref 1 00000100:00100000:1.0:1789760786.407667:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc410d0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554822144:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760786.407713:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65773568, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f5e8 00010000:00010000:1.0:1789760786.407723:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c614c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.407738:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760786.407743:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.407749:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c614c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65773568->65777663] (req 65773568->65777663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.407762:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c614c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65773568->65777663] (req 65773568->65777663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.419217:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983367 is committed 00002000:00010000:1.0:1789760786.419343:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65773568 -> 65777664 00002000:00010000:1.0:1789760786.419348:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128488 -> 128496 00010000:00010000:1.0:1789760786.419362:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c614c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65773568->65777663] (req 65773568->65777663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.419375:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c614c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65773568->65777663] (req 65773568->65777663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.419425:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c614c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65773568->65777663] (req 65773568->65777663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.419432:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760786.419438:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c614c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65773568->65777663] (req 65773568->65777663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.419880:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c614c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65773568->65777663] (req 65773568->65777663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.419906:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c614c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65773568->65777663] (req 65773568->65777663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.419920:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c614c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65773568->65777663] (req 65773568->65777663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.420017:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc410d0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554822144:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12362us (16435us total) trans 4294983367 rc 0/0 00000100:00100000:1.0:1789760786.420027:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16069 00000100:00100000:1.0:1789760786.420036:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729633604235 time 729653255688 state 0x2 ref 0 00000100:00100000:2.0:1789760786.425919:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760786.426036:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554822272 00000100:00100000:0.0:1789760786.426047:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554822272 00000100:00100000:0.0:1789760786.426101:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729633604235 time 729653255688 state 0x0 ref 1 00000100:00100000:0.0:1789760786.426245:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729653516104 time 729653255688 state 0x1 ref 1 00000100:00100000:0.0:1789760786.426278:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729653516104 time 729671896523 state 0x0 ref 1 00000100:00100000:0.0:1789760786.426290:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03cc9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554822272:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760786.426348:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65777664, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f5f0 00010000:00010000:0.0:1789760786.426357:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6153 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.426373:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760786.426704:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.426712:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6153 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65777664->65781759] (req 65777664->65781759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.426727:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6153 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65777664->65781759] (req 65777664->65781759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.433232:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983368 is committed 00002000:00010000:2.0:1789760786.433509:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65777664 -> 65781760 00002000:00010000:2.0:1789760786.433515:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128496 -> 128504 00010000:00010000:2.0:1789760786.433527:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6153 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65777664->65781759] (req 65777664->65781759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.433540:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6153 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65777664->65781759] (req 65777664->65781759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.433550:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6153 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65777664->65781759] (req 65777664->65781759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.433558:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760786.433563:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6153 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65777664->65781759] (req 65777664->65781759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.433572:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6153 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65777664->65781759] (req 65777664->65781759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.433596:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6153 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65777664->65781759] (req 65777664->65781759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.433604:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6153 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65777664->65781759] (req 65777664->65781759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.433708:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03cc9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554822272:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7415us (7794us total) trans 4294983368 rc 0/0 00000100:00100000:2.0:1789760786.433719:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16070 00000100:00100000:2.0:1789760786.433729:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729653516104 time 729671896523 state 0x2 ref 0 00000100:00100000:2.0:1789760786.442549:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760786.442753:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554822400 00000100:00100000:3.0:1789760786.442763:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554822400 00000100:00100000:3.0:1789760786.442824:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729653516104 time 729671896523 state 0x0 ref 1 00000100:00100000:3.0:1789760786.442841:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729672156939 time 729671896523 state 0x1 ref 1 00000100:00100000:3.0:1789760786.442876:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729672156939 time 729688494260 state 0x0 ref 1 00000100:00100000:3.0:1789760786.442889:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03cca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554822400:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760786.442936:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65781760, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f5f8 00010000:00010000:3.0:1789760786.442945:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c615a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.442966:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760786.442971:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.442978:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c615a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65781760->65785855] (req 65781760->65785855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.442992:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c615a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65781760->65785855] (req 65781760->65785855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.457259:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983369 is committed 00002000:00010000:2.0:1789760786.457487:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65781760 -> 65785856 00002000:00010000:2.0:1789760786.457493:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128504 -> 128512 00010000:00010000:2.0:1789760786.457506:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c615a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65781760->65785855] (req 65781760->65785855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.457520:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c615a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65781760->65785855] (req 65781760->65785855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.457529:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c615a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65781760->65785855] (req 65781760->65785855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.457537:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760786.457541:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c615a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65781760->65785855] (req 65781760->65785855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.457550:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c615a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65781760->65785855] (req 65781760->65785855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.457569:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c615a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65781760->65785855] (req 65781760->65785855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.457576:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c615a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65781760->65785855] (req 65781760->65785855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.457680:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03cca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554822400:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14789us (15136us total) trans 4294983369 rc 0/0 00000100:00100000:2.0:1789760786.457691:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16071 00000100:00100000:2.0:1789760786.457700:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729672156939 time 729688494260 state 0x2 ref 0 00000100:00100000:2.0:1789760786.472955:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760786.473016:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554822528 00000100:00100000:3.0:1789760786.473026:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554822528 00000100:00100000:3.0:1789760786.473088:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729672156939 time 729688494260 state 0x0 ref 1 00000100:00100000:3.0:1789760786.473103:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729688754676 time 729688494260 state 0x1 ref 1 00000100:00100000:3.0:1789760786.473220:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729688754676 time 729718837255 state 0x0 ref 1 00000100:00100000:3.0:1789760786.473233:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a384e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554822528:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760786.473278:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65785856, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f600 00010000:00010000:3.0:1789760786.473288:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6161 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.473314:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760786.473319:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.473326:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6161 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65785856->65789951] (req 65785856->65789951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.473346:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6161 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65785856->65789951] (req 65785856->65789951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.481180:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983370 is committed 00002000:00010000:2.0:1789760786.481298:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65785856 -> 65789952 00002000:00010000:2.0:1789760786.481303:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128512 -> 128520 00010000:00010000:2.0:1789760786.481317:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6161 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65785856->65789951] (req 65785856->65789951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.481329:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6161 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65785856->65789951] (req 65785856->65789951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.481338:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6161 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65785856->65789951] (req 65785856->65789951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.481351:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760786.481356:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6161 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65785856->65789951] (req 65785856->65789951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.481366:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6161 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65785856->65789951] (req 65785856->65789951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.481415:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6161 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65785856->65789951] (req 65785856->65789951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.481424:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6161 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65785856->65789951] (req 65785856->65789951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.481517:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a384e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554822528:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8281us (8565us total) trans 4294983370 rc 0/0 00000100:00100000:2.0:1789760786.481528:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16072 00000100:00100000:2.0:1789760786.481536:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729688754676 time 729718837255 state 0x2 ref 0 00000100:00100000:3.0:1789760786.486795:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760786.490154:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554822656 00000100:00100000:1.0:1789760786.490169:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554822656 00000100:00100000:1.0:1789760786.490241:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729688754676 time 729718837255 state 0x0 ref 1 00000100:00100000:1.0:1789760786.490257:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729719097671 time 729718837255 state 0x1 ref 1 00000100:00100000:1.0:1789760786.490327:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729719097671 time 729735944718 state 0x0 ref 1 00000100:00100000:1.0:1789760786.490342:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc410d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554822656:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760786.490415:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65789952, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f608 00010000:00010000:1.0:1789760786.490424:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6168 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.490441:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760786.490445:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.490451:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6168 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65789952->65794047] (req 65789952->65794047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.490465:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6168 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65789952->65794047] (req 65789952->65794047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.496792:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983371 is committed 00002000:00010000:2.0:1789760786.497035:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65789952 -> 65794048 00002000:00010000:2.0:1789760786.497040:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128520 -> 128528 00010000:00010000:2.0:1789760786.497053:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6168 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65789952->65794047] (req 65789952->65794047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.497066:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6168 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65789952->65794047] (req 65789952->65794047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.497081:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6168 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65789952->65794047] (req 65789952->65794047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.497088:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760786.497093:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6168 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65789952->65794047] (req 65789952->65794047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.497101:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6168 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65789952->65794047] (req 65789952->65794047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.497120:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6168 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65789952->65794047] (req 65789952->65794047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.497127:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6168 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65789952->65794047] (req 65789952->65794047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.497253:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc410d3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554822656:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6910us (10463us total) trans 4294983371 rc 0/0 00000100:00100000:2.0:1789760786.497264:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16073 00000100:00100000:2.0:1789760786.497274:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729719097671 time 729735944718 state 0x2 ref 0 00000100:00100000:1.0:1789760786.501286:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760786.501375:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554822784 00000100:00100000:2.0:1789760786.501425:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554822784 00000100:00100000:2.0:1789760786.501478:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729719097671 time 729735944718 state 0x0 ref 1 00000100:00100000:2.0:1789760786.501493:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729736205134 time 729735944718 state 0x1 ref 1 00000100:00100000:2.0:1789760786.501526:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729736205134 time 729747143744 state 0x0 ref 1 00000100:00100000:2.0:1789760786.501539:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554822784:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760786.501586:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65794048, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f610 00010000:00010000:2.0:1789760786.501596:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c616f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.501612:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760786.501627:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.501633:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c616f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65794048->65798143] (req 65794048->65798143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.501660:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c616f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65794048->65798143] (req 65794048->65798143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.507517:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983372 is committed 00002000:00010000:2.0:1789760786.507864:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65794048 -> 65798144 00002000:00010000:2.0:1789760786.507869:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128528 -> 128536 00010000:00010000:2.0:1789760786.507883:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c616f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65794048->65798143] (req 65794048->65798143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.507896:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c616f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65794048->65798143] (req 65794048->65798143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.507904:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c616f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65794048->65798143] (req 65794048->65798143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.507917:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760786.507921:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c616f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65794048->65798143] (req 65794048->65798143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.507930:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c616f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65794048->65798143] (req 65794048->65798143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.507950:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c616f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65794048->65798143] (req 65794048->65798143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.507958:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c616f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65794048->65798143] (req 65794048->65798143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.508098:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554822784:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6557us (6818us total) trans 4294983372 rc 0/0 00000100:00100000:2.0:1789760786.508109:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16074 00000100:00100000:2.0:1789760786.508119:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729736205134 time 729747143744 state 0x2 ref 0 00000100:00100000:0.0:1789760786.517547:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760786.517661:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554822912 00000100:00100000:2.0:1789760786.517673:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554822912 00000100:00100000:2.0:1789760786.517739:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729736205134 time 729747143744 state 0x0 ref 1 00000100:00100000:2.0:1789760786.517754:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729747404160 time 729747143744 state 0x1 ref 1 00000100:00100000:2.0:1789760786.517789:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729747404160 time 729763406942 state 0x0 ref 1 00000100:00100000:2.0:1789760786.517805:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554822912:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760786.517862:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65798144, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f618 00010000:00010000:2.0:1789760786.517871:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6176 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.517889:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760786.517894:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.517900:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6176 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65798144->65802239] (req 65798144->65802239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.517916:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6176 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65798144->65802239] (req 65798144->65802239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.526292:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983373 is committed 00002000:00010000:2.0:1789760786.526503:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65798144 -> 65802240 00002000:00010000:2.0:1789760786.526508:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128536 -> 128544 00010000:00010000:2.0:1789760786.526521:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6176 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65798144->65802239] (req 65798144->65802239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.526534:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6176 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65798144->65802239] (req 65798144->65802239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.526544:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6176 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65798144->65802239] (req 65798144->65802239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.526551:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760786.526558:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6176 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65798144->65802239] (req 65798144->65802239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.526567:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6176 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65798144->65802239] (req 65798144->65802239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.526586:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6176 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65798144->65802239] (req 65798144->65802239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.526594:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6176 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65798144->65802239] (req 65798144->65802239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.526940:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554822912:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9120us (9388us total) trans 4294983373 rc 0/0 00000100:00100000:2.0:1789760786.526951:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16075 00000100:00100000:2.0:1789760786.526963:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729747404160 time 729763406942 state 0x2 ref 0 00000100:00100000:2.0:1789760786.530986:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760786.531103:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554823040 00000100:00100000:3.0:1789760786.531112:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554823040 00000100:00100000:3.0:1789760786.533418:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729747404160 time 729763406942 state 0x0 ref 1 00000100:00100000:3.0:1789760786.533454:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729763667358 time 729763406942 state 0x1 ref 1 00000100:00100000:3.0:1789760786.533684:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729763667358 time 729779300835 state 0x0 ref 1 00000100:00100000:3.0:1789760786.533707:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a386d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554823040:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760786.533818:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65802240, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f620 00010000:00010000:3.0:1789760786.533826:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c617d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.533936:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760786.533941:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.533947:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c617d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65802240->65806335] (req 65802240->65806335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.533961:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c617d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65802240->65806335] (req 65802240->65806335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.539295:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983374 is committed 00002000:00010000:3.0:1789760786.541036:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65802240 -> 65806336 00002000:00010000:3.0:1789760786.541044:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128544 -> 128552 00010000:00010000:3.0:1789760786.541057:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c617d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65802240->65806335] (req 65802240->65806335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.541070:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c617d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65802240->65806335] (req 65802240->65806335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.541079:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c617d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65802240->65806335] (req 65802240->65806335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.541087:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760786.541092:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c617d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65802240->65806335] (req 65802240->65806335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.541101:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c617d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65802240->65806335] (req 65802240->65806335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.541120:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c617d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65802240->65806335] (req 65802240->65806335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.541129:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c617d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65802240->65806335] (req 65802240->65806335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.541220:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a386d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554823040:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7519us (10238us total) trans 4294983374 rc 0/0 00000100:00100000:3.0:1789760786.541232:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16076 00000100:00100000:3.0:1789760786.541241:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729763667358 time 729779300835 state 0x2 ref 0 00000100:00100000:1.0:1789760786.545118:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760786.548097:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554823168 00000100:00100000:2.0:1789760786.548109:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554823168 00000100:00100000:2.0:1789760786.548158:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729763667358 time 729779300835 state 0x0 ref 1 00000100:00100000:2.0:1789760786.548178:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729779561251 time 729779300835 state 0x1 ref 1 00000100:00100000:2.0:1789760786.548238:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729779561251 time 729793856843 state 0x0 ref 1 00000100:00100000:2.0:1789760786.548248:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554823168:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760786.548291:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65806336, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f628 00010000:00010000:2.0:1789760786.548298:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6184 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.548311:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760786.548315:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.548320:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6184 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65806336->65810431] (req 65806336->65810431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.548333:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6184 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65806336->65810431] (req 65806336->65810431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.554230:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983375 is committed 00002000:00010000:2.0:1789760786.557691:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65806336 -> 65810432 00002000:00010000:2.0:1789760786.557701:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128552 -> 128560 00010000:00010000:2.0:1789760786.557715:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6184 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65806336->65810431] (req 65806336->65810431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.557728:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6184 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65806336->65810431] (req 65806336->65810431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.557736:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6184 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65806336->65810431] (req 65806336->65810431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.557743:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760786.557749:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6184 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65806336->65810431] (req 65806336->65810431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.557764:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6184 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65806336->65810431] (req 65806336->65810431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.557784:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6184 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65806336->65810431] (req 65806336->65810431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.557793:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6184 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65806336->65810431] (req 65806336->65810431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.558748:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554823168:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10489us (13628us total) trans 4294983375 rc 0/0 00000100:00100000:2.0:1789760786.558761:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16077 00000100:00100000:2.0:1789760786.558770:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729779561251 time 729793856843 state 0x2 ref 0 00000100:00100000:0.0:1789760786.564503:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760786.564642:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554823296 00000100:00100000:2.0:1789760786.564660:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554823296 00000100:00100000:2.0:1789760786.564713:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729779561251 time 729793856843 state 0x0 ref 1 00000100:00100000:2.0:1789760786.564728:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729794117259 time 729793856843 state 0x1 ref 1 00000100:00100000:2.0:1789760786.564757:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729794117259 time 729810375278 state 0x0 ref 1 00000100:00100000:2.0:1789760786.564768:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554823296:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760786.564815:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65810432, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f630 00010000:00010000:2.0:1789760786.564825:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c618b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.564842:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760786.564846:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.564853:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c618b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65810432->65814527] (req 65810432->65814527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.564867:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c618b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65810432->65814527] (req 65810432->65814527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.569954:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983376 is committed 00002000:00010000:2.0:1789760786.570088:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65810432 -> 65814528 00002000:00010000:2.0:1789760786.570094:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128560 -> 128568 00010000:00010000:2.0:1789760786.570107:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c618b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65810432->65814527] (req 65810432->65814527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.570120:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c618b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65810432->65814527] (req 65810432->65814527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.570128:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c618b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65810432->65814527] (req 65810432->65814527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.570135:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760786.570141:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c618b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65810432->65814527] (req 65810432->65814527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.570150:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c618b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65810432->65814527] (req 65810432->65814527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.572558:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c618b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65810432->65814527] (req 65810432->65814527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.572570:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c618b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65810432->65814527] (req 65810432->65814527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.572751:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554823296:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7979us (8251us total) trans 4294983376 rc 0/0 00000100:00100000:2.0:1789760786.572764:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16078 00000100:00100000:2.0:1789760786.572773:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729794117259 time 729810375278 state 0x2 ref 0 00000100:00100000:3.0:1789760786.578644:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760786.578843:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554823424 00000100:00100000:2.0:1789760786.578853:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554823424 00000100:00100000:2.0:1789760786.578906:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729794117259 time 729810375278 state 0x0 ref 1 00000100:00100000:2.0:1789760786.578921:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729810635694 time 729810375278 state 0x1 ref 1 00000100:00100000:2.0:1789760786.578954:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729810635694 time 729824572576 state 0x0 ref 1 00000100:00100000:2.0:1789760786.578966:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554823424:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760786.579013:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65814528, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f638 00010000:00010000:2.0:1789760786.579023:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6192 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.579037:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760786.579042:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.579047:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6192 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65814528->65818623] (req 65814528->65818623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.579059:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6192 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65814528->65818623] (req 65814528->65818623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.586284:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983377 is committed 00002000:00010000:1.0:1789760786.586713:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65814528 -> 65818624 00002000:00010000:1.0:1789760786.586719:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128568 -> 128576 00010000:00010000:1.0:1789760786.586732:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6192 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65814528->65818623] (req 65814528->65818623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.586744:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6192 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65814528->65818623] (req 65814528->65818623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.586754:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6192 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65814528->65818623] (req 65814528->65818623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.586762:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760786.586777:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6192 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65814528->65818623] (req 65814528->65818623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.586788:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6192 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65814528->65818623] (req 65814528->65818623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.586808:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6192 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65814528->65818623] (req 65814528->65818623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.586816:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6192 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65814528->65818623] (req 65814528->65818623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.586910:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554823424:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7940us (8270us total) trans 4294983377 rc 0/0 00000100:00100000:1.0:1789760786.586922:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16079 00000100:00100000:1.0:1789760786.586932:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729810635694 time 729824572576 state 0x2 ref 0 00000100:00100000:1.0:1789760786.592463:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760786.592586:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554823552 00000100:00100000:3.0:1789760786.592598:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554823552 00000100:00100000:3.0:1789760786.592665:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729810635694 time 729824572576 state 0x0 ref 1 00000100:00100000:3.0:1789760786.592680:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729824832992 time 729824572576 state 0x1 ref 1 00000100:00100000:3.0:1789760786.592702:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729824832992 time 729838320967 state 0x0 ref 1 00000100:00100000:3.0:1789760786.592712:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554823552:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760786.592759:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65818624, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f640 00010000:00010000:3.0:1789760786.592768:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6199 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.592784:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760786.592788:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.592794:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6199 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65818624->65822719] (req 65818624->65822719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.592807:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6199 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65818624->65822719] (req 65818624->65822719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.596321:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983378 is committed 00002000:00010000:3.0:1789760786.596479:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65818624 -> 65822720 00002000:00010000:3.0:1789760786.596485:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128576 -> 128584 00010000:00010000:3.0:1789760786.596497:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6199 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65818624->65822719] (req 65818624->65822719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.596510:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6199 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65818624->65822719] (req 65818624->65822719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.596519:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6199 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65818624->65822719] (req 65818624->65822719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.596526:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760786.596531:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6199 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65818624->65822719] (req 65818624->65822719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.596540:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6199 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65818624->65822719] (req 65818624->65822719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.596558:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6199 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65818624->65822719] (req 65818624->65822719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.596566:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6199 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65818624->65822719] (req 65818624->65822719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.596825:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554823552:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4108us (4366us total) trans 4294983378 rc 0/0 00000100:00100000:3.0:1789760786.596837:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16080 00000100:00100000:3.0:1789760786.596846:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729824832992 time 729838320967 state 0x2 ref 0 00000100:00100000:0.0:1789760786.600890:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760786.600985:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554823680 00000100:00100000:1.0:1789760786.600996:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554823680 00000100:00100000:1.0:1789760786.601054:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729824832992 time 729838320967 state 0x0 ref 1 00000100:00100000:1.0:1789760786.601069:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729838581383 time 729838320967 state 0x1 ref 1 00000100:00100000:1.0:1789760786.601124:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729838581383 time 729846742576 state 0x0 ref 1 00000100:00100000:1.0:1789760786.601136:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554823680:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760786.601179:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65822720, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f648 00010000:00010000:1.0:1789760786.601187:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61a0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.601202:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760786.601207:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.601214:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65822720->65826815] (req 65822720->65826815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.601228:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65822720->65826815] (req 65822720->65826815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.604743:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983379 is committed 00002000:00010000:1.0:1789760786.604920:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65822720 -> 65826816 00002000:00010000:1.0:1789760786.604925:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128584 -> 128592 00010000:00010000:1.0:1789760786.604938:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65822720->65826815] (req 65822720->65826815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.604951:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61a0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65822720->65826815] (req 65822720->65826815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.604959:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65822720->65826815] (req 65822720->65826815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.604966:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760786.604971:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65822720->65826815] (req 65822720->65826815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.604980:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61a0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65822720->65826815] (req 65822720->65826815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.605003:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61a0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65822720->65826815] (req 65822720->65826815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.605011:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61a0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65822720->65826815] (req 65822720->65826815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.605230:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554823680:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4091us (4345us total) trans 4294983379 rc 0/0 00000100:00100000:1.0:1789760786.605239:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16081 00000100:00100000:1.0:1789760786.605249:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729838581383 time 729846742576 state 0x2 ref 0 00000100:00100000:2.0:1789760786.609753:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760786.609862:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554823808 00000100:00100000:0.0:1789760786.609872:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554823808 00000100:00100000:0.0:1789760786.610175:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729838581383 time 729846742576 state 0x0 ref 1 00000100:00100000:0.0:1789760786.610194:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729847002992 time 729846742576 state 0x1 ref 1 00000100:00100000:0.0:1789760786.610226:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729847002992 time 729855844260 state 0x0 ref 1 00000100:00100000:0.0:1789760786.610239:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554823808:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760786.610287:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65826816, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f650 00010000:00010000:0.0:1789760786.610297:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c61a7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.610314:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760786.610323:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.610329:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c61a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65826816->65830911] (req 65826816->65830911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.610346:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c61a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65826816->65830911] (req 65826816->65830911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.617410:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983380 is committed 00002000:00010000:0.0:1789760786.617527:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65826816 -> 65830912 00002000:00010000:0.0:1789760786.617533:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128592 -> 128600 00010000:00010000:0.0:1789760786.617545:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c61a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65826816->65830911] (req 65826816->65830911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.617559:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c61a7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65826816->65830911] (req 65826816->65830911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.617568:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c61a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65826816->65830911] (req 65826816->65830911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.617575:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760786.617580:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c61a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65826816->65830911] (req 65826816->65830911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.617589:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c61a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65826816->65830911] (req 65826816->65830911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.617610:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c61a7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65826816->65830911] (req 65826816->65830911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.617618:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c61a7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65826816->65830911] (req 65826816->65830911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.618984:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554823808:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8738us (9232us total) trans 4294983380 rc 0/0 00000100:00100000:0.0:1789760786.618996:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16082 00000100:00100000:0.0:1789760786.619005:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729847002992 time 729855844260 state 0x2 ref 0 00000100:00100000:2.0:1789760786.625906:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760786.625999:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554823936 00000100:00100000:3.0:1789760786.626009:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554823936 00000100:00100000:3.0:1789760786.626077:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729847002992 time 729855844260 state 0x0 ref 1 00000100:00100000:3.0:1789760786.626092:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729856104676 time 729855844260 state 0x1 ref 1 00000100:00100000:3.0:1789760786.626126:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729856104676 time 729871744067 state 0x0 ref 1 00000100:00100000:3.0:1789760786.626139:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe18450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554823936:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760786.626185:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65830912, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f658 00010000:00010000:3.0:1789760786.626195:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c61ae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.626211:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760786.626215:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.626222:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c61ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65830912->65835007] (req 65830912->65835007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.626236:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c61ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65830912->65835007] (req 65830912->65835007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.630875:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983381 is committed 00002000:00010000:1.0:1789760786.630992:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65830912 -> 65835008 00002000:00010000:1.0:1789760786.630997:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128600 -> 128608 00010000:00010000:1.0:1789760786.631010:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c61ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65830912->65835007] (req 65830912->65835007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.631162:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c61ae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65830912->65835007] (req 65830912->65835007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.631172:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c61ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65830912->65835007] (req 65830912->65835007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.631179:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760786.631184:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c61ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65830912->65835007] (req 65830912->65835007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.631193:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c61ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65830912->65835007] (req 65830912->65835007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.631211:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c61ae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65830912->65835007] (req 65830912->65835007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.631223:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c61ae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65830912->65835007] (req 65830912->65835007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.631313:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe18450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554823936:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5172us (5412us total) trans 4294983381 rc 0/0 00000100:00100000:1.0:1789760786.631323:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16083 00000100:00100000:1.0:1789760786.631332:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729856104676 time 729871744067 state 0x2 ref 0 00000100:00100000:2.0:1789760786.640194:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760786.640267:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554824064 00000100:00100000:3.0:1789760786.640276:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554824064 00000100:00100000:3.0:1789760786.640324:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729856104676 time 729871744067 state 0x0 ref 1 00000100:00100000:3.0:1789760786.640338:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729872004483 time 729871744067 state 0x1 ref 1 00000100:00100000:3.0:1789760786.640368:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729872004483 time 729885986794 state 0x0 ref 1 00000100:00100000:3.0:1789760786.640415:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a386a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554824064:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760786.640464:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65835008, mtime: 0x6aad9511, atime: 0x0, ctime: 0x6aad9511, blocks: 0x1f660 00010000:00010000:3.0:1789760786.640473:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c61b5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.640489:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:3.0:1789760786.640493:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.640499:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c61b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65835008->65839103] (req 65835008->65839103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.640513:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c61b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65835008->65839103] (req 65835008->65839103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.648209:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983382 is committed 00002000:00010000:3.0:1789760786.648423:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65835008 -> 65839104 00002000:00010000:3.0:1789760786.648429:0:8443:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760785 -> 1789760786 00002000:00010000:3.0:1789760786.648431:0:8443:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760785 -> 1789760786 00002000:00010000:3.0:1789760786.648433:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128608 -> 128616 00010000:00010000:3.0:1789760786.648445:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c61b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65835008->65839103] (req 65835008->65839103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.648458:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c61b5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65835008->65839103] (req 65835008->65839103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.648468:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c61b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65835008->65839103] (req 65835008->65839103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.648476:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760786.648481:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c61b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65835008->65839103] (req 65835008->65839103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.648490:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c61b5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65835008->65839103] (req 65835008->65839103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.648511:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c61b5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65835008->65839103] (req 65835008->65839103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.648519:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c61b5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65835008->65839103] (req 65835008->65839103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.648684:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a386a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554824064:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8301us (8493us total) trans 4294983382 rc 0/0 00000100:00100000:3.0:1789760786.648695:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16084 00000100:00100000:3.0:1789760786.648705:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729872004483 time 729885986794 state 0x2 ref 0 00000100:00100000:2.0:1789760786.651849:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760786.652172:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554824192 00000100:00100000:1.0:1789760786.652181:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554824192 00000100:00100000:1.0:1789760786.652227:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729872004483 time 729885986794 state 0x0 ref 1 00000100:00100000:1.0:1789760786.652238:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729886247210 time 729885986794 state 0x1 ref 1 00000100:00100000:1.0:1789760786.652262:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729886247210 time 729897882337 state 0x0 ref 1 00000100:00100000:1.0:1789760786.652271:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a386300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554824192:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760786.652306:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65839104, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f668 00010000:00010000:1.0:1789760786.652312:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61bc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.652325:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760786.652328:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.652332:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65839104->65843199] (req 65839104->65843199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.652345:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65839104->65843199] (req 65839104->65843199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.658084:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983383 is committed 00002000:00010000:2.0:1789760786.658222:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65839104 -> 65843200 00002000:00010000:2.0:1789760786.658228:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128616 -> 128624 00010000:00010000:2.0:1789760786.658241:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65839104->65843199] (req 65839104->65843199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.658253:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61bc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65839104->65843199] (req 65839104->65843199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.658261:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65839104->65843199] (req 65839104->65843199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.658268:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760786.658273:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65839104->65843199] (req 65839104->65843199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.658281:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61bc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65839104->65843199] (req 65839104->65843199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.658301:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61bc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65839104->65843199] (req 65839104->65843199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.658309:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61bc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65839104->65843199] (req 65839104->65843199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.658662:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a386300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554824192:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6376us (6807us total) trans 4294983383 rc 0/0 00000100:00100000:2.0:1789760786.658673:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16085 00000100:00100000:2.0:1789760786.658683:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729886247210 time 729897882337 state 0x2 ref 0 00000100:00100000:2.0:1789760786.666732:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760786.666832:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554824320 00000100:00100000:0.0:1789760786.666842:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554824320 00000100:00100000:0.0:1789760786.666896:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729886247210 time 729897882337 state 0x0 ref 1 00000100:00100000:0.0:1789760786.666911:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729898142753 time 729897882337 state 0x1 ref 1 00000100:00100000:0.0:1789760786.666944:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729898142753 time 729912562375 state 0x0 ref 1 00000100:00100000:0.0:1789760786.666957:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554824320:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760786.667003:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65843200, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f670 00010000:00010000:0.0:1789760786.667012:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c61c3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.667700:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:0.0:1789760786.667708:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.667715:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c61c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65843200->65847295] (req 65843200->65847295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.667731:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c61c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65843200->65847295] (req 65843200->65847295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.677553:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983384 is committed 00002000:00010000:2.0:1789760786.677946:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65843200 -> 65847296 00002000:00010000:2.0:1789760786.677952:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128624 -> 128632 00010000:00010000:2.0:1789760786.677964:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c61c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65843200->65847295] (req 65843200->65847295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.677978:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c61c3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65843200->65847295] (req 65843200->65847295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.677987:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c61c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65843200->65847295] (req 65843200->65847295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.677994:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760786.677999:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c61c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65843200->65847295] (req 65843200->65847295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.678008:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c61c3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65843200->65847295] (req 65843200->65847295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.678092:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c61c3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65843200->65847295] (req 65843200->65847295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.678101:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c61c3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65843200->65847295] (req 65843200->65847295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.678235:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554824320:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11275us (11508us total) trans 4294983384 rc 0/0 00000100:00100000:2.0:1789760786.678245:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16086 00000100:00100000:2.0:1789760786.678255:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729898142753 time 729912562375 state 0x2 ref 0 00000100:00100000:3.0:1789760786.687531:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760786.687759:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554824448 00000100:00100000:1.0:1789760786.687768:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554824448 00000100:00100000:1.0:1789760786.687820:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729898142753 time 729912562375 state 0x0 ref 1 00000100:00100000:1.0:1789760786.687835:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729912822791 time 729912562375 state 0x1 ref 1 00000100:00100000:1.0:1789760786.687923:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729912822791 time 729933540465 state 0x0 ref 1 00000100:00100000:1.0:1789760786.687936:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554824448:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760786.687998:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65847296, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f678 00010000:00010000:1.0:1789760786.688007:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c61ca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.688034:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760786.688039:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.688045:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c61ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65847296->65851391] (req 65847296->65851391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.688059:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c61ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65847296->65851391] (req 65847296->65851391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.695957:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983385 is committed 00002000:00010000:3.0:1789760786.696080:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65847296 -> 65851392 00002000:00010000:3.0:1789760786.696085:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128632 -> 128640 00010000:00010000:3.0:1789760786.696099:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c61ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65847296->65851391] (req 65847296->65851391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.696110:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c61ca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65847296->65851391] (req 65847296->65851391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.696118:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c61ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65847296->65851391] (req 65847296->65851391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.696125:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760786.696131:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c61ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65847296->65851391] (req 65847296->65851391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.696140:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c61ca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65847296->65851391] (req 65847296->65851391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.696159:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c61ca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65847296->65851391] (req 65847296->65851391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.696166:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c61ca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65847296->65851391] (req 65847296->65851391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.696325:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554824448:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8386us (8799us total) trans 4294983385 rc 0/0 00000100:00100000:3.0:1789760786.696336:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16087 00000100:00100000:3.0:1789760786.696345:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729912822791 time 729933540465 state 0x2 ref 0 00000100:00100000:3.0:1789760786.703470:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760786.703844:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554824576 00000100:00100000:2.0:1789760786.703853:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554824576 00000100:00100000:2.0:1789760786.703909:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729912822791 time 729933540465 state 0x0 ref 1 00000100:00100000:2.0:1789760786.703924:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729933800881 time 729933540465 state 0x1 ref 1 00000100:00100000:2.0:1789760786.704012:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729933800881 time 729949629813 state 0x0 ref 1 00000100:00100000:2.0:1789760786.704025:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554824576:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760786.704071:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65851392, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f680 00010000:00010000:2.0:1789760786.704080:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c61d1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.704095:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760786.704099:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.704105:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c61d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65851392->65855487] (req 65851392->65855487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.704131:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c61d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65851392->65855487] (req 65851392->65855487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.710734:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983386 is committed 00002000:00010000:2.0:1789760786.710989:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65851392 -> 65855488 00002000:00010000:2.0:1789760786.710995:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128640 -> 128648 00010000:00010000:2.0:1789760786.711008:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c61d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65851392->65855487] (req 65851392->65855487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.711019:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c61d1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65851392->65855487] (req 65851392->65855487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.711135:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c61d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65851392->65855487] (req 65851392->65855487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.711143:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760786.711147:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c61d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65851392->65855487] (req 65851392->65855487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.711155:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c61d1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65851392->65855487] (req 65851392->65855487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.711183:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c61d1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65851392->65855487] (req 65851392->65855487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.711192:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c61d1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65851392->65855487] (req 65851392->65855487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.711351:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554824576:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7324us (7886us total) trans 4294983386 rc 0/0 00000100:00100000:2.0:1789760786.711362:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16088 00000100:00100000:2.0:1789760786.711371:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729933800881 time 729949629813 state 0x2 ref 0 00000100:00100000:2.0:1789760786.718023:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760786.718136:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554824704 00000100:00100000:1.0:1789760786.718146:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554824704 00000100:00100000:1.0:1789760786.718201:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729933800881 time 729949629813 state 0x0 ref 1 00000100:00100000:1.0:1789760786.718217:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729949890229 time 729949629813 state 0x1 ref 1 00000100:00100000:1.0:1789760786.718244:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729949890229 time 729963863225 state 0x0 ref 1 00000100:00100000:1.0:1789760786.718256:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a387480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554824704:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760786.718304:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65855488, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f688 00010000:00010000:1.0:1789760786.718313:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c61d8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.718329:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:1.0:1789760786.718334:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.718341:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c61d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65855488->65859583] (req 65855488->65859583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.718355:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c61d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65855488->65859583] (req 65855488->65859583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.724010:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983387 is committed 00002000:00010000:1.0:1789760786.724188:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65855488 -> 65859584 00002000:00010000:1.0:1789760786.724193:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128648 -> 128656 00010000:00010000:1.0:1789760786.724207:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c61d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65855488->65859583] (req 65855488->65859583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.724219:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c61d8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65855488->65859583] (req 65855488->65859583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.724229:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c61d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65855488->65859583] (req 65855488->65859583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.724236:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760786.724241:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c61d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65855488->65859583] (req 65855488->65859583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.724250:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c61d8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65855488->65859583] (req 65855488->65859583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.724273:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c61d8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65855488->65859583] (req 65855488->65859583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.724280:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c61d8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65855488->65859583] (req 65855488->65859583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.724414:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a387480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554824704:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6117us (6359us total) trans 4294983387 rc 0/0 00000100:00100000:1.0:1789760786.724426:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16089 00000100:00100000:1.0:1789760786.724445:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729949890229 time 729963863225 state 0x2 ref 0 00000100:00100000:2.0:1789760786.729276:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760786.729310:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554824832 00000100:00100000:1.0:1789760786.729319:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554824832 00000100:00100000:1.0:1789760786.729371:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729949890229 time 729963863225 state 0x0 ref 1 00000100:00100000:1.0:1789760786.729416:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729964123641 time 729963863225 state 0x1 ref 1 00000100:00100000:2.0:1789760786.729551:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729964123641 time 729975167204 state 0x0 ref 1 00000100:00100000:2.0:1789760786.729566:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a387b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554824832:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760786.729623:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65859584, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f690 00010000:00010000:2.0:1789760786.729634:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c61df lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.729659:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=729s v=(0 0) 00010000:00010000:2.0:1789760786.729664:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.729669:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c61df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65859584->65863679] (req 65859584->65863679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.729688:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c61df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65859584->65863679] (req 65859584->65863679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.739231:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983388 is committed 00002000:00010000:2.0:1789760786.739449:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65859584 -> 65863680 00002000:00010000:2.0:1789760786.739454:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128656 -> 128664 00010000:00010000:2.0:1789760786.739466:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c61df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65859584->65863679] (req 65859584->65863679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.739784:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c61df lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65859584->65863679] (req 65859584->65863679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.739794:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c61df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65859584->65863679] (req 65859584->65863679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.739800:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760786.739804:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c61df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65859584->65863679] (req 65859584->65863679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.739813:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c61df lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65859584->65863679] (req 65859584->65863679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.739831:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c61df lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65859584->65863679] (req 65859584->65863679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.739838:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c61df lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65859584->65863679] (req 65859584->65863679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.739945:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a387b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554824832:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10378us (10675us total) trans 4294983388 rc 0/0 00000100:00100000:2.0:1789760786.739955:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16090 00000100:00100000:2.0:1789760786.739965:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729964123641 time 729975167204 state 0x2 ref 0 00000100:00100000:3.0:1789760786.753967:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760786.761979:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554824960 00000100:00100000:1.0:1789760786.762004:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554824960 00000100:00100000:1.0:1789760786.762335:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729964123641 time 729975167204 state 0x0 ref 1 00000100:00100000:1.0:1789760786.762351:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729975427620 time 729975167204 state 0x1 ref 1 00000100:00100000:1.0:1789760786.762413:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729975427620 time 730008031100 state 0x0 ref 1 00000100:00100000:1.0:1789760786.762427:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554824960:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760786.762954:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65863680, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f698 00010000:00010000:1.0:1789760786.762964:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61e6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.762978:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760786.762983:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.762989:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65863680->65867775] (req 65863680->65867775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.763014:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65863680->65867775] (req 65863680->65867775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.771764:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983389 is committed 00002000:00010000:1.0:1789760786.772451:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65863680 -> 65867776 00002000:00010000:1.0:1789760786.772456:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128664 -> 128672 00010000:00010000:1.0:1789760786.772469:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65863680->65867775] (req 65863680->65867775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.772482:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65863680->65867775] (req 65863680->65867775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.772491:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65863680->65867775] (req 65863680->65867775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.772498:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760786.772503:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65863680->65867775] (req 65863680->65867775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.772511:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65863680->65867775] (req 65863680->65867775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.772531:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61e6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65863680->65867775] (req 65863680->65867775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.772538:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61e6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65863680->65867775] (req 65863680->65867775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.772641:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554824960:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10202us (18669us total) trans 4294983389 rc 0/0 00000100:00100000:1.0:1789760786.772661:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16091 00000100:00100000:1.0:1789760786.772670:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729975427620 time 730008031100 state 0x2 ref 0 00000100:00100000:1.0:1789760786.787808:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760786.787899:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554825088 00000100:00100000:2.0:1789760786.787908:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554825088 00000100:00100000:2.0:1789760786.787967:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 729975427620 time 730008031100 state 0x0 ref 1 00000100:00100000:2.0:1789760786.787982:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730008291516 time 730008031100 state 0x1 ref 1 00000100:00100000:2.0:1789760786.788013:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730008291516 time 730033631676 state 0x0 ref 1 00000100:00100000:2.0:1789760786.788036:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554825088:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760786.788083:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65867776, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f6a0 00010000:00010000:2.0:1789760786.788092:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c61ed lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.788106:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760786.788110:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.788117:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c61ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65867776->65871871] (req 65867776->65871871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.788130:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c61ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65867776->65871871] (req 65867776->65871871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.797607:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983390 is committed 00002000:00010000:2.0:1789760786.797732:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65867776 -> 65871872 00002000:00010000:2.0:1789760786.797737:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128672 -> 128680 00010000:00010000:2.0:1789760786.797752:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c61ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65867776->65871871] (req 65867776->65871871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.797764:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c61ed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65867776->65871871] (req 65867776->65871871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.797773:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c61ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65867776->65871871] (req 65867776->65871871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.797780:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760786.797785:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c61ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65867776->65871871] (req 65867776->65871871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.797795:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c61ed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65867776->65871871] (req 65867776->65871871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.797815:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c61ed lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65867776->65871871] (req 65867776->65871871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.797822:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c61ed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65867776->65871871] (req 65867776->65871871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.797950:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554825088:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9920us (10145us total) trans 4294983390 rc 0/0 00000100:00100000:2.0:1789760786.797961:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16092 00000100:00100000:2.0:1789760786.797970:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730008291516 time 730033631676 state 0x2 ref 0 00000100:00100000:2.0:1789760786.807784:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760786.807892:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554825216 00000100:00100000:1.0:1789760786.807901:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554825216 00000100:00100000:1.0:1789760786.807957:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730008291516 time 730033631676 state 0x0 ref 1 00000100:00100000:1.0:1789760786.807972:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730033892092 time 730033631676 state 0x1 ref 1 00000100:00100000:1.0:1789760786.808004:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730033892092 time 730053623169 state 0x0 ref 1 00000100:00100000:1.0:1789760786.808016:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a384a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554825216:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760786.808065:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65871872, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f6a8 00010000:00010000:1.0:1789760786.808074:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61f4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.808090:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760786.808095:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.808102:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65871872->65875967] (req 65871872->65875967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.808116:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65871872->65875967] (req 65871872->65875967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.815042:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983391 is committed 00002000:00010000:0.0:1789760786.815201:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65871872 -> 65875968 00002000:00010000:0.0:1789760786.815206:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128680 -> 128688 00010000:00010000:0.0:1789760786.815218:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65871872->65875967] (req 65871872->65875967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.815231:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61f4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65871872->65875967] (req 65871872->65875967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.815241:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65871872->65875967] (req 65871872->65875967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.815248:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760786.815253:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65871872->65875967] (req 65871872->65875967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.815262:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61f4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65871872->65875967] (req 65871872->65875967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.815282:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61f4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65871872->65875967] (req 65871872->65875967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.815289:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c61f4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65871872->65875967] (req 65871872->65875967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.815429:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a384a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554825216:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7363us (7604us total) trans 4294983391 rc 0/0 00000100:00100000:0.0:1789760786.815440:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16093 00000100:00100000:0.0:1789760786.815448:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730033892092 time 730053623169 state 0x2 ref 0 00000100:00100000:3.0:1789760786.823908:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760786.824118:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554825344 00000100:00100000:2.0:1789760786.824129:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554825344 00000100:00100000:2.0:1789760786.824185:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730033892092 time 730053623169 state 0x0 ref 1 00000100:00100000:2.0:1789760786.824200:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730053883585 time 730053623169 state 0x1 ref 1 00000100:00100000:2.0:1789760786.824233:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730053883585 time 730069851652 state 0x0 ref 1 00000100:00100000:2.0:1789760786.824245:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554825344:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760786.824290:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65875968, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f6b0 00010000:00010000:2.0:1789760786.824298:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c61fb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.824313:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760786.824317:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.824324:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c61fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65875968->65880063] (req 65875968->65880063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.824337:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c61fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65875968->65880063] (req 65875968->65880063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760786.837223:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983392 is committed 00002000:00010000:2.0:1789760786.837530:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65875968 -> 65880064 00002000:00010000:2.0:1789760786.837536:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128688 -> 128696 00010000:00010000:2.0:1789760786.837548:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c61fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65875968->65880063] (req 65875968->65880063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.837560:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c61fb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65875968->65880063] (req 65875968->65880063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.837569:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c61fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65875968->65880063] (req 65875968->65880063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.837576:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760786.837581:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c61fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65875968->65880063] (req 65875968->65880063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.837590:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c61fb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65875968->65880063] (req 65875968->65880063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.837608:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c61fb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65875968->65880063] (req 65875968->65880063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.837616:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c61fb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65875968->65880063] (req 65875968->65880063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.837885:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554825344:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13635us (13979us total) trans 4294983392 rc 0/0 00000100:00100000:2.0:1789760786.837897:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16094 00000100:00100000:2.0:1789760786.837907:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730053883585 time 730069851652 state 0x2 ref 0 00000100:00100000:0.0:1789760786.844984:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760786.845196:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554825472 00000100:00100000:3.0:1789760786.845206:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554825472 00000100:00100000:3.0:1789760786.845263:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730053883585 time 730069851652 state 0x0 ref 1 00000100:00100000:3.0:1789760786.845278:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730070112068 time 730069851652 state 0x1 ref 1 00000100:00100000:3.0:1789760786.845508:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730070112068 time 730091124080 state 0x0 ref 1 00000100:00100000:3.0:1789760786.845522:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554825472:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760786.845570:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65880064, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f6b8 00010000:00010000:3.0:1789760786.845580:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6202 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.845606:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760786.845611:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.845618:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6202 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65880064->65884159] (req 65880064->65884159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.845632:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6202 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65880064->65884159] (req 65880064->65884159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.853248:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983393 is committed 00002000:00010000:3.0:1789760786.853370:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65880064 -> 65884160 00002000:00010000:3.0:1789760786.853398:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128696 -> 128704 00010000:00010000:3.0:1789760786.853422:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6202 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65880064->65884159] (req 65880064->65884159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.853442:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6202 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65880064->65884159] (req 65880064->65884159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.853450:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6202 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65880064->65884159] (req 65880064->65884159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.853457:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760786.853462:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6202 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65880064->65884159] (req 65880064->65884159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.853470:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6202 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65880064->65884159] (req 65880064->65884159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.853489:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6202 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65880064->65884159] (req 65880064->65884159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.853496:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6202 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65880064->65884159] (req 65880064->65884159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.853591:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554825472:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8066us (8611us total) trans 4294983393 rc 0/0 00000100:00100000:3.0:1789760786.853602:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16095 00000100:00100000:3.0:1789760786.853611:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730070112068 time 730091124080 state 0x2 ref 0 00000100:00100000:0.0:1789760786.858901:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760786.859045:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554825600 00000100:00100000:1.0:1789760786.859054:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554825600 00000100:00100000:1.0:1789760786.859106:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730070112068 time 730091124080 state 0x0 ref 1 00000100:00100000:1.0:1789760786.859121:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730091384496 time 730091124080 state 0x1 ref 1 00000100:00100000:1.0:1789760786.859152:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730091384496 time 730104770823 state 0x0 ref 1 00000100:00100000:1.0:1789760786.859164:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554825600:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760786.859212:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65884160, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f6c0 00010000:00010000:1.0:1789760786.859222:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6209 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.859238:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760786.859243:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.859249:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6209 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65884160->65888255] (req 65884160->65888255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.859261:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6209 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65884160->65888255] (req 65884160->65888255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.867632:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983394 is committed 00002000:00010000:1.0:1789760786.867788:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65884160 -> 65888256 00002000:00010000:1.0:1789760786.867794:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128704 -> 128712 00010000:00010000:1.0:1789760786.867808:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6209 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65884160->65888255] (req 65884160->65888255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.867821:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6209 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65884160->65888255] (req 65884160->65888255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.867829:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6209 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65884160->65888255] (req 65884160->65888255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.867836:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760786.867841:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6209 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65884160->65888255] (req 65884160->65888255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.867850:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6209 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65884160->65888255] (req 65884160->65888255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.867941:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6209 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65884160->65888255] (req 65884160->65888255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.867950:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6209 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65884160->65888255] (req 65884160->65888255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.868043:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554825600:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8875us (9146us total) trans 4294983394 rc 0/0 00000100:00100000:1.0:1789760786.868053:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16096 00000100:00100000:1.0:1789760786.868061:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730091384496 time 730104770823 state 0x2 ref 0 00000100:00100000:2.0:1789760786.873595:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760786.873690:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554825728 00000100:00100000:0.0:1789760786.873700:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554825728 00000100:00100000:0.0:1789760786.873764:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730091384496 time 730104770823 state 0x0 ref 1 00000100:00100000:0.0:1789760786.873780:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730105031239 time 730104770823 state 0x1 ref 1 00000100:00100000:0.0:1789760786.873810:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730105031239 time 730119428330 state 0x0 ref 1 00000100:00100000:0.0:1789760786.873822:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554825728:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760786.874042:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65888256, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f6c8 00010000:00010000:0.0:1789760786.874054:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6210 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.874068:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760786.874073:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.874080:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6210 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65888256->65892351] (req 65888256->65892351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.874093:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6210 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65888256->65892351] (req 65888256->65892351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.882316:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983395 is committed 00002000:00010000:0.0:1789760786.882464:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65888256 -> 65892352 00002000:00010000:0.0:1789760786.882470:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128712 -> 128720 00010000:00010000:0.0:1789760786.882589:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6210 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65888256->65892351] (req 65888256->65892351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.882604:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6210 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65888256->65892351] (req 65888256->65892351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.882619:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6210 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65888256->65892351] (req 65888256->65892351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.882626:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760786.882630:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6210 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65888256->65892351] (req 65888256->65892351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.882640:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6210 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65888256->65892351] (req 65888256->65892351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.882671:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6210 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65888256->65892351] (req 65888256->65892351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.882679:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6210 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65888256->65892351] (req 65888256->65892351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.882838:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554825728:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9013us (9259us total) trans 4294983395 rc 0/0 00000100:00100000:0.0:1789760786.882849:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16097 00000100:00100000:0.0:1789760786.882858:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730105031239 time 730119428330 state 0x2 ref 0 00000100:00100000:0.0:1789760786.889626:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760786.889892:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554825856 00000100:00100000:2.0:1789760786.889903:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554825856 00000100:00100000:2.0:1789760786.889954:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730105031239 time 730119428330 state 0x0 ref 1 00000100:00100000:2.0:1789760786.889969:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730119688746 time 730119428330 state 0x1 ref 1 00000100:00100000:2.0:1789760786.889999:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730119688746 time 730135617636 state 0x0 ref 1 00000100:00100000:2.0:1789760786.890010:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554825856:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760786.890055:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65892352, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f6d0 00010000:00010000:2.0:1789760786.890065:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6217 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.890082:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760786.890086:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.890092:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6217 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65892352->65896447] (req 65892352->65896447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.890106:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6217 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65892352->65896447] (req 65892352->65896447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760786.899150:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983396 is committed 00002000:00010000:2.0:1789760786.899261:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65892352 -> 65896448 00002000:00010000:2.0:1789760786.899267:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128720 -> 128728 00010000:00010000:2.0:1789760786.899279:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6217 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65892352->65896447] (req 65892352->65896447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.899298:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6217 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65892352->65896447] (req 65892352->65896447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.899308:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6217 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65892352->65896447] (req 65892352->65896447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.899315:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760786.899320:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6217 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65892352->65896447] (req 65892352->65896447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.899328:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6217 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65892352->65896447] (req 65892352->65896447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.899346:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6217 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65892352->65896447] (req 65892352->65896447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.899353:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6217 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65892352->65896447] (req 65892352->65896447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.899503:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554825856:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9490us (9883us total) trans 4294983396 rc 0/0 00000100:00100000:2.0:1789760786.899514:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16098 00000100:00100000:2.0:1789760786.899523:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730119688746 time 730135617636 state 0x2 ref 0 00000100:00100000:2.0:1789760786.908269:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760786.908417:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554825984 00000100:00100000:3.0:1789760786.908426:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554825984 00000100:00100000:3.0:1789760786.908476:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730119688746 time 730135617636 state 0x0 ref 1 00000100:00100000:3.0:1789760786.908491:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730135878052 time 730135617636 state 0x1 ref 1 00000100:00100000:3.0:1789760786.908511:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730135878052 time 730154129963 state 0x0 ref 1 00000100:00100000:3.0:1789760786.908521:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a386680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554825984:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760786.908567:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65896448, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f6d8 00010000:00010000:3.0:1789760786.908576:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c621e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.908592:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760786.908596:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.908602:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c621e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65896448->65900543] (req 65896448->65900543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.908615:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c621e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65896448->65900543] (req 65896448->65900543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.916581:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983397 is committed 00002000:00010000:3.0:1789760786.916934:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65896448 -> 65900544 00002000:00010000:3.0:1789760786.916940:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128728 -> 128736 00010000:00010000:3.0:1789760786.916954:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c621e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65896448->65900543] (req 65896448->65900543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.916973:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c621e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65896448->65900543] (req 65896448->65900543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.916982:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c621e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65896448->65900543] (req 65896448->65900543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.916990:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760786.916995:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c621e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65896448->65900543] (req 65896448->65900543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.917005:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c621e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65896448->65900543] (req 65896448->65900543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.917035:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c621e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65896448->65900543] (req 65896448->65900543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.917044:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c621e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65896448->65900543] (req 65896448->65900543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760786.917217:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a386680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554825984:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8683us (8945us total) trans 4294983397 rc 0/0 00000100:00100000:3.0:1789760786.917229:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16099 00000100:00100000:3.0:1789760786.917239:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730135878052 time 730154129963 state 0x2 ref 0 00000100:00100000:3.0:1789760786.923753:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760786.923927:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554826112 00000100:00100000:1.0:1789760786.923936:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554826112 00000100:00100000:1.0:1789760786.923992:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730135878052 time 730154129963 state 0x0 ref 1 00000100:00100000:1.0:1789760786.924009:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730154390379 time 730154129963 state 0x1 ref 1 00000100:00100000:1.0:1789760786.924103:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730154390379 time 730169720358 state 0x0 ref 1 00000100:00100000:1.0:1789760786.924118:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554826112:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760786.924179:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65900544, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f6e0 00010000:00010000:1.0:1789760786.924188:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6225 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.924215:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760786.924221:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.924228:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6225 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65900544->65904639] (req 65900544->65904639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.924243:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6225 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65900544->65904639] (req 65900544->65904639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760786.931424:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983398 is committed 00002000:00010000:1.0:1789760786.931710:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65900544 -> 65904640 00002000:00010000:1.0:1789760786.931717:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128736 -> 128744 00010000:00010000:1.0:1789760786.931730:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6225 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65900544->65904639] (req 65900544->65904639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.931749:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6225 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65900544->65904639] (req 65900544->65904639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.931758:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6225 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65900544->65904639] (req 65900544->65904639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.931765:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760786.931770:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6225 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65900544->65904639] (req 65900544->65904639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.931780:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6225 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65900544->65904639] (req 65900544->65904639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.931801:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6225 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65900544->65904639] (req 65900544->65904639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.931810:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6225 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65900544->65904639] (req 65900544->65904639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760786.931902:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554826112:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7781us (8153us total) trans 4294983398 rc 0/0 00000100:00100000:1.0:1789760786.931912:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16100 00000100:00100000:1.0:1789760786.931921:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730154390379 time 730169720358 state 0x2 ref 0 00000100:00100000:0.0:1789760786.940657:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760786.940761:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554826240 00000100:00100000:2.0:1789760786.940771:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554826240 00000100:00100000:2.0:1789760786.940826:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730154390379 time 730169720358 state 0x0 ref 1 00000100:00100000:2.0:1789760786.940841:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730169980774 time 730169720358 state 0x1 ref 1 00000100:00100000:2.0:1789760786.940873:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730169980774 time 730186490947 state 0x0 ref 1 00000100:00100000:2.0:1789760786.940885:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554826240:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760786.940934:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65904640, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f6e8 00010000:00010000:2.0:1789760786.940944:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c622c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.940970:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760786.940975:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760786.940981:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c622c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65904640->65908735] (req 65904640->65908735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.940994:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c622c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65904640->65908735] (req 65904640->65908735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.946765:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983399 is committed 00002000:00010000:2.0:1789760786.947766:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65904640 -> 65908736 00002000:00010000:2.0:1789760786.947774:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128744 -> 128752 00010000:00010000:2.0:1789760786.947789:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c622c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65904640->65908735] (req 65904640->65908735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.947807:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c622c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65904640->65908735] (req 65904640->65908735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.947816:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c622c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65904640->65908735] (req 65904640->65908735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.947824:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760786.947829:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c622c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65904640->65908735] (req 65904640->65908735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.947838:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c622c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65904640->65908735] (req 65904640->65908735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.947858:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c622c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65904640->65908735] (req 65904640->65908735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.947866:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c622c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65904640->65908735] (req 65904640->65908735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.949449:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554826240:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8555us (8800us total) trans 4294983399 rc 0/0 00000100:00100000:2.0:1789760786.949465:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16101 00000100:00100000:2.0:1789760786.949474:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730169980774 time 730186490947 state 0x2 ref 0 00000100:00100000:3.0:1789760786.958695:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760786.959580:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554826368 00000100:00100000:0.0:1789760786.959592:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554826368 00000100:00100000:0.0:1789760786.959664:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730169980774 time 730186490947 state 0x0 ref 1 00000100:00100000:0.0:1789760786.959681:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730186751363 time 730186490947 state 0x1 ref 1 00000100:00100000:0.0:1789760786.959712:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730186751363 time 730205330573 state 0x0 ref 1 00000100:00100000:0.0:1789760786.959726:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554826368:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760786.959773:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65908736, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f6f0 00010000:00010000:0.0:1789760786.959782:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6233 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.959796:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760786.959801:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760786.959807:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6233 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65908736->65912831] (req 65908736->65912831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.959820:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6233 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65908736->65912831] (req 65908736->65912831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.966798:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983400 is committed 00002000:00010000:0.0:1789760786.966919:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65908736 -> 65912832 00002000:00010000:0.0:1789760786.966924:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128752 -> 128760 00010000:00010000:0.0:1789760786.966937:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6233 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65908736->65912831] (req 65908736->65912831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.966949:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6233 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65908736->65912831] (req 65908736->65912831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.966962:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6233 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65908736->65912831] (req 65908736->65912831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.966970:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760786.966976:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6233 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65908736->65912831] (req 65908736->65912831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.966984:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6233 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65908736->65912831] (req 65908736->65912831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.967003:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6233 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65908736->65912831] (req 65908736->65912831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760786.967011:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6233 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65908736->65912831] (req 65908736->65912831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760786.967124:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554826368:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7396us (8433us total) trans 4294983400 rc 0/0 00000100:00100000:0.0:1789760786.967134:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16102 00000100:00100000:0.0:1789760786.967143:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730186751363 time 730205330573 state 0x2 ref 0 00000100:00100000:2.0:1789760786.970831:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760786.971354:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554826496 00000100:00100000:3.0:1789760786.971364:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554826496 00000100:00100000:3.0:1789760786.971449:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730186751363 time 730205330573 state 0x0 ref 1 00000100:00100000:3.0:1789760786.971465:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730205590989 time 730205330573 state 0x1 ref 1 00000100:00100000:3.0:1789760786.971497:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730205590989 time 730217115267 state 0x0 ref 1 00000100:00100000:3.0:1789760786.971511:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554826496:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760786.971563:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65912832, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f6f8 00010000:00010000:3.0:1789760786.971573:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c623a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.971590:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760786.971595:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760786.971602:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c623a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65912832->65916927] (req 65912832->65916927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760786.971616:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c623a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65912832->65916927] (req 65912832->65916927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760786.985666:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983401 is committed 00002000:00010000:2.0:1789760786.985852:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65912832 -> 65916928 00002000:00010000:2.0:1789760786.985858:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128760 -> 128768 00010000:00010000:2.0:1789760786.985871:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c623a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65912832->65916927] (req 65912832->65916927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.985886:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c623a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65912832->65916927] (req 65912832->65916927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.985895:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c623a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65912832->65916927] (req 65912832->65916927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.985903:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760786.985907:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c623a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65912832->65916927] (req 65912832->65916927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.985932:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c623a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65912832->65916927] (req 65912832->65916927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.985952:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c623a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65912832->65916927] (req 65912832->65916927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760786.985959:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c623a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65912832->65916927] (req 65912832->65916927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760786.986065:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554826496:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14552us (15239us total) trans 4294983401 rc 0/0 00000100:00100000:2.0:1789760786.986075:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16103 00000100:00100000:2.0:1789760786.986085:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730205590989 time 730217115267 state 0x2 ref 0 00000100:00100000:2.0:1789760786.996786:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760786.996867:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554826624 00000100:00100000:1.0:1789760786.996878:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554826624 00000100:00100000:1.0:1789760786.997007:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730205590989 time 730217115267 state 0x0 ref 1 00000100:00100000:1.0:1789760786.997022:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730217375683 time 730217115267 state 0x1 ref 1 00000100:00100000:1.0:1789760786.997100:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730217375683 time 730242717881 state 0x0 ref 1 00000100:00100000:1.0:1789760786.997113:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554826624:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760786.997161:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65916928, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f700 00010000:00010000:1.0:1789760786.997171:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6241 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.997186:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760786.997190:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760786.997196:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6241 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65916928->65921023] (req 65916928->65921023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760786.997224:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6241 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65916928->65921023] (req 65916928->65921023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.004372:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983402 is committed 00002000:00010000:1.0:1789760787.004781:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65916928 -> 65921024 00002000:00010000:1.0:1789760787.004786:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128768 -> 128776 00010000:00010000:1.0:1789760787.004799:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6241 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65916928->65921023] (req 65916928->65921023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.004820:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6241 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65916928->65921023] (req 65916928->65921023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.004830:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6241 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65916928->65921023] (req 65916928->65921023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.004849:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760787.004854:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6241 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65916928->65921023] (req 65916928->65921023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.004865:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6241 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65916928->65921023] (req 65916928->65921023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.004886:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6241 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65916928->65921023] (req 65916928->65921023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.004894:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6241 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65916928->65921023] (req 65916928->65921023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.005031:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554826624:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7907us (8243us total) trans 4294983402 rc 0/0 00000100:00100000:1.0:1789760787.005043:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16104 00000100:00100000:1.0:1789760787.005052:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730217375683 time 730242717881 state 0x2 ref 0 00000100:00100000:0.0:1789760787.015044:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760787.015156:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554826752 00000100:00100000:2.0:1789760787.015166:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554826752 00000100:00100000:2.0:1789760787.015234:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730217375683 time 730242717881 state 0x0 ref 1 00000100:00100000:2.0:1789760787.015250:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730242978297 time 730242717881 state 0x1 ref 1 00000100:00100000:2.0:1789760787.015275:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730242978297 time 730260893906 state 0x0 ref 1 00000100:00100000:2.0:1789760787.015287:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554826752:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760787.015334:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65921024, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f708 00010000:00010000:2.0:1789760787.015343:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6248 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.015359:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760787.015364:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.015370:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6248 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65921024->65925119] (req 65921024->65925119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.015429:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6248 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65921024->65925119] (req 65921024->65925119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760787.016344:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760787.022922:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983403 is committed 00002000:00010000:2.0:1789760787.023194:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65921024 -> 65925120 00002000:00010000:2.0:1789760787.023200:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128776 -> 128784 00010000:00010000:2.0:1789760787.023213:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6248 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65921024->65925119] (req 65921024->65925119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.023227:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6248 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65921024->65925119] (req 65921024->65925119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.023236:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6248 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65921024->65925119] (req 65921024->65925119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.023243:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760787.023249:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6248 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65921024->65925119] (req 65921024->65925119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.023257:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6248 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65921024->65925119] (req 65921024->65925119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.023277:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6248 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65921024->65925119] (req 65921024->65925119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.023285:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6248 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65921024->65925119] (req 65921024->65925119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.023479:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554826752:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8189us (8440us total) trans 4294983403 rc 0/0 00000100:00100000:2.0:1789760787.023491:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16105 00000100:00100000:2.0:1789760787.023499:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730242978297 time 730260893906 state 0x2 ref 0 00010000:00010000:3.0:1789760787.026021:0:8453:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789760787.030074:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760787.030249:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554826880 00000100:00100000:0.0:1789760787.030259:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554826880 00000100:00100000:0.0:1789760787.030313:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730242978297 time 730260893906 state 0x0 ref 1 00000100:00100000:0.0:1789760787.030329:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730261154322 time 730260893906 state 0x1 ref 1 00000100:00100000:0.0:1789760787.030359:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730261154322 time 730275977982 state 0x0 ref 1 00000100:00100000:0.0:1789760787.030368:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a384e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554826880:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760787.030444:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65925120, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f710 00010000:00010000:0.0:1789760787.030453:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c624f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.030468:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760787.030472:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.030479:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c624f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65925120->65929215] (req 65925120->65929215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.030493:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c624f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65925120->65929215] (req 65925120->65929215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.040206:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983404 is committed 00002000:00010000:0.0:1789760787.040321:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65925120 -> 65929216 00002000:00010000:0.0:1789760787.040326:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128784 -> 128792 00010000:00010000:0.0:1789760787.040338:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c624f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65925120->65929215] (req 65925120->65929215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.040351:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c624f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65925120->65929215] (req 65925120->65929215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.040361:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c624f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65925120->65929215] (req 65925120->65929215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.040369:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760787.040374:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c624f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65925120->65929215] (req 65925120->65929215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.040413:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c624f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65925120->65929215] (req 65925120->65929215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.040431:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c624f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65925120->65929215] (req 65925120->65929215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.040438:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c624f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65925120->65929215] (req 65925120->65929215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.040531:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a384e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554826880:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10159us (10461us total) trans 4294983404 rc 0/0 00000100:00100000:0.0:1789760787.040541:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16106 00000100:00100000:0.0:1789760787.040550:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730261154322 time 730275977982 state 0x2 ref 0 00000100:00100000:0.0:1789760787.046258:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760787.051578:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554827008 00000100:00100000:1.0:1789760787.051595:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554827008 00000100:00100000:1.0:1789760787.051676:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730261154322 time 730275977982 state 0x0 ref 1 00000100:00100000:1.0:1789760787.051692:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730276238398 time 730275977982 state 0x1 ref 1 00000100:00100000:1.0:1789760787.051787:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730276238398 time 730297404943 state 0x0 ref 1 00000100:00100000:1.0:1789760787.051802:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554827008:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760787.051860:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65929216, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f718 00010000:00010000:1.0:1789760787.051869:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6256 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.051884:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760787.051889:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.051895:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6256 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65929216->65933311] (req 65929216->65933311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.051909:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6256 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65929216->65933311] (req 65929216->65933311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.058045:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983405 is committed 00002000:00010000:1.0:1789760787.058197:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65929216 -> 65933312 00002000:00010000:1.0:1789760787.058203:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128792 -> 128800 00010000:00010000:1.0:1789760787.058216:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6256 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65929216->65933311] (req 65929216->65933311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.058236:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6256 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65929216->65933311] (req 65929216->65933311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.058245:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6256 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65929216->65933311] (req 65929216->65933311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.058252:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760787.058258:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6256 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65929216->65933311] (req 65929216->65933311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.058268:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6256 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65929216->65933311] (req 65929216->65933311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.058285:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6256 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65929216->65933311] (req 65929216->65933311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.058292:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6256 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65929216->65933311] (req 65929216->65933311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.058409:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554827008:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6579us (12129us total) trans 4294983405 rc 0/0 00000100:00100000:1.0:1789760787.058419:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16107 00000100:00100000:1.0:1789760787.058428:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730276238398 time 730297404943 state 0x2 ref 0 00000100:00100000:2.0:1789760787.067951:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760787.068026:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554827136 00000100:00100000:3.0:1789760787.068035:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554827136 00000100:00100000:3.0:1789760787.068083:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730276238398 time 730297404943 state 0x0 ref 1 00000100:00100000:3.0:1789760787.068102:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730297665359 time 730297404943 state 0x1 ref 1 00000100:00100000:3.0:1789760787.068131:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730297665359 time 730313749447 state 0x0 ref 1 00000100:00100000:3.0:1789760787.068142:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a386d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554827136:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760787.068186:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65933312, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f720 00010000:00010000:3.0:1789760787.068195:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c625d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.068212:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760787.068216:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.068223:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c625d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65933312->65937407] (req 65933312->65937407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.068236:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c625d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65933312->65937407] (req 65933312->65937407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.073340:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983406 is committed 00002000:00010000:3.0:1789760787.073556:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65933312 -> 65937408 00002000:00010000:3.0:1789760787.073563:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128800 -> 128808 00010000:00010000:3.0:1789760787.073581:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c625d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65933312->65937407] (req 65933312->65937407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.073597:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c625d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65933312->65937407] (req 65933312->65937407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.073608:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c625d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65933312->65937407] (req 65933312->65937407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.073617:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760787.073624:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c625d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65933312->65937407] (req 65933312->65937407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.073634:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c625d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65933312->65937407] (req 65933312->65937407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.073669:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c625d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65933312->65937407] (req 65933312->65937407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.073677:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c625d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65933312->65937407] (req 65933312->65937407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.073866:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a386d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554827136:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5716us (5915us total) trans 4294983406 rc 0/0 00000100:00100000:3.0:1789760787.073881:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16108 00000100:00100000:3.0:1789760787.073894:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730297665359 time 730313749447 state 0x2 ref 0 00000100:00100000:2.0:1789760787.079750:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760787.083118:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554827264 00000100:00100000:0.0:1789760787.083138:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554827264 00000100:00100000:0.0:1789760787.083225:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730297665359 time 730313749447 state 0x0 ref 1 00000100:00100000:0.0:1789760787.083247:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730314009863 time 730313749447 state 0x1 ref 1 00000100:00100000:0.0:1789760787.083295:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730314009863 time 730328912115 state 0x0 ref 1 00000100:00100000:0.0:1789760787.083312:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554827264:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760787.083375:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65937408, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f728 00010000:00010000:0.0:1789760787.083539:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6264 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.083560:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760787.083565:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.083573:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6264 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65937408->65941503] (req 65937408->65941503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.083591:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6264 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65937408->65941503] (req 65937408->65941503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.088151:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983407 is committed 00002000:00010000:2.0:1789760787.088299:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65937408 -> 65941504 00002000:00010000:2.0:1789760787.088306:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128808 -> 128816 00010000:00010000:2.0:1789760787.088325:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6264 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65937408->65941503] (req 65937408->65941503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.088339:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6264 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65937408->65941503] (req 65937408->65941503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.088350:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6264 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65937408->65941503] (req 65937408->65941503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.088358:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760787.088364:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6264 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65937408->65941503] (req 65937408->65941503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.088375:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6264 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65937408->65941503] (req 65937408->65941503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.088429:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6264 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65937408->65941503] (req 65937408->65941503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.088439:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6264 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65937408->65941503] (req 65937408->65941503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.088658:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554827264:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5332us (8905us total) trans 4294983407 rc 0/0 00000100:00100000:2.0:1789760787.088672:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16109 00000100:00100000:2.0:1789760787.088683:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730314009863 time 730328912115 state 0x2 ref 0 00000100:00100000:0.0:1789760787.098494:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760787.098777:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554827392 00000100:00100000:2.0:1789760787.098792:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554827392 00000100:00100000:2.0:1789760787.098872:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730314009863 time 730328912115 state 0x0 ref 1 00000100:00100000:2.0:1789760787.098892:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730329172531 time 730328912115 state 0x1 ref 1 00000100:00100000:2.0:1789760787.098931:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730329172531 time 730344549385 state 0x0 ref 1 00000100:00100000:2.0:1789760787.098946:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554827392:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760787.099012:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65941504, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f730 00010000:00010000:2.0:1789760787.099023:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c626b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.099042:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760787.099048:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.099056:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c626b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65941504->65945599] (req 65941504->65945599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.099073:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c626b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65941504->65945599] (req 65941504->65945599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.112211:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983408 is committed 00002000:00010000:2.0:1789760787.112362:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65941504 -> 65945600 00002000:00010000:2.0:1789760787.112369:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128816 -> 128824 00010000:00010000:2.0:1789760787.112483:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c626b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65941504->65945599] (req 65941504->65945599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.112500:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c626b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65941504->65945599] (req 65941504->65945599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.112510:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c626b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65941504->65945599] (req 65941504->65945599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.112518:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760787.112523:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c626b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65941504->65945599] (req 65941504->65945599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.112534:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c626b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65941504->65945599] (req 65941504->65945599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.112560:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c626b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65941504->65945599] (req 65941504->65945599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.112569:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c626b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65941504->65945599] (req 65941504->65945599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.112807:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554827392:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13856us (14321us total) trans 4294983408 rc 0/0 00000100:00100000:2.0:1789760787.112820:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16110 00000100:00100000:2.0:1789760787.112832:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730329172531 time 730344549385 state 0x2 ref 0 00000100:00100000:0.0:1789760787.120964:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760787.121213:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554827520 00000100:00100000:1.0:1789760787.121229:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554827520 00000100:00100000:1.0:1789760787.121311:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730329172531 time 730344549385 state 0x0 ref 1 00000100:00100000:1.0:1789760787.121334:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730344809801 time 730344549385 state 0x1 ref 1 00000100:00100000:1.0:1789760787.121482:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730344809801 time 730367098555 state 0x0 ref 1 00000100:00100000:1.0:1789760787.121499:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554827520:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760787.121572:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65945600, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f738 00010000:00010000:1.0:1789760787.121585:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6272 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.121607:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760787.121613:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.121622:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6272 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65945600->65949695] (req 65945600->65949695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.121642:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6272 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65945600->65949695] (req 65945600->65949695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.131671:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983409 is committed 00002000:00010000:1.0:1789760787.131835:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65945600 -> 65949696 00002000:00010000:1.0:1789760787.131842:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128824 -> 128832 00010000:00010000:1.0:1789760787.131859:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6272 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65945600->65949695] (req 65945600->65949695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.131875:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6272 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65945600->65949695] (req 65945600->65949695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.131886:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6272 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65945600->65949695] (req 65945600->65949695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.131894:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760787.131901:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6272 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65945600->65949695] (req 65945600->65949695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.131911:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6272 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65945600->65949695] (req 65945600->65949695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.131936:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6272 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65945600->65949695] (req 65945600->65949695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.131945:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6272 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65945600->65949695] (req 65945600->65949695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.132081:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554827520:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10579us (11125us total) trans 4294983409 rc 0/0 00000100:00100000:1.0:1789760787.132094:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16111 00000100:00100000:1.0:1789760787.132106:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730344809801 time 730367098555 state 0x2 ref 0 00000100:00100000:0.0:1789760787.138669:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760787.140984:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554827648 00000100:00100000:3.0:1789760787.141008:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554827648 00000100:00100000:3.0:1789760787.141106:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730344809801 time 730367098555 state 0x0 ref 1 00000100:00100000:3.0:1789760787.141127:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730367358971 time 730367098555 state 0x1 ref 1 00000100:00100000:3.0:1789760787.141171:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730367358971 time 730386788600 state 0x0 ref 1 00000100:00100000:3.0:1789760787.141191:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554827648:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760787.141264:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65949696, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f740 00010000:00010000:3.0:1789760787.141277:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6279 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.141298:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760787.141304:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.141313:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6279 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65949696->65953791] (req 65949696->65953791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.141329:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6279 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65949696->65953791] (req 65949696->65953791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.157701:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983410 is committed 00002000:00010000:2.0:1789760787.157973:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65949696 -> 65953792 00002000:00010000:2.0:1789760787.157979:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128832 -> 128840 00010000:00010000:2.0:1789760787.157997:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6279 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65949696->65953791] (req 65949696->65953791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.158012:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6279 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65949696->65953791] (req 65949696->65953791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.158032:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6279 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65949696->65953791] (req 65949696->65953791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.158041:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760787.158047:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6279 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65949696->65953791] (req 65949696->65953791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.158056:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6279 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65949696->65953791] (req 65949696->65953791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.158081:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6279 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65949696->65953791] (req 65949696->65953791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.158088:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6279 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65949696->65953791] (req 65949696->65953791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.158263:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554827648:12345-192.168.203.22@tcp:4:dd.0 Request processed in 17069us (19609us total) trans 4294983410 rc 0/0 00000100:00100000:2.0:1789760787.158277:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16112 00000100:00100000:2.0:1789760787.158290:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730367358971 time 730386788600 state 0x2 ref 0 00000100:00100000:1.0:1789760787.168468:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760787.168661:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554827776 00000100:00100000:2.0:1789760787.168675:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554827776 00000100:00100000:2.0:1789760787.168748:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730367358971 time 730386788600 state 0x0 ref 1 00000100:00100000:2.0:1789760787.168767:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730387049016 time 730386788600 state 0x1 ref 1 00000100:00100000:2.0:1789760787.168838:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730387049016 time 730414455534 state 0x0 ref 1 00000100:00100000:2.0:1789760787.168854:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554827776:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760787.168924:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65953792, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f748 00010000:00010000:2.0:1789760787.168936:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6280 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.168957:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760787.168962:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.168969:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6280 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65953792->65957887] (req 65953792->65957887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.168986:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6280 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65953792->65957887] (req 65953792->65957887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.175910:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983411 is committed 00002000:00010000:2.0:1789760787.176435:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65953792 -> 65957888 00002000:00010000:2.0:1789760787.176442:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128840 -> 128848 00010000:00010000:2.0:1789760787.176460:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6280 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65953792->65957887] (req 65953792->65957887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.176477:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6280 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65953792->65957887] (req 65953792->65957887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.176488:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6280 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65953792->65957887] (req 65953792->65957887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.176497:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760787.176503:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6280 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65953792->65957887] (req 65953792->65957887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.176515:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6280 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65953792->65957887] (req 65953792->65957887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.176540:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6280 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65953792->65957887] (req 65953792->65957887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.176548:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6280 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65953792->65957887] (req 65953792->65957887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.176797:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554827776:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7938us (8337us total) trans 4294983411 rc 0/0 00000100:00100000:2.0:1789760787.176812:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16113 00000100:00100000:2.0:1789760787.176824:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730387049016 time 730414455534 state 0x2 ref 0 00000100:00100000:0.0:1789760787.183814:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760787.183926:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554827904 00000100:00100000:2.0:1789760787.183948:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554827904 00000100:00100000:2.0:1789760787.184022:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730387049016 time 730414455534 state 0x0 ref 1 00000100:00100000:2.0:1789760787.184042:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730414715950 time 730414455534 state 0x1 ref 1 00000100:00100000:2.0:1789760787.184137:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730414715950 time 730429754933 state 0x0 ref 1 00000100:00100000:2.0:1789760787.184152:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554827904:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760787.184217:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65957888, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f750 00010000:00010000:2.0:1789760787.184227:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6287 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.184248:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760787.184253:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.184260:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6287 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65957888->65961983] (req 65957888->65961983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.184276:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6287 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65957888->65961983] (req 65957888->65961983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.198459:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983412 is committed 00002000:00010000:2.0:1789760787.199447:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65957888 -> 65961984 00002000:00010000:2.0:1789760787.199457:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128848 -> 128856 00010000:00010000:2.0:1789760787.199476:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6287 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65957888->65961983] (req 65957888->65961983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.199491:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6287 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65957888->65961983] (req 65957888->65961983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.199500:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6287 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65957888->65961983] (req 65957888->65961983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.199508:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760787.199514:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6287 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65957888->65961983] (req 65957888->65961983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.199525:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6287 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65957888->65961983] (req 65957888->65961983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.199551:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6287 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65957888->65961983] (req 65957888->65961983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.199626:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6287 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65957888->65961983] (req 65957888->65961983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.199812:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554827904:12345-192.168.203.22@tcp:4:dd.0 Request processed in 15613us (15964us total) trans 4294983412 rc 0/0 00000100:00100000:2.0:1789760787.199826:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16114 00000100:00100000:2.0:1789760787.199839:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730414715950 time 730429754933 state 0x2 ref 0 00000100:00100000:0.0:1789760787.205737:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760787.206503:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554828032 00000100:00100000:1.0:1789760787.206591:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554828032 00000100:00100000:1.0:1789760787.206688:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730414715950 time 730429754933 state 0x0 ref 1 00000100:00100000:1.0:1789760787.206709:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730430015349 time 730429754933 state 0x1 ref 1 00000100:00100000:1.0:1789760787.206789:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730430015349 time 730452404976 state 0x0 ref 1 00000100:00100000:1.0:1789760787.206806:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554828032:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760787.206882:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65961984, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f758 00010000:00010000:1.0:1789760787.206897:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c628e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.206919:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760787.206925:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.206943:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c628e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65961984->65966079] (req 65961984->65966079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.206961:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c628e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65961984->65966079] (req 65961984->65966079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.213883:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983413 is committed 00002000:00010000:1.0:1789760787.214065:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65961984 -> 65966080 00002000:00010000:1.0:1789760787.214071:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128856 -> 128864 00010000:00010000:1.0:1789760787.214090:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c628e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65961984->65966079] (req 65961984->65966079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.214105:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c628e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65961984->65966079] (req 65961984->65966079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.214121:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c628e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65961984->65966079] (req 65961984->65966079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.214129:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760787.214135:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c628e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65961984->65966079] (req 65961984->65966079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.214145:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c628e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65961984->65966079] (req 65961984->65966079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.214172:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c628e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65961984->65966079] (req 65961984->65966079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.214181:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c628e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65961984->65966079] (req 65961984->65966079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.214350:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554828032:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7539us (8619us total) trans 4294983413 rc 0/0 00000100:00100000:1.0:1789760787.214362:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16115 00000100:00100000:1.0:1789760787.214374:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730430015349 time 730452404976 state 0x2 ref 0 00000100:00100000:1.0:1789760787.224235:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760787.224695:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554828160 00000100:00100000:2.0:1789760787.224710:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554828160 00000100:00100000:2.0:1789760787.224781:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730430015349 time 730452404976 state 0x0 ref 1 00000100:00100000:2.0:1789760787.224801:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730452665392 time 730452404976 state 0x1 ref 1 00000100:00100000:2.0:1789760787.224838:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730452665392 time 730470455694 state 0x0 ref 1 00000100:00100000:2.0:1789760787.224851:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554828160:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760787.224916:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65966080, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f760 00010000:00010000:2.0:1789760787.224927:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6295 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.224947:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760787.224952:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.224960:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6295 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65966080->65970175] (req 65966080->65970175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.224976:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6295 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65966080->65970175] (req 65966080->65970175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.231168:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983414 is committed 00002000:00010000:3.0:1789760787.231309:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65966080 -> 65970176 00002000:00010000:3.0:1789760787.231316:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128864 -> 128872 00010000:00010000:3.0:1789760787.231336:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6295 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65966080->65970175] (req 65966080->65970175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.231352:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6295 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65966080->65970175] (req 65966080->65970175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.231362:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6295 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65966080->65970175] (req 65966080->65970175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.231370:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760787.232036:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6295 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65966080->65970175] (req 65966080->65970175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.232055:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6295 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65966080->65970175] (req 65966080->65970175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.232082:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6295 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65966080->65970175] (req 65966080->65970175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.232091:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6295 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65966080->65970175] (req 65966080->65970175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.232227:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554828160:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7371us (8003us total) trans 4294983414 rc 0/0 00000100:00100000:3.0:1789760787.232242:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16116 00000100:00100000:3.0:1789760787.232253:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730452665392 time 730470455694 state 0x2 ref 0 00000100:00100000:0.0:1789760787.240837:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760787.241172:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554828288 00000100:00100000:2.0:1789760787.241187:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554828288 00000100:00100000:2.0:1789760787.241260:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730452665392 time 730470455694 state 0x0 ref 1 00000100:00100000:2.0:1789760787.241279:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730470716110 time 730470455694 state 0x1 ref 1 00000100:00100000:2.0:1789760787.241322:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730470716110 time 730486939955 state 0x0 ref 1 00000100:00100000:2.0:1789760787.241338:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554828288:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760787.241432:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65970176, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f768 00010000:00010000:2.0:1789760787.241445:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c629c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.241465:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760787.241471:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.241479:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c629c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65970176->65974271] (req 65970176->65974271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.241495:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c629c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65970176->65974271] (req 65970176->65974271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.248566:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983415 is committed 00002000:00010000:2.0:1789760787.248730:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65970176 -> 65974272 00002000:00010000:2.0:1789760787.248738:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128872 -> 128880 00010000:00010000:2.0:1789760787.248756:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c629c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65970176->65974271] (req 65970176->65974271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.248773:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c629c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65970176->65974271] (req 65970176->65974271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.248785:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c629c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65970176->65974271] (req 65970176->65974271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.248801:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760787.248808:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c629c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65970176->65974271] (req 65970176->65974271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.248820:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c629c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65970176->65974271] (req 65970176->65974271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.248848:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c629c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65970176->65974271] (req 65970176->65974271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.248858:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c629c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65970176->65974271] (req 65970176->65974271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.249002:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554828288:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7653us (8167us total) trans 4294983415 rc 0/0 00000100:00100000:2.0:1789760787.249016:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16117 00000100:00100000:2.0:1789760787.249037:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730470716110 time 730486939955 state 0x2 ref 0 00000100:00100000:0.0:1789760787.253607:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760787.256190:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554828416 00000100:00100000:2.0:1789760787.256213:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554828416 00000100:00100000:2.0:1789760787.256304:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730470716110 time 730486939955 state 0x0 ref 1 00000100:00100000:2.0:1789760787.256324:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730487200371 time 730486939955 state 0x1 ref 1 00000100:00100000:2.0:1789760787.256470:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730487200371 time 730502086093 state 0x0 ref 1 00000100:00100000:2.0:1789760787.256489:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554828416:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760787.256559:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65974272, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f770 00010000:00010000:2.0:1789760787.256571:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c62a3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.256591:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760787.256596:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.256605:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c62a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65974272->65978367] (req 65974272->65978367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.256624:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c62a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65974272->65978367] (req 65974272->65978367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.266828:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983416 is committed 00002000:00010000:2.0:1789760787.267585:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65974272 -> 65978368 00002000:00010000:2.0:1789760787.267593:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128880 -> 128888 00010000:00010000:2.0:1789760787.267611:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c62a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65974272->65978367] (req 65974272->65978367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.267629:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c62a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65974272->65978367] (req 65974272->65978367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.267639:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c62a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65974272->65978367] (req 65974272->65978367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.267658:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760787.267666:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c62a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65974272->65978367] (req 65974272->65978367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.267676:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c62a3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65974272->65978367] (req 65974272->65978367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.267701:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c62a3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65974272->65978367] (req 65974272->65978367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.267711:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c62a3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65974272->65978367] (req 65974272->65978367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.267899:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554828416:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11405us (14300us total) trans 4294983416 rc 0/0 00000100:00100000:2.0:1789760787.267912:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16118 00000100:00100000:2.0:1789760787.267924:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730487200371 time 730502086093 state 0x2 ref 0 00000100:00100000:1.0:1789760787.272630:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760787.272759:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554828544 00000100:00100000:2.0:1789760787.272772:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554828544 00000100:00100000:2.0:1789760787.272851:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730487200371 time 730502086093 state 0x0 ref 1 00000100:00100000:2.0:1789760787.272869:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730502346509 time 730502086093 state 0x1 ref 1 00000100:00100000:2.0:1789760787.272963:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730502346509 time 730518579768 state 0x0 ref 1 00000100:00100000:2.0:1789760787.272979:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554828544:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760787.273061:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65978368, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f778 00010000:00010000:2.0:1789760787.273075:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62aa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.273096:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760787.273101:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.273109:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65978368->65982463] (req 65978368->65982463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.273126:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65978368->65982463] (req 65978368->65982463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.280680:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983417 is committed 00002000:00010000:2.0:1789760787.280832:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65978368 -> 65982464 00002000:00010000:2.0:1789760787.280837:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128888 -> 128896 00010000:00010000:2.0:1789760787.280853:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65978368->65982463] (req 65978368->65982463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.280864:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62aa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65978368->65982463] (req 65978368->65982463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.280872:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65978368->65982463] (req 65978368->65982463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.280877:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760787.280881:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65978368->65982463] (req 65978368->65982463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.280888:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62aa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65978368->65982463] (req 65978368->65982463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.280914:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62aa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65978368->65982463] (req 65978368->65982463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.280923:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62aa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65978368->65982463] (req 65978368->65982463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.281072:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554828544:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8087us (8447us total) trans 4294983417 rc 0/0 00000100:00100000:2.0:1789760787.281084:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16119 00000100:00100000:2.0:1789760787.281100:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730502346509 time 730518579768 state 0x2 ref 0 00000100:00100000:1.0:1789760787.292203:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760787.292353:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554828672 00000100:00100000:3.0:1789760787.292367:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554828672 00000100:00100000:3.0:1789760787.292484:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730502346509 time 730518579768 state 0x0 ref 1 00000100:00100000:3.0:1789760787.292505:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730518840184 time 730518579768 state 0x1 ref 1 00000100:00100000:3.0:1789760787.292545:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730518840184 time 730538163494 state 0x0 ref 1 00000100:00100000:3.0:1789760787.292559:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554828672:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760787.292629:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65982464, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f780 00010000:00010000:3.0:1789760787.292650:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62b1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.292670:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760787.292675:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.292682:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65982464->65986559] (req 65982464->65986559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.292698:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65982464->65986559] (req 65982464->65986559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.298834:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983418 is committed 00002000:00010000:3.0:1789760787.298977:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65982464 -> 65986560 00002000:00010000:3.0:1789760787.298983:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128896 -> 128904 00010000:00010000:3.0:1789760787.299001:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65982464->65986559] (req 65982464->65986559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.299027:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62b1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65982464->65986559] (req 65982464->65986559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.299039:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65982464->65986559] (req 65982464->65986559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.299048:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760787.299053:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65982464->65986559] (req 65982464->65986559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.299071:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62b1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65982464->65986559] (req 65982464->65986559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.299098:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62b1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65982464->65986559] (req 65982464->65986559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.299107:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62b1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65982464->65986559] (req 65982464->65986559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.299272:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554828672:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6708us (7076us total) trans 4294983418 rc 0/0 00000100:00100000:3.0:1789760787.299286:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16120 00000100:00100000:3.0:1789760787.299298:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730518840184 time 730538163494 state 0x2 ref 0 00000100:00100000:0.0:1789760787.322171:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760787.322287:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554828800 00000100:00100000:2.0:1789760787.322299:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554828800 00000100:00100000:2.0:1789760787.322374:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730518840184 time 730538163494 state 0x0 ref 1 00000100:00100000:2.0:1789760787.322443:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730538423910 time 730538163494 state 0x1 ref 1 00000100:00100000:2.0:1789760787.322480:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730538423910 time 730568098638 state 0x0 ref 1 00000100:00100000:2.0:1789760787.322494:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554828800:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760787.322560:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65986560, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f788 00010000:00010000:2.0:1789760787.322571:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62b8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.322591:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760787.322597:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.322605:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65986560->65990655] (req 65986560->65990655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.322621:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65986560->65990655] (req 65986560->65990655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.329262:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983419 is committed 00002000:00010000:2.0:1789760787.329481:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65986560 -> 65990656 00002000:00010000:2.0:1789760787.329487:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128904 -> 128912 00010000:00010000:2.0:1789760787.329504:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65986560->65990655] (req 65986560->65990655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.329520:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62b8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65986560->65990655] (req 65986560->65990655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.329530:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65986560->65990655] (req 65986560->65990655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.329538:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760787.329544:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65986560->65990655] (req 65986560->65990655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.329554:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65986560->65990655] (req 65986560->65990655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.329579:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62b8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65986560->65990655] (req 65986560->65990655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.329593:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62b8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65986560->65990655] (req 65986560->65990655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.329813:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554828800:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7313us (7648us total) trans 4294983419 rc 0/0 00000100:00100000:2.0:1789760787.329827:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16121 00000100:00100000:2.0:1789760787.329840:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730538423910 time 730568098638 state 0x2 ref 0 00000100:00100000:1.0:1789760787.343969:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760787.344160:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554828928 00000100:00100000:2.0:1789760787.344174:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554828928 00000100:00100000:2.0:1789760787.344929:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730538423910 time 730568098638 state 0x0 ref 1 00000100:00100000:2.0:1789760787.344955:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730568359054 time 730568098638 state 0x1 ref 1 00000100:00100000:2.0:1789760787.345566:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730568359054 time 730591180288 state 0x0 ref 1 00000100:00100000:2.0:1789760787.345594:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554828928:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760787.345677:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65990656, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f790 00010000:00010000:2.0:1789760787.345689:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62bf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.345710:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760787.345715:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.345722:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65990656->65994751] (req 65990656->65994751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.345744:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65990656->65994751] (req 65990656->65994751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.347099:0:3647:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cc40fb4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3647:1876699573918976:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789760787.349014:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb046fb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573919360:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.349154:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.349349:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb046f9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3646:1876699573919872:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.349423:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.349454:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03e92a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3646:1876699573920384:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.349478:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.349588:0:3646:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:3.0:1789760787.349595:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb046fa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3646:1876699573919488:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.349623:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.349640:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03e92d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573920000:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.349677:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.349698:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03e93800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573920512:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.349721:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.349752:0:3649:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb046f9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3649:1876699573919232:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.349777:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.349799:0:3649:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb046f9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3649:1876699573919744:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.349817:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.349833:0:3649:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03e91180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3649:1876699573920256:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.349854:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.349867:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 4 async RPCs [2->3] 00000100:00100000:3.0:1789760787.349874:0:3649:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb046f8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:5711db52-8702-4d6b-b42c-8543e9bf1cb2:3649:1876699573919104:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.349906:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.349930:0:3649:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb046fa300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3649:1876699573919616:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.349953:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.349971:0:3649:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03e90000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3649:1876699573920128:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.349994:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.350011:0:3649:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03e91c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3649:1876699573920640:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.350033:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760787.350067:0:7831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573919360 00000100:00100000:3.0:1789760787.350076:0:7831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573919360 00000100:00100000:3.0:1789760787.350142:0:7831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573919616 00000100:00100000:3.0:1789760787.350145:0:7831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573919616 00000100:00100000:3.0:1789760787.350164:0:7831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 390 00000100:00100000:3.0:1789760787.350172:0:7831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3646:x1876699573919360:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.350254:0:7831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3646:x1876699573919360:12345-0@lo:400:kworker.0 Request processed in 73us (1091us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760787.350263:0:7831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 390 00000100:00100000:3.0:1789760787.350284:0:7831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 391 00000100:00100000:3.0:1789760787.350290:0:7831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb020b2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3649:x1876699573919616:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.350326:0:7831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb020b2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3649:x1876699573919616:12345-0@lo:400:kworker.0 Request processed in 37us (372us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760787.350333:0:7831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 391 00000100:00100000:3.0:1789760787.350352:0:6522:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573919872 00000100:00100000:3.0:1789760787.350356:0:6522:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573919872 00000100:00100000:3.0:1789760787.350409:0:6522:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573920384 00000100:00100000:3.0:1789760787.350412:0:6522:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573920384 00000100:00100000:3.0:1789760787.350426:0:6522:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573919488 00000100:00100000:2.0:1789760787.351074:0:6520:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573919232 00000100:00100000:2.0:1789760787.351091:0:6520:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573919232 00000100:00100000:2.0:1789760787.351117:0:6520:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573919744 00000100:00100000:2.0:1789760787.351120:0:6520:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573919744 00000100:00100000:2.0:1789760787.351133:0:6520:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573920256 00000100:00100000:2.0:1789760787.351135:0:6520:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573920256 00000100:00100000:2.0:1789760787.351151:0:6520:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 889 00000100:00100000:2.0:1789760787.351160:0:6520:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb020b1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3646:x1876699573919872:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760787.351219:0:6520:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb020b1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3646:x1876699573919872:12345-0@lo:400:kworker.0 Request processed in 60us (1796us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760787.351226:0:6520:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 889 00000100:00100000:2.0:1789760787.351240:0:6520:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 890 00000100:00100000:2.0:1789760787.351259:0:6520:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb020b1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3646:x1876699573920384:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760787.351289:0:6520:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb020b1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3646:x1876699573920384:12345-0@lo:400:kworker.0 Request processed in 31us (1810us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760787.351295:0:6520:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 890 00000100:00100000:2.0:1789760787.351307:0:6520:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 891 00000100:00100000:2.0:1789760787.351314:0:6520:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb020b0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3649:x1876699573919232:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760787.351342:0:6520:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb020b0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3649:x1876699573919232:12345-0@lo:400:kworker.0 Request processed in 28us (1564us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760787.351347:0:6520:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 891 00000100:00100000:2.0:1789760787.351361:0:6520:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 892 00000100:00100000:2.0:1789760787.351367:0:6520:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb020b1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3649:x1876699573919744:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760787.351618:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760787.351715:0:6520:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb020b1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3649:x1876699573919744:12345-0@lo:400:kworker.0 Request processed in 344us (1892us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760787.351724:0:6520:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 892 00000100:00100000:0.0:1789760787.351737:0:3648:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789760787.351748:0:6520:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 893 00000100:00100000:0.0:1789760787.351759:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb09858000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3648:1876699573920768:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789760787.351760:0:6520:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb020b2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1876699573920256:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760787.351824:0:6520:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb020b2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3649:x1876699573920256:12345-0@lo:400:kworker.0 Request processed in 67us (1967us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760787.351832:0:6520:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 893 00000100:00100000:0.0:1789760787.351843:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760787.351864:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573918976 00000100:00100000:2.0:1789760787.351873:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573918976 00000100:00100000:0.0:1789760787.351898:0:11526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573920768 00000100:00100000:2.0:1789760787.351907:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 392 00000100:00100000:0.0:1789760787.351908:0:11526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573920768 00000100:00100000:2.0:1789760787.351921:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3647:x1876699573918976:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1789760787.351956:0:6525:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446227968 avail 1299988480 00000100:00100000:0.0:1789760787.351969:0:11526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 366 00000100:00100000:0.0:1789760787.351984:0:11526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3648:x1876699573920768:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789760787.352017:0:11526:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:2.0:1789760787.352019:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3647:x1876699573918976:12345-0@lo:41:osp-pre-0-1.0 Request processed in 99us (405us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760787.352029:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 392 00000100:00100000:0.0:1789760787.352080:0:11526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3648:x1876699573920768:12345-0@lo:13:osp-pre-1-0.0 Request processed in 98us (238us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760787.352089:0:11526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 366 00000100:00100000:2.0:1789760787.352146:0:3647:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cc40fb4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3647:1876699573918976:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789760787.352160:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb09858000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3648:1876699573920768:0@lo:13:osp-pre-1-0.0 00000100:00080000:0.0:1789760787.352682:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789760787.352690:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789760787.352693:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789760787.352730:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb046fb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573919360:0@lo:400:kworker.0 00000100:00100000:0.0:1789760787.352746:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb046f9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3646:1876699573919872:0@lo:400:kworker.0 00000100:00100000:0.0:1789760787.352759:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03e92a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3646:1876699573920384:0@lo:400:kworker.0 00000100:00100000:0.0:1789760787.352795:0:8438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573920000 00000100:00100000:0.0:1789760787.352800:0:8438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573920000 00000100:00100000:0.0:1789760787.352844:0:8438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573920512 00000100:00100000:0.0:1789760787.352847:0:8438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573920512 00000100:00100000:0.0:1789760787.352863:0:8438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573920128 00000100:00100000:0.0:1789760787.352866:0:8438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573920128 00000100:00100000:0.0:1789760787.352880:0:8438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573920640 00000100:00100000:0.0:1789760787.352883:0:8438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573920640 00000100:00100000:0.0:1789760787.352897:0:8438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 299 00000100:00100000:0.0:1789760787.352904:0:8438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb020b2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3646:x1876699573920000:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760787.352965:0:8438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb020b2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3646:x1876699573920000:12345-0@lo:400:kworker.0 Request processed in 61us (3287us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760787.352971:0:8438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 299 00000100:00100000:0.0:1789760787.352986:0:8438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 300 00000100:00100000:0.0:1789760787.352992:0:8438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb020b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3646:x1876699573920512:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760787.353029:0:8438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb020b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3646:x1876699573920512:12345-0@lo:400:kworker.0 Request processed in 37us (3307us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760787.353034:0:8438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 300 00000100:00100000:0.0:1789760787.353047:0:8438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 301 00000100:00100000:0.0:1789760787.353053:0:8438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb020b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3649:x1876699573920128:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760787.353091:0:8438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb020b1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3649:x1876699573920128:12345-0@lo:400:kworker.0 Request processed in 38us (3096us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760787.353096:0:8438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 301 00000100:00100000:0.0:1789760787.353107:0:8438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 302 00000100:00100000:0.0:1789760787.353113:0:8438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb020b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3649:x1876699573920640:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760787.353138:0:8438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb020b0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3649:x1876699573920640:12345-0@lo:400:kworker.0 Request processed in 27us (3105us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760787.353148:0:8438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 302 00000100:00100000:0.0:1789760787.353177:0:6514:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573919104 00000100:00100000:0.0:1789760787.353181:0:6514:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573919104 00000100:00100000:0.0:1789760787.353206:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 473 00000100:00100000:0.0:1789760787.353212:0:6514:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc320b5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5711db52-8702-4d6b-b42c-8543e9bf1cb2+24:3649:x1876699573919104:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760787.353242:0:6514:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc320b5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:5711db52-8702-4d6b-b42c-8543e9bf1cb2+24:3649:x1876699573919104:12345-0@lo:400:kworker.0 Request processed in 30us (3335us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760787.353261:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 473 00000100:00080000:0.0:1789760787.353307:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789760787.353309:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760787.353329:0:3649:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb046f8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:5711db52-8702-4d6b-b42c-8543e9bf1cb2:3649:1876699573919104:0@lo:400:kworker.0 00000100:00100000:0.0:1789760787.353344:0:3649:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb046fa300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3649:1876699573919616:0@lo:400:kworker.0 00000100:00100000:0.0:1789760787.353355:0:3649:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03e90000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3649:1876699573920128:0@lo:400:kworker.0 00000100:00100000:0.0:1789760787.353366:0:3649:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03e91c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3649:1876699573920640:0@lo:400:kworker.0 00000100:00100000:0.0:1789760787.353401:0:3649:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb046f9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3649:1876699573919232:0@lo:400:kworker.0 00000100:00100000:0.0:1789760787.353415:0:3649:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb046f9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3649:1876699573919744:0@lo:400:kworker.0 00000100:00100000:0.0:1789760787.353427:0:3649:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03e91180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3649:1876699573920256:0@lo:400:kworker.0 00000100:00100000:0.0:1789760787.353874:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb098ddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573920896:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1789760787.353920:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760787.354003:0:8439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573920896 00000100:00100000:1.0:1789760787.354009:0:8439:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573920896 00000100:00100000:1.0:1789760787.354044:0:8439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 367 00000100:00100000:1.0:1789760787.354052:0:8439:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3646:x1876699573920896:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1789760787.354070:0:8439:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:1.0:1789760787.354113:0:8439:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3646:x1876699573920896:12345-0@lo:13:osp-pre-1-1.0 Request processed in 62us (193us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760787.354121:0:8439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 367 00000100:00100000:1.0:1789760787.354459:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb098ddc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573920896:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789760787.354480:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03e92d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573920000:0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.354734:0:6522:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573919488 00000100:00100000:3.0:1789760787.354784:0:6522:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 894 00000100:00100000:3.0:1789760787.354792:0:6522:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb020b3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3646:x1876699573919488:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760787.354844:0:6522:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb020b3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3646:x1876699573919488:12345-0@lo:400:kworker.0 Request processed in 53us (5221us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760787.354851:0:6522:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 894 00000100:00100000:1.0:1789760787.354892:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03e93800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573920512:0@lo:400:kworker.0 00000100:00100000:1.0:1789760787.354933:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb046fa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3646:1876699573919488:0@lo:400:kworker.0 00000001:00080000:2.0:1789760787.358207:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983420 is committed 00002000:00010000:0.0:1789760787.358940:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65990656 -> 65994752 00002000:00010000:0.0:1789760787.358948:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128912 -> 128920 00010000:00010000:0.0:1789760787.358965:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65990656->65994751] (req 65990656->65994751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.358982:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62bf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65990656->65994751] (req 65990656->65994751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.358991:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65990656->65994751] (req 65990656->65994751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.358999:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760787.359004:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65990656->65994751] (req 65990656->65994751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.359026:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62bf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65990656->65994751] (req 65990656->65994751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.359052:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62bf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65990656->65994751] (req 65990656->65994751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.359060:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c62bf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65990656->65994751] (req 65990656->65994751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.359661:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554828928:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14062us (15692us total) trans 4294983420 rc 0/0 00000100:00100000:0.0:1789760787.359675:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16122 00000100:00100000:0.0:1789760787.359687:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730568359054 time 730591180288 state 0x2 ref 0 00000100:00100000:1.0:1789760787.363976:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760787.368063:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554829056 00000100:00100000:2.0:1789760787.368089:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554829056 00000100:00100000:2.0:1789760787.368247:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730568359054 time 730591180288 state 0x0 ref 1 00000100:00100000:2.0:1789760787.368267:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730591440704 time 730591180288 state 0x1 ref 1 00000100:00100000:2.0:1789760787.368364:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730591440704 time 730613981157 state 0x0 ref 1 00000100:00100000:2.0:1789760787.368399:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554829056:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760787.368467:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65994752, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f798 00010000:00010000:2.0:1789760787.368479:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c62c6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.368500:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760787.368505:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.368513:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c62c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65994752->65998847] (req 65994752->65998847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.368528:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c62c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65994752->65998847] (req 65994752->65998847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.375989:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983421 is committed 00002000:00010000:1.0:1789760787.377760:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65994752 -> 65998848 00002000:00010000:1.0:1789760787.377772:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128920 -> 128928 00010000:00010000:1.0:1789760787.377790:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c62c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65994752->65998847] (req 65994752->65998847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.377806:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c62c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65994752->65998847] (req 65994752->65998847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.377816:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c62c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65994752->65998847] (req 65994752->65998847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.377824:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760787.377832:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c62c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65994752->65998847] (req 65994752->65998847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.377842:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c62c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65994752->65998847] (req 65994752->65998847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.377869:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c62c6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65994752->65998847] (req 65994752->65998847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.377878:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c62c6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65994752->65998847] (req 65994752->65998847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.378056:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554829056:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9670us (14087us total) trans 4294983421 rc 0/0 00000100:00100000:1.0:1789760787.378070:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16123 00000100:00100000:1.0:1789760787.378147:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730591440704 time 730613981157 state 0x2 ref 0 00000100:00100000:1.0:1789760787.392568:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760787.392793:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554829184 00000100:00100000:3.0:1789760787.392807:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554829184 00000100:00100000:3.0:1789760787.392886:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730591440704 time 730613981157 state 0x0 ref 1 00000100:00100000:3.0:1789760787.392905:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730614241573 time 730613981157 state 0x1 ref 1 00000100:00100000:3.0:1789760787.392942:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730614241573 time 730638560225 state 0x0 ref 1 00000100:00100000:3.0:1789760787.392958:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554829184:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760787.393022:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 65998848, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f7a0 00010000:00010000:3.0:1789760787.393034:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62cd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.393054:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760787.393059:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.393067:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65998848->66002943] (req 65998848->66002943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.393097:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65998848->66002943] (req 65998848->66002943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.401183:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983422 is committed 00002000:00010000:3.0:1789760787.401617:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 65998848 -> 66002944 00002000:00010000:3.0:1789760787.401624:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128928 -> 128936 00010000:00010000:3.0:1789760787.401653:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65998848->66002943] (req 65998848->66002943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.401668:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65998848->66002943] (req 65998848->66002943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.401679:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65998848->66002943] (req 65998848->66002943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.401688:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760787.401694:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65998848->66002943] (req 65998848->66002943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.401705:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62cd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65998848->66002943] (req 65998848->66002943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.401733:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62cd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65998848->66002943] (req 65998848->66002943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.401741:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62cd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [65998848->66002943] (req 65998848->66002943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.401923:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554829184:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8961us (9362us total) trans 4294983422 rc 0/0 00000100:00100000:3.0:1789760787.401937:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16124 00000100:00100000:3.0:1789760787.401950:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730614241573 time 730638560225 state 0x2 ref 0 00000100:00100000:1.0:1789760787.407908:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760787.408041:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554829312 00000100:00100000:0.0:1789760787.408055:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554829312 00000100:00100000:0.0:1789760787.408139:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730614241573 time 730638560225 state 0x0 ref 1 00000100:00100000:0.0:1789760787.408158:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730638820641 time 730638560225 state 0x1 ref 1 00000100:00100000:0.0:1789760787.408193:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730638820641 time 730653810894 state 0x0 ref 1 00000100:00100000:0.0:1789760787.408207:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554829312:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760787.408280:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66002944, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f7a8 00010000:00010000:0.0:1789760787.408290:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62d4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.408310:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760787.408315:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.408323:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66002944->66007039] (req 66002944->66007039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.408340:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66002944->66007039] (req 66002944->66007039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.415195:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983423 is committed 00002000:00010000:0.0:1789760787.415335:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66002944 -> 66007040 00002000:00010000:0.0:1789760787.415341:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128936 -> 128944 00010000:00010000:0.0:1789760787.415356:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66002944->66007039] (req 66002944->66007039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.415368:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66002944->66007039] (req 66002944->66007039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.415422:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66002944->66007039] (req 66002944->66007039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.415430:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760787.415435:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66002944->66007039] (req 66002944->66007039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.415444:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62d4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66002944->66007039] (req 66002944->66007039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.415464:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62d4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66002944->66007039] (req 66002944->66007039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.415472:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62d4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66002944->66007039] (req 66002944->66007039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.415554:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554829312:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7346us (7656us total) trans 4294983423 rc 0/0 00000100:00100000:0.0:1789760787.415564:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16125 00000100:00100000:0.0:1789760787.415573:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730638820641 time 730653810894 state 0x2 ref 0 00000100:00100000:1.0:1789760787.424635:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760787.425194:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554829440 00000100:00100000:0.0:1789760787.425206:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554829440 00000100:00100000:0.0:1789760787.425261:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730638820641 time 730653810894 state 0x0 ref 1 00000100:00100000:0.0:1789760787.425278:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730654071310 time 730653810894 state 0x1 ref 1 00000100:00100000:0.0:1789760787.425466:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730654071310 time 730671082736 state 0x0 ref 1 00000100:00100000:0.0:1789760787.425481:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554829440:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760787.425528:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66007040, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f7b0 00010000:00010000:0.0:1789760787.425537:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62db lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.425552:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760787.425556:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.425562:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66007040->66011135] (req 66007040->66011135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.425579:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66007040->66011135] (req 66007040->66011135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.432584:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983424 is committed 00002000:00010000:0.0:1789760787.434168:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66007040 -> 66011136 00002000:00010000:0.0:1789760787.434176:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128944 -> 128952 00010000:00010000:0.0:1789760787.434189:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66007040->66011135] (req 66007040->66011135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.434203:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66007040->66011135] (req 66007040->66011135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.434212:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66007040->66011135] (req 66007040->66011135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.434226:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760787.434231:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66007040->66011135] (req 66007040->66011135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.434241:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62db lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66007040->66011135] (req 66007040->66011135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.434261:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62db lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66007040->66011135] (req 66007040->66011135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.434270:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62db lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66007040->66011135] (req 66007040->66011135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.434591:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554829440:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9108us (9960us total) trans 4294983424 rc 0/0 00000100:00100000:0.0:1789760787.434604:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16126 00000100:00100000:0.0:1789760787.434614:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730654071310 time 730671082736 state 0x2 ref 0 00000100:00100000:1.0:1789760787.444139:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760787.444511:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554829568 00000100:00100000:2.0:1789760787.444522:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554829568 00000100:00100000:2.0:1789760787.444577:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730654071310 time 730671082736 state 0x0 ref 1 00000100:00100000:2.0:1789760787.444593:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730671343152 time 730671082736 state 0x1 ref 1 00000100:00100000:2.0:1789760787.444624:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730671343152 time 730690241830 state 0x0 ref 1 00000100:00100000:2.0:1789760787.444637:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554829568:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760787.444814:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66011136, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f7b8 00010000:00010000:2.0:1789760787.444825:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c62e2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.444841:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760787.444845:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.444852:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c62e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66011136->66015231] (req 66011136->66015231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.444866:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c62e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66011136->66015231] (req 66011136->66015231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.453955:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983425 is committed 00002000:00010000:2.0:1789760787.454411:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66011136 -> 66015232 00002000:00010000:2.0:1789760787.454416:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128952 -> 128960 00010000:00010000:2.0:1789760787.454430:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c62e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66011136->66015231] (req 66011136->66015231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.454442:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c62e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66011136->66015231] (req 66011136->66015231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.454451:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c62e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66011136->66015231] (req 66011136->66015231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.454459:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760787.454464:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c62e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66011136->66015231] (req 66011136->66015231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.454473:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c62e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66011136->66015231] (req 66011136->66015231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.454493:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c62e2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66011136->66015231] (req 66011136->66015231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.454501:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c62e2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66011136->66015231] (req 66011136->66015231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.454637:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554829568:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9997us (10500us total) trans 4294983425 rc 0/0 00000100:00100000:2.0:1789760787.454660:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16127 00000100:00100000:2.0:1789760787.454669:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730671343152 time 730690241830 state 0x2 ref 0 00000100:00100000:1.0:1789760787.466674:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760787.466940:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554829696 00000100:00100000:3.0:1789760787.466952:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554829696 00000100:00100000:3.0:1789760787.467077:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730671343152 time 730690241830 state 0x0 ref 1 00000100:00100000:3.0:1789760787.467150:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730690502246 time 730690241830 state 0x1 ref 1 00000100:00100000:3.0:1789760787.467297:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730690502246 time 730712913737 state 0x0 ref 1 00000100:00100000:3.0:1789760787.467312:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554829696:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760787.467371:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66015232, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f7c0 00010000:00010000:3.0:1789760787.467427:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62e9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.467453:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:3.0:1789760787.467458:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.467471:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66015232->66019327] (req 66015232->66019327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.467488:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66015232->66019327] (req 66015232->66019327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.474255:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983426 is committed 00002000:00010000:2.0:1789760787.474479:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66015232 -> 66019328 00002000:00010000:2.0:1789760787.474669:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128960 -> 128968 00010000:00010000:2.0:1789760787.474688:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66015232->66019327] (req 66015232->66019327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.474704:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66015232->66019327] (req 66015232->66019327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.474714:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66015232->66019327] (req 66015232->66019327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.474722:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760787.474728:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66015232->66019327] (req 66015232->66019327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.474743:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66015232->66019327] (req 66015232->66019327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.474768:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62e9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66015232->66019327] (req 66015232->66019327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.474776:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c62e9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66015232->66019327] (req 66015232->66019327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.474903:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554829696:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7586us (8234us total) trans 4294983426 rc 0/0 00000100:00100000:2.0:1789760787.474925:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16128 00000100:00100000:2.0:1789760787.474943:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730690502246 time 730712913737 state 0x2 ref 0 00000100:00100000:1.0:1789760787.478883:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760787.479183:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554829824 00000100:00100000:0.0:1789760787.479222:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554829824 00000100:00100000:0.0:1789760787.479297:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730690502246 time 730712913737 state 0x0 ref 1 00000100:00100000:0.0:1789760787.479316:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730713174153 time 730712913737 state 0x1 ref 1 00000100:00100000:0.0:1789760787.479349:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730713174153 time 730724967455 state 0x0 ref 1 00000100:00100000:0.0:1789760787.479364:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554829824:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760787.479466:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66019328, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f7c8 00010000:00010000:0.0:1789760787.479478:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62f0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.479497:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760787.479502:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.479510:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66019328->66023423] (req 66019328->66023423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.479526:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66019328->66023423] (req 66019328->66023423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.485842:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983427 is committed 00002000:00010000:0.0:1789760787.486013:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66019328 -> 66023424 00002000:00010000:0.0:1789760787.486020:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128968 -> 128976 00010000:00010000:0.0:1789760787.486039:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66019328->66023423] (req 66019328->66023423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.486056:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62f0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66019328->66023423] (req 66019328->66023423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.486067:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66019328->66023423] (req 66019328->66023423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.486079:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760787.486084:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66019328->66023423] (req 66019328->66023423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.486135:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62f0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66019328->66023423] (req 66019328->66023423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.486158:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62f0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66019328->66023423] (req 66019328->66023423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.486166:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c62f0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66019328->66023423] (req 66019328->66023423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.486323:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554829824:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6954us (7447us total) trans 4294983427 rc 0/0 00000100:00100000:0.0:1789760787.486336:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16129 00000100:00100000:0.0:1789760787.486348:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730713174153 time 730724967455 state 0x2 ref 0 00000100:00100000:1.0:1789760787.492092:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760787.492174:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554829952 00000100:00100000:0.0:1789760787.492188:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554829952 00000100:00100000:0.0:1789760787.492269:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730713174153 time 730724967455 state 0x0 ref 1 00000100:00100000:0.0:1789760787.492288:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730725227871 time 730724967455 state 0x1 ref 1 00000100:00100000:0.0:1789760787.492326:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730725227871 time 730737944120 state 0x0 ref 1 00000100:00100000:0.0:1789760787.492354:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554829952:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760787.492457:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66023424, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f7d0 00010000:00010000:0.0:1789760787.492469:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c62f7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.492488:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760787.492493:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.492500:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c62f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66023424->66027519] (req 66023424->66027519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.492527:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c62f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66023424->66027519] (req 66023424->66027519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.500460:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983428 is committed 00002000:00010000:1.0:1789760787.506167:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66023424 -> 66027520 00002000:00010000:1.0:1789760787.506180:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128976 -> 128984 00010000:00010000:1.0:1789760787.506197:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c62f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66023424->66027519] (req 66023424->66027519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.506213:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c62f7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66023424->66027519] (req 66023424->66027519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.506222:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c62f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66023424->66027519] (req 66023424->66027519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.506230:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760787.506235:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c62f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66023424->66027519] (req 66023424->66027519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.506245:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c62f7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66023424->66027519] (req 66023424->66027519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.506267:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c62f7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66023424->66027519] (req 66023424->66027519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.506274:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c62f7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66023424->66027519] (req 66023424->66027519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.506445:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554829952:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14097us (14359us total) trans 4294983428 rc 0/0 00000100:00100000:1.0:1789760787.506457:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16130 00000100:00100000:1.0:1789760787.506468:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730725227871 time 730737944120 state 0x2 ref 0 00000100:00100000:1.0:1789760787.511034:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760787.511946:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554830080 00000100:00100000:2.0:1789760787.511969:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554830080 00000100:00100000:2.0:1789760787.512043:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730725227871 time 730737944120 state 0x0 ref 1 00000100:00100000:2.0:1789760787.512061:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730738204536 time 730737944120 state 0x1 ref 1 00000100:00100000:2.0:1789760787.512097:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730738204536 time 730757716132 state 0x0 ref 1 00000100:00100000:2.0:1789760787.512109:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554830080:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760787.512162:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66027520, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f7d8 00010000:00010000:2.0:1789760787.512171:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c62fe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.512187:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760787.512191:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.512196:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c62fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66027520->66031615] (req 66027520->66031615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.512210:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c62fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66027520->66031615] (req 66027520->66031615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.520148:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983429 is committed 00002000:00010000:2.0:1789760787.520864:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66027520 -> 66031616 00002000:00010000:2.0:1789760787.520872:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128984 -> 128992 00010000:00010000:2.0:1789760787.520891:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c62fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66027520->66031615] (req 66027520->66031615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.520906:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c62fe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66027520->66031615] (req 66027520->66031615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.520916:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c62fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66027520->66031615] (req 66027520->66031615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.520925:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760787.520930:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c62fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66027520->66031615] (req 66027520->66031615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.520940:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c62fe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66027520->66031615] (req 66027520->66031615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.520974:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c62fe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66027520->66031615] (req 66027520->66031615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.521047:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c62fe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66027520->66031615] (req 66027520->66031615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.521248:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554830080:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9131us (10222us total) trans 4294983429 rc 0/0 00000100:00100000:2.0:1789760787.521262:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16131 00000100:00100000:2.0:1789760787.521274:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730738204536 time 730757716132 state 0x2 ref 0 00000100:00100000:2.0:1789760787.532841:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760787.532916:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554830208 00000100:00100000:1.0:1789760787.532930:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554830208 00000100:00100000:1.0:1789760787.533011:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730738204536 time 730757716132 state 0x0 ref 1 00000100:00100000:1.0:1789760787.533030:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730757976548 time 730757716132 state 0x1 ref 1 00000100:00100000:1.0:1789760787.533068:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730757976548 time 730778685677 state 0x0 ref 1 00000100:00100000:1.0:1789760787.533081:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09859500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554830208:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760787.533146:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66031616, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f7e0 00010000:00010000:1.0:1789760787.533163:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6305 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.533184:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760787.533190:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.533196:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6305 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66031616->66035711] (req 66031616->66035711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.533211:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6305 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66031616->66035711] (req 66031616->66035711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.541132:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983430 is committed 00002000:00010000:1.0:1789760787.541309:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66031616 -> 66035712 00002000:00010000:1.0:1789760787.541315:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 128992 -> 129000 00010000:00010000:1.0:1789760787.541333:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6305 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66031616->66035711] (req 66031616->66035711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.541347:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6305 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66031616->66035711] (req 66031616->66035711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.541357:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6305 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66031616->66035711] (req 66031616->66035711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.541366:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760787.541372:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6305 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66031616->66035711] (req 66031616->66035711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.541493:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6305 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66031616->66035711] (req 66031616->66035711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.541519:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6305 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66031616->66035711] (req 66031616->66035711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.541526:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6305 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66031616->66035711] (req 66031616->66035711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.541879:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09859500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554830208:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8792us (9045us total) trans 4294983430 rc 0/0 00000100:00100000:1.0:1789760787.541890:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16132 00000100:00100000:1.0:1789760787.541901:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730757976548 time 730778685677 state 0x2 ref 0 00000100:00100000:0.0:1789760787.549337:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760787.549696:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554830336 00000100:00100000:2.0:1789760787.549709:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554830336 00000100:00100000:2.0:1789760787.549784:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730757976548 time 730778685677 state 0x0 ref 1 00000100:00100000:2.0:1789760787.549804:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730778946093 time 730778685677 state 0x1 ref 1 00000100:00100000:2.0:1789760787.549841:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730778946093 time 730795458444 state 0x0 ref 1 00000100:00100000:2.0:1789760787.549855:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554830336:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760787.549920:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66035712, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f7e8 00010000:00010000:2.0:1789760787.549931:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c630c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.550098:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760787.550105:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.550113:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c630c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66035712->66039807] (req 66035712->66039807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.550131:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c630c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66035712->66039807] (req 66035712->66039807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.558488:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983431 is committed 00002000:00010000:2.0:1789760787.558737:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66035712 -> 66039808 00002000:00010000:2.0:1789760787.558743:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129000 -> 129008 00010000:00010000:2.0:1789760787.558761:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c630c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66035712->66039807] (req 66035712->66039807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.558776:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c630c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66035712->66039807] (req 66035712->66039807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.558786:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c630c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66035712->66039807] (req 66035712->66039807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.558794:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760787.558800:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c630c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66035712->66039807] (req 66035712->66039807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.558809:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c630c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66035712->66039807] (req 66035712->66039807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.558838:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c630c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66035712->66039807] (req 66035712->66039807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.558847:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c630c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66035712->66039807] (req 66035712->66039807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.558972:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554830336:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9113us (9644us total) trans 4294983431 rc 0/0 00000100:00100000:2.0:1789760787.558984:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16133 00000100:00100000:2.0:1789760787.559673:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730778946093 time 730795458444 state 0x2 ref 0 00000100:00100000:2.0:1789760787.563828:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760787.564206:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554830464 00000100:00100000:1.0:1789760787.564220:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554830464 00000100:00100000:1.0:1789760787.564294:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730778946093 time 730795458444 state 0x0 ref 1 00000100:00100000:1.0:1789760787.564311:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730795718860 time 730795458444 state 0x1 ref 1 00000100:00100000:1.0:1789760787.564410:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730795718860 time 730810027079 state 0x0 ref 1 00000100:00100000:1.0:1789760787.564432:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09858e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554830464:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760787.564494:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66039808, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f7f0 00010000:00010000:1.0:1789760787.564505:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6313 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.564524:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760787.564529:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.564541:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6313 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66039808->66043903] (req 66039808->66043903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.564659:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6313 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66039808->66043903] (req 66039808->66043903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.570504:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983432 is committed 00002000:00010000:1.0:1789760787.570658:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66039808 -> 66043904 00002000:00010000:1.0:1789760787.570664:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129008 -> 129016 00010000:00010000:1.0:1789760787.570681:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6313 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66039808->66043903] (req 66039808->66043903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.570695:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6313 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66039808->66043903] (req 66039808->66043903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.570704:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6313 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66039808->66043903] (req 66039808->66043903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.570718:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760787.570723:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6313 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66039808->66043903] (req 66039808->66043903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.570732:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6313 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66039808->66043903] (req 66039808->66043903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.570755:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6313 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66039808->66043903] (req 66039808->66043903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.570763:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6313 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66039808->66043903] (req 66039808->66043903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.570883:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09858e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554830464:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6447us (7062us total) trans 4294983432 rc 0/0 00000100:00100000:1.0:1789760787.570896:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16134 00000100:00100000:1.0:1789760787.570907:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730795718860 time 730810027079 state 0x2 ref 0 00000100:00100000:1.0:1789760787.574478:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760787.575689:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554830592 00000100:00100000:2.0:1789760787.575705:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554830592 00000100:00100000:2.0:1789760787.575787:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730795718860 time 730810027079 state 0x0 ref 1 00000100:00100000:2.0:1789760787.575808:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730810287495 time 730810027079 state 0x1 ref 1 00000100:00100000:2.0:1789760787.575852:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730810287495 time 730821470236 state 0x0 ref 1 00000100:00100000:2.0:1789760787.575866:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554830592:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760787.575933:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66043904, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f7f8 00010000:00010000:2.0:1789760787.577160:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c631a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.577193:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760787.577198:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.577205:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c631a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66043904->66047999] (req 66043904->66047999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.577222:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c631a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66043904->66047999] (req 66043904->66047999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.584436:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983433 is committed 00002000:00010000:2.0:1789760787.584586:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66043904 -> 66048000 00002000:00010000:2.0:1789760787.584593:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129016 -> 129024 00010000:00010000:2.0:1789760787.584609:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c631a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66043904->66047999] (req 66043904->66047999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.584624:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c631a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66043904->66047999] (req 66043904->66047999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.584635:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c631a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66043904->66047999] (req 66043904->66047999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.584652:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760787.584658:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c631a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66043904->66047999] (req 66043904->66047999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.584669:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c631a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66043904->66047999] (req 66043904->66047999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.584700:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c631a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66043904->66047999] (req 66043904->66047999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.584708:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c631a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66043904->66047999] (req 66043904->66047999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.584837:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554830592:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8967us (10367us total) trans 4294983433 rc 0/0 00000100:00100000:2.0:1789760787.584849:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16135 00000100:00100000:2.0:1789760787.584861:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730810287495 time 730821470236 state 0x2 ref 0 00000100:00100000:0.0:1789760787.588515:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760787.588681:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554830720 00000100:00100000:2.0:1789760787.588694:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554830720 00000100:00100000:2.0:1789760787.588762:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730810287495 time 730821470236 state 0x0 ref 1 00000100:00100000:2.0:1789760787.588777:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730821730652 time 730821470236 state 0x1 ref 1 00000100:00100000:2.0:1789760787.588804:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730821730652 time 730834423889 state 0x0 ref 1 00000100:00100000:2.0:1789760787.588815:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554830720:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760787.588866:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66048000, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f800 00010000:00010000:2.0:1789760787.588874:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6321 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.588890:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760787.588895:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.588901:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6321 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66048000->66052095] (req 66048000->66052095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.588913:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6321 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66048000->66052095] (req 66048000->66052095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.595803:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983434 is committed 00002000:00010000:2.0:1789760787.595981:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66048000 -> 66052096 00002000:00010000:2.0:1789760787.595987:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129024 -> 129032 00010000:00010000:2.0:1789760787.596004:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6321 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66048000->66052095] (req 66048000->66052095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.596020:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6321 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66048000->66052095] (req 66048000->66052095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.596030:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6321 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66048000->66052095] (req 66048000->66052095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.596047:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760787.596053:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6321 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66048000->66052095] (req 66048000->66052095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.596064:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6321 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66048000->66052095] (req 66048000->66052095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.596089:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6321 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66048000->66052095] (req 66048000->66052095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.596104:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6321 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66048000->66052095] (req 66048000->66052095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.596241:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554830720:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7419us (7730us total) trans 4294983434 rc 0/0 00000100:00100000:2.0:1789760787.596259:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16136 00000100:00100000:2.0:1789760787.596272:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730821730652 time 730834423889 state 0x2 ref 0 00000100:00100000:1.0:1789760787.607200:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760787.607298:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554830848 00000100:00100000:0.0:1789760787.607312:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554830848 00000100:00100000:0.0:1789760787.607417:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730821730652 time 730834423889 state 0x0 ref 1 00000100:00100000:0.0:1789760787.607435:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730834684305 time 730834423889 state 0x1 ref 1 00000100:00100000:0.0:1789760787.607471:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730834684305 time 730853089395 state 0x0 ref 1 00000100:00100000:0.0:1789760787.607485:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554830848:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760787.607547:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66052096, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f808 00010000:00010000:0.0:1789760787.607559:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6328 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.607578:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760787.607584:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.607592:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6328 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66052096->66056191] (req 66052096->66056191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.607608:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6328 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66052096->66056191] (req 66052096->66056191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.615850:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983435 is committed 00002000:00010000:0.0:1789760787.616199:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66052096 -> 66056192 00002000:00010000:0.0:1789760787.616206:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129032 -> 129040 00010000:00010000:0.0:1789760787.616223:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6328 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66052096->66056191] (req 66052096->66056191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.616238:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6328 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66052096->66056191] (req 66052096->66056191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.616247:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6328 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66052096->66056191] (req 66052096->66056191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.616256:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760787.616262:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6328 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66052096->66056191] (req 66052096->66056191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.616278:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6328 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66052096->66056191] (req 66052096->66056191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.616303:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6328 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66052096->66056191] (req 66052096->66056191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.616312:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6328 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66052096->66056191] (req 66052096->66056191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.616653:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554830848:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9155us (9452us total) trans 4294983435 rc 0/0 00000100:00100000:0.0:1789760787.616667:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16137 00000100:00100000:0.0:1789760787.616679:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730834684305 time 730853089395 state 0x2 ref 0 00000100:00100000:2.0:1789760787.619523:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760787.619725:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554830976 00000100:00100000:1.0:1789760787.619737:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554830976 00000100:00100000:1.0:1789760787.619806:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730834684305 time 730853089395 state 0x0 ref 1 00000100:00100000:1.0:1789760787.619823:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730853349811 time 730853089395 state 0x1 ref 1 00000100:00100000:1.0:1789760787.619861:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730853349811 time 730865479413 state 0x0 ref 1 00000100:00100000:1.0:1789760787.619875:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0985b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554830976:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760787.619938:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66056192, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f810 00010000:00010000:1.0:1789760787.619949:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c632f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.619969:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760787.619974:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.619982:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c632f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66056192->66060287] (req 66056192->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.619998:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c632f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66056192->66060287] (req 66056192->66060287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.628972:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983436 is committed 00002000:00010000:1.0:1789760787.633429:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66056192 -> 66060288 00002000:00010000:1.0:1789760787.633444:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129040 -> 129048 00010000:00010000:1.0:1789760787.633462:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c632f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66056192->66060287] (req 66056192->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.633477:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c632f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66056192->66060287] (req 66056192->66060287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.633487:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c632f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66056192->66060287] (req 66056192->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.633501:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760787.633569:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c632f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66056192->66060287] (req 66056192->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.633581:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c632f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66056192->66060287] (req 66056192->66060287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.633606:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c632f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66056192->66060287] (req 66056192->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.633615:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c632f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66056192->66060287] (req 66056192->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.633938:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0985b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554830976:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14057us (14429us total) trans 4294983436 rc 0/0 00000100:00100000:1.0:1789760787.633953:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16138 00000100:00100000:1.0:1789760787.633966:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730853349811 time 730865479413 state 0x2 ref 0 00000100:00100000:2.0:1789760787.641787:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760787.641914:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554831104 00000100:00100000:1.0:1789760787.641927:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554831104 00000100:00100000:1.0:1789760787.642079:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730853349811 time 730865479413 state 0x0 ref 1 00000100:00100000:1.0:1789760787.642101:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730865739829 time 730865479413 state 0x1 ref 1 00000100:00100000:1.0:1789760787.642139:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730865739829 time 730887756479 state 0x0 ref 1 00000100:00100000:1.0:1789760787.642159:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0985ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554831104:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760787.642277:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66060288, mtime: 0x6aad9512, atime: 0x0, ctime: 0x6aad9512, blocks: 0x1f818 00010000:00010000:1.0:1789760787.642289:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6336 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.642310:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760787.642315:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.642323:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6336 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66060288->66064383] (req 66060288->66064383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.642340:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6336 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66060288->66064383] (req 66060288->66064383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.649773:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983437 is committed 00002000:00010000:1.0:1789760787.650032:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66060288 -> 66064384 00002000:00010000:1.0:1789760787.650037:0:15101:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760786 -> 1789760787 00002000:00010000:1.0:1789760787.650040:0:15101:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760786 -> 1789760787 00002000:00010000:1.0:1789760787.650043:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129048 -> 129056 00010000:00010000:1.0:1789760787.650060:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6336 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66060288->66064383] (req 66060288->66064383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.650075:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6336 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66060288->66064383] (req 66060288->66064383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.650086:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6336 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66060288->66064383] (req 66060288->66064383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.650100:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760787.650106:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6336 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66060288->66064383] (req 66060288->66064383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.650117:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6336 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66060288->66064383] (req 66060288->66064383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.650140:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6336 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66060288->66064383] (req 66060288->66064383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.650148:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6336 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66060288->66064383] (req 66060288->66064383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.650330:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0985ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554831104:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8168us (8551us total) trans 4294983437 rc 0/0 00000100:00100000:1.0:1789760787.650345:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16139 00000100:00100000:1.0:1789760787.650358:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730865739829 time 730887756479 state 0x2 ref 0 00000100:00100000:2.0:1789760787.654136:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760787.654236:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554831232 00000100:00100000:1.0:1789760787.654250:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554831232 00000100:00100000:1.0:1789760787.654326:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730865739829 time 730887756479 state 0x0 ref 1 00000100:00100000:1.0:1789760787.654345:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730888016895 time 730887756479 state 0x1 ref 1 00000100:00100000:1.0:1789760787.654427:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730888016895 time 730900044349 state 0x0 ref 1 00000100:00100000:1.0:1789760787.654445:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe1b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554831232:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760787.654510:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66064384, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f820 00010000:00010000:1.0:1789760787.654521:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c633d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.654541:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760787.654546:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.654554:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c633d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66064384->66068479] (req 66064384->66068479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.654571:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c633d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66064384->66068479] (req 66064384->66068479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.660196:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983438 is committed 00002000:00010000:1.0:1789760787.660812:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66064384 -> 66068480 00002000:00010000:1.0:1789760787.660820:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129056 -> 129064 00010000:00010000:1.0:1789760787.660838:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c633d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66064384->66068479] (req 66064384->66068479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.660853:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c633d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66064384->66068479] (req 66064384->66068479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.660863:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c633d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66064384->66068479] (req 66064384->66068479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.660870:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760787.660877:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c633d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66064384->66068479] (req 66064384->66068479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.660888:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c633d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66064384->66068479] (req 66064384->66068479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.660913:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c633d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66064384->66068479] (req 66064384->66068479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.660921:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c633d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66064384->66068479] (req 66064384->66068479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.662047:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe1b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554831232:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7589us (7911us total) trans 4294983438 rc 0/0 00000100:00100000:1.0:1789760787.662063:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16140 00000100:00100000:1.0:1789760787.662075:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730888016895 time 730900044349 state 0x2 ref 0 00000100:00100000:2.0:1789760787.673226:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760787.675252:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554831360 00000100:00100000:0.0:1789760787.675271:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554831360 00000100:00100000:0.0:1789760787.675360:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730888016895 time 730900044349 state 0x0 ref 1 00000100:00100000:0.0:1789760787.675397:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730900304765 time 730900044349 state 0x1 ref 1 00000100:00100000:0.0:1789760787.675438:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730900304765 time 730921056115 state 0x0 ref 1 00000100:00100000:0.0:1789760787.675453:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09858700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554831360:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760787.675923:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66068480, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f828 00010000:00010000:0.0:1789760787.675937:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6344 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.675958:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:0.0:1789760787.675963:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.675971:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6344 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66068480->66072575] (req 66068480->66072575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.675989:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6344 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66068480->66072575] (req 66068480->66072575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.682149:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983439 is committed 00002000:00010000:0.0:1789760787.682307:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66068480 -> 66072576 00002000:00010000:0.0:1789760787.682314:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129064 -> 129072 00010000:00010000:0.0:1789760787.682333:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6344 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66068480->66072575] (req 66068480->66072575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.682350:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6344 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66068480->66072575] (req 66068480->66072575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.682359:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6344 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66068480->66072575] (req 66068480->66072575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.682367:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760787.682373:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6344 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66068480->66072575] (req 66068480->66072575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.682744:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6344 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66068480->66072575] (req 66068480->66072575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.682771:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6344 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66068480->66072575] (req 66068480->66072575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.682779:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6344 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66068480->66072575] (req 66068480->66072575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.682962:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09858700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554831360:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7504us (9756us total) trans 4294983439 rc 0/0 00000100:00100000:0.0:1789760787.682976:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16141 00000100:00100000:0.0:1789760787.682987:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730900304765 time 730921056115 state 0x2 ref 0 00000100:00100000:2.0:1789760787.688705:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760787.688816:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554831488 00000100:00100000:1.0:1789760787.688829:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554831488 00000100:00100000:1.0:1789760787.688899:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730900304765 time 730921056115 state 0x0 ref 1 00000100:00100000:1.0:1789760787.688917:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730921316531 time 730921056115 state 0x1 ref 1 00000100:00100000:1.0:1789760787.688953:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730921316531 time 730934572018 state 0x0 ref 1 00000100:00100000:1.0:1789760787.688966:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0985bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554831488:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760787.689039:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66072576, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f830 00010000:00010000:1.0:1789760787.689051:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c634b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.689070:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760787.689075:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.689082:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c634b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66072576->66076671] (req 66072576->66076671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.689098:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c634b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66072576->66076671] (req 66072576->66076671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.695741:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983440 is committed 00002000:00010000:1.0:1789760787.695902:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66072576 -> 66076672 00002000:00010000:1.0:1789760787.695908:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129072 -> 129080 00010000:00010000:1.0:1789760787.695924:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c634b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66072576->66076671] (req 66072576->66076671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.695938:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c634b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66072576->66076671] (req 66072576->66076671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.697953:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c634b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66072576->66076671] (req 66072576->66076671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.697968:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760787.697973:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c634b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66072576->66076671] (req 66072576->66076671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.697985:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c634b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66072576->66076671] (req 66072576->66076671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.698025:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c634b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66072576->66076671] (req 66072576->66076671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.698034:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c634b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66072576->66076671] (req 66072576->66076671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.698172:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0985bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554831488:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9201us (9577us total) trans 4294983440 rc 0/0 00000100:00100000:1.0:1789760787.698185:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16142 00000100:00100000:1.0:1789760787.698197:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730921316531 time 730934572018 state 0x2 ref 0 00000100:00100000:3.0:1789760787.703280:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760787.703523:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554831616 00000100:00100000:1.0:1789760787.703537:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554831616 00000100:00100000:1.0:1789760787.703608:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730921316531 time 730934572018 state 0x0 ref 1 00000100:00100000:1.0:1789760787.703627:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730934832434 time 730934572018 state 0x1 ref 1 00000100:00100000:1.0:1789760787.703663:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730934832434 time 730949280315 state 0x0 ref 1 00000100:00100000:1.0:1789760787.703676:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb020b0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554831616:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760787.703739:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66076672, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f838 00010000:00010000:1.0:1789760787.703750:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6352 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.703769:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760787.703775:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.703782:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6352 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66076672->66080767] (req 66076672->66080767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.703798:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6352 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66076672->66080767] (req 66076672->66080767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.708453:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983441 is committed 00002000:00010000:2.0:1789760787.708759:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66076672 -> 66080768 00002000:00010000:2.0:1789760787.708766:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129080 -> 129088 00010000:00010000:2.0:1789760787.708783:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6352 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66076672->66080767] (req 66076672->66080767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.708799:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6352 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66076672->66080767] (req 66076672->66080767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.708808:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6352 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66076672->66080767] (req 66076672->66080767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.708823:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760787.708829:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6352 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66076672->66080767] (req 66076672->66080767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.708838:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6352 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66076672->66080767] (req 66076672->66080767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.708862:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6352 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66076672->66080767] (req 66076672->66080767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.708871:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6352 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66076672->66080767] (req 66076672->66080767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.709235:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb020b0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554831616:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5552us (5960us total) trans 4294983441 rc 0/0 00000100:00100000:2.0:1789760787.709249:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16143 00000100:00100000:2.0:1789760787.709261:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730934832434 time 730949280315 state 0x2 ref 0 00000100:00100000:2.0:1789760787.717997:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760787.718813:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554831744 00000100:00100000:1.0:1789760787.718827:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554831744 00000100:00100000:1.0:1789760787.718902:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730934832434 time 730949280315 state 0x0 ref 1 00000100:00100000:1.0:1789760787.718922:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730949540731 time 730949280315 state 0x1 ref 1 00000100:00100000:1.0:1789760787.718956:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730949540731 time 730964573673 state 0x0 ref 1 00000100:00100000:1.0:1789760787.718970:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0985b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554831744:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760787.719039:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66080768, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f840 00010000:00010000:1.0:1789760787.719050:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6359 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.719069:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:1.0:1789760787.719074:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.719082:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6359 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66080768->66084863] (req 66080768->66084863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.719098:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6359 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66080768->66084863] (req 66080768->66084863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.724590:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983442 is committed 00002000:00010000:1.0:1789760787.727354:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66080768 -> 66084864 00002000:00010000:1.0:1789760787.727367:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129088 -> 129096 00010000:00010000:1.0:1789760787.727466:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6359 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66080768->66084863] (req 66080768->66084863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.727480:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6359 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66080768->66084863] (req 66080768->66084863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.727489:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6359 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66080768->66084863] (req 66080768->66084863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.727496:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760787.727502:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6359 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66080768->66084863] (req 66080768->66084863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.727523:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6359 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66080768->66084863] (req 66080768->66084863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.727549:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6359 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66080768->66084863] (req 66080768->66084863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.727557:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6359 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66080768->66084863] (req 66080768->66084863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.727864:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0985b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554831744:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8888us (9876us total) trans 4294983442 rc 0/0 00000100:00100000:1.0:1789760787.727879:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16144 00000100:00100000:1.0:1789760787.727891:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730949540731 time 730964573673 state 0x2 ref 0 00000100:00100000:0.0:1789760787.732452:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760787.732607:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554831872 00000100:00100000:2.0:1789760787.732619:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554831872 00000100:00100000:2.0:1789760787.732703:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730949540731 time 730964573673 state 0x0 ref 1 00000100:00100000:2.0:1789760787.732727:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730964834089 time 730964573673 state 0x1 ref 1 00000100:00100000:2.0:1789760787.732768:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730964834089 time 730978385857 state 0x0 ref 1 00000100:00100000:2.0:1789760787.732785:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554831872:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760787.732846:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66084864, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f848 00010000:00010000:2.0:1789760787.732858:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6360 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.732878:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760787.732884:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.732892:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6360 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66084864->66088959] (req 66084864->66088959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.732921:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6360 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66084864->66088959] (req 66084864->66088959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.738081:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983443 is committed 00002000:00010000:2.0:1789760787.741000:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66084864 -> 66088960 00002000:00010000:2.0:1789760787.741013:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129096 -> 129104 00010000:00010000:2.0:1789760787.741101:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6360 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66084864->66088959] (req 66084864->66088959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.741117:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6360 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66084864->66088959] (req 66084864->66088959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.741127:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6360 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66084864->66088959] (req 66084864->66088959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.741135:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760787.741141:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6360 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66084864->66088959] (req 66084864->66088959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.741151:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6360 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66084864->66088959] (req 66084864->66088959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.741175:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6360 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66084864->66088959] (req 66084864->66088959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.741182:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6360 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66084864->66088959] (req 66084864->66088959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.741323:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554831872:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8533us (8876us total) trans 4294983443 rc 0/0 00000100:00100000:2.0:1789760787.741338:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16145 00000100:00100000:2.0:1789760787.741351:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730964834089 time 730978385857 state 0x2 ref 0 00000100:00100000:0.0:1789760787.751187:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760787.751264:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554832000 00000100:00100000:2.0:1789760787.751282:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554832000 00000100:00100000:2.0:1789760787.751362:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730964834089 time 730978385857 state 0x0 ref 1 00000100:00100000:2.0:1789760787.751432:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730978646273 time 730978385857 state 0x1 ref 1 00000100:00100000:2.0:1789760787.751475:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730978646273 time 730997092845 state 0x0 ref 1 00000100:00100000:2.0:1789760787.751490:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554832000:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760787.751567:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66088960, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f850 00010000:00010000:2.0:1789760787.751580:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6367 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.751601:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=730s v=(0 0) 00010000:00010000:2.0:1789760787.751606:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.751614:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6367 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66088960->66093055] (req 66088960->66093055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.751631:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6367 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66088960->66093055] (req 66088960->66093055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.755893:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983444 is committed 00002000:00010000:2.0:1789760787.756027:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66088960 -> 66093056 00002000:00010000:2.0:1789760787.756033:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129104 -> 129112 00010000:00010000:2.0:1789760787.756051:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6367 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66088960->66093055] (req 66088960->66093055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.756066:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6367 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66088960->66093055] (req 66088960->66093055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.756077:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6367 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66088960->66093055] (req 66088960->66093055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.756086:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760787.756091:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6367 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66088960->66093055] (req 66088960->66093055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.756102:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6367 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66088960->66093055] (req 66088960->66093055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.756133:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6367 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66088960->66093055] (req 66088960->66093055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.756141:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6367 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66088960->66093055] (req 66088960->66093055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.756264:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554832000:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4770us (5080us total) trans 4294983444 rc 0/0 00000100:00100000:2.0:1789760787.756277:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16146 00000100:00100000:2.0:1789760787.756290:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730978646273 time 730997092845 state 0x2 ref 0 00000100:00100000:1.0:1789760787.759088:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760787.759152:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554832128 00000100:00100000:0.0:1789760787.759164:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554832128 00000100:00100000:0.0:1789760787.759242:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730978646273 time 730997092845 state 0x0 ref 1 00000100:00100000:0.0:1789760787.759261:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730997353261 time 730997092845 state 0x1 ref 1 00000100:00100000:0.0:1789760787.759302:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730997353261 time 731004920868 state 0x0 ref 1 00000100:00100000:0.0:1789760787.759315:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554832128:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760787.759770:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66093056, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f858 00010000:00010000:0.0:1789760787.759785:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c636e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.759804:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760787.759809:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.759818:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c636e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66093056->66097151] (req 66093056->66097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.759834:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c636e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66093056->66097151] (req 66093056->66097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.764733:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983445 is committed 00002000:00010000:0.0:1789760787.764887:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66093056 -> 66097152 00002000:00010000:0.0:1789760787.764893:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129112 -> 129120 00010000:00010000:0.0:1789760787.764911:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c636e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66093056->66097151] (req 66093056->66097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.764925:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c636e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66093056->66097151] (req 66093056->66097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.764936:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c636e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66093056->66097151] (req 66093056->66097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.764950:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760787.764955:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c636e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66093056->66097151] (req 66093056->66097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.764965:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c636e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66093056->66097151] (req 66093056->66097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.764991:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c636e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66093056->66097151] (req 66093056->66097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.765000:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c636e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66093056->66097151] (req 66093056->66097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.765126:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554832128:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5805us (6041us total) trans 4294983445 rc 0/0 00000100:00100000:0.0:1789760787.765138:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16147 00000100:00100000:0.0:1789760787.765150:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730997353261 time 731004920868 state 0x2 ref 0 00000100:00100000:0.0:1789760787.768542:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760787.768651:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554832256 00000100:00100000:1.0:1789760787.768663:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554832256 00000100:00100000:1.0:1789760787.768744:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 730997353261 time 731004920868 state 0x0 ref 1 00000100:00100000:1.0:1789760787.768762:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731005181284 time 731004920868 state 0x1 ref 1 00000100:00100000:1.0:1789760787.768799:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731005181284 time 731014416885 state 0x0 ref 1 00000100:00100000:1.0:1789760787.768811:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554832256:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760787.768871:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66097152, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f860 00010000:00010000:1.0:1789760787.768882:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6375 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.768902:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760787.768907:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.768915:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6375 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66097152->66101247] (req 66097152->66101247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.768932:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6375 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66097152->66101247] (req 66097152->66101247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.780790:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983446 is committed 00002000:00010000:3.0:1789760787.783063:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66097152 -> 66101248 00002000:00010000:3.0:1789760787.783075:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129120 -> 129128 00010000:00010000:3.0:1789760787.783171:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6375 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66097152->66101247] (req 66097152->66101247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.783188:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6375 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66097152->66101247] (req 66097152->66101247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.783197:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6375 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66097152->66101247] (req 66097152->66101247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.783210:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760787.783217:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6375 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66097152->66101247] (req 66097152->66101247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.783226:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6375 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66097152->66101247] (req 66097152->66101247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.783250:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6375 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66097152->66101247] (req 66097152->66101247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.783260:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6375 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66097152->66101247] (req 66097152->66101247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.783513:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554832256:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14695us (14979us total) trans 4294983446 rc 0/0 00000100:00100000:3.0:1789760787.783527:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16148 00000100:00100000:3.0:1789760787.783539:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731005181284 time 731014416885 state 0x2 ref 0 00000100:00100000:0.0:1789760787.794438:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760787.794883:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554832384 00000100:00100000:1.0:1789760787.794896:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554832384 00000100:00100000:1.0:1789760787.794973:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731005181284 time 731014416885 state 0x0 ref 1 00000100:00100000:1.0:1789760787.794992:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731014677301 time 731014416885 state 0x1 ref 1 00000100:00100000:1.0:1789760787.795298:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731014677301 time 731040913931 state 0x0 ref 1 00000100:00100000:1.0:1789760787.795315:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554832384:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760787.795586:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66101248, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f868 00010000:00010000:1.0:1789760787.795599:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c637c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.795620:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760787.795625:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.795633:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c637c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66101248->66105343] (req 66101248->66105343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.795661:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c637c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66101248->66105343] (req 66101248->66105343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.804326:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983447 is committed 00002000:00010000:2.0:1789760787.804844:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66101248 -> 66105344 00002000:00010000:2.0:1789760787.804853:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129128 -> 129136 00010000:00010000:2.0:1789760787.804871:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c637c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66101248->66105343] (req 66101248->66105343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.805027:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c637c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66101248->66105343] (req 66101248->66105343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.805038:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c637c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66101248->66105343] (req 66101248->66105343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.805044:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655de00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760787.805048:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c637c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66101248->66105343] (req 66101248->66105343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.805059:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c637c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66101248->66105343] (req 66101248->66105343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.805079:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c637c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66101248->66105343] (req 66101248->66105343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.805083:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c637c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66101248->66105343] (req 66101248->66105343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.805187:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554832384:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9871us (10813us total) trans 4294983447 rc 0/0 00000100:00100000:2.0:1789760787.805196:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16149 00000100:00100000:2.0:1789760787.805206:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731014677301 time 731040913931 state 0x2 ref 0 00000100:00100000:1.0:1789760787.811224:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760787.811634:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554832512 00000100:00100000:2.0:1789760787.811656:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554832512 00000100:00100000:2.0:1789760787.811741:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731014677301 time 731040913931 state 0x0 ref 1 00000100:00100000:2.0:1789760787.811760:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731041174347 time 731040913931 state 0x1 ref 1 00000100:00100000:2.0:1789760787.811796:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731041174347 time 731057414022 state 0x0 ref 1 00000100:00100000:2.0:1789760787.811811:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554832512:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760787.811888:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66105344, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f870 00010000:00010000:2.0:1789760787.811900:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6383 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.811921:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760787.811927:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.811934:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6383 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66105344->66109439] (req 66105344->66109439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.811951:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6383 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66105344->66109439] (req 66105344->66109439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.824447:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983448 is committed 00002000:00010000:2.0:1789760787.825856:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66105344 -> 66109440 00002000:00010000:2.0:1789760787.825879:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129136 -> 129144 00010000:00010000:2.0:1789760787.825897:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6383 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66105344->66109439] (req 66105344->66109439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.825912:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6383 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66105344->66109439] (req 66105344->66109439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.825922:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6383 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66105344->66109439] (req 66105344->66109439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.825930:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760787.825935:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6383 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66105344->66109439] (req 66105344->66109439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.825945:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6383 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66105344->66109439] (req 66105344->66109439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.825976:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6383 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66105344->66109439] (req 66105344->66109439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.825985:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6383 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66105344->66109439] (req 66105344->66109439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.826129:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554832512:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14313us (14910us total) trans 4294983448 rc 0/0 00000100:00100000:2.0:1789760787.826143:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16150 00000100:00100000:2.0:1789760787.826156:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731041174347 time 731057414022 state 0x2 ref 0 00000100:00100000:0.0:1789760787.831602:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760787.831734:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554832640 00000100:00100000:3.0:1789760787.831747:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554832640 00000100:00100000:3.0:1789760787.831833:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731041174347 time 731057414022 state 0x0 ref 1 00000100:00100000:3.0:1789760787.831853:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731057674438 time 731057414022 state 0x1 ref 1 00000100:00100000:3.0:1789760787.831896:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731057674438 time 731077514403 state 0x0 ref 1 00000100:00100000:3.0:1789760787.831915:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554832640:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760787.831978:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66109440, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f878 00010000:00010000:3.0:1789760787.831989:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c638a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.832009:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760787.832015:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.832023:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c638a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66109440->66113535] (req 66109440->66113535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.832039:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c638a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66109440->66113535] (req 66109440->66113535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.842080:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983449 is committed 00002000:00010000:3.0:1789760787.843473:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66109440 -> 66113536 00002000:00010000:3.0:1789760787.843485:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129144 -> 129152 00010000:00010000:3.0:1789760787.843499:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c638a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66109440->66113535] (req 66109440->66113535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.843512:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c638a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66109440->66113535] (req 66109440->66113535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.843521:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c638a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66109440->66113535] (req 66109440->66113535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.843529:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760787.843534:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c638a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66109440->66113535] (req 66109440->66113535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.843542:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c638a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66109440->66113535] (req 66109440->66113535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.843561:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c638a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66109440->66113535] (req 66109440->66113535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.843568:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c638a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66109440->66113535] (req 66109440->66113535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760787.843678:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554832640:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11762us (12086us total) trans 4294983449 rc 0/0 00000100:00100000:3.0:1789760787.843691:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16151 00000100:00100000:3.0:1789760787.843700:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731057674438 time 731077514403 state 0x2 ref 0 00000100:00100000:0.0:1789760787.850722:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760787.850800:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554832768 00000100:00100000:3.0:1789760787.850811:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554832768 00000100:00100000:3.0:1789760787.850862:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731057674438 time 731077514403 state 0x0 ref 1 00000100:00100000:3.0:1789760787.850879:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731077774819 time 731077514403 state 0x1 ref 1 00000100:00100000:3.0:1789760787.850912:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731077774819 time 731096529836 state 0x0 ref 1 00000100:00100000:3.0:1789760787.850924:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554832768:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760787.850975:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66113536, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f880 00010000:00010000:3.0:1789760787.850986:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6391 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.851025:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760787.851030:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.851038:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6391 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66113536->66117631] (req 66113536->66117631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.851053:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6391 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66113536->66117631] (req 66113536->66117631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.862259:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983450 is committed 00002000:00010000:2.0:1789760787.869732:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66113536 -> 66117632 00002000:00010000:2.0:1789760787.869746:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129152 -> 129160 00010000:00010000:2.0:1789760787.869762:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6391 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66113536->66117631] (req 66113536->66117631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.869778:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6391 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66113536->66117631] (req 66113536->66117631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.869788:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6391 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66113536->66117631] (req 66113536->66117631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.869797:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760787.869802:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6391 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66113536->66117631] (req 66113536->66117631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.869818:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6391 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66113536->66117631] (req 66113536->66117631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.869843:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6391 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66113536->66117631] (req 66113536->66117631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.869852:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6391 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66113536->66117631] (req 66113536->66117631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.869994:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554832768:12345-192.168.203.22@tcp:4:dd.0 Request processed in 19065us (19275us total) trans 4294983450 rc 0/0 00000100:00100000:2.0:1789760787.870007:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16152 00000100:00100000:2.0:1789760787.870018:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731077774819 time 731096529836 state 0x2 ref 0 00000100:00100000:1.0:1789760787.880400:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760787.880709:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554832896 00000100:00100000:3.0:1789760787.880720:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554832896 00000100:00100000:3.0:1789760787.880773:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731077774819 time 731096529836 state 0x0 ref 1 00000100:00100000:3.0:1789760787.880787:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731096790252 time 731096529836 state 0x1 ref 1 00000100:00100000:3.0:1789760787.880809:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731096790252 time 731126428296 state 0x0 ref 1 00000100:00100000:3.0:1789760787.880821:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554832896:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760787.880867:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66117632, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f888 00010000:00010000:3.0:1789760787.880877:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6398 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.880892:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760787.880897:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.880903:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6398 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66117632->66121727] (req 66117632->66121727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.880918:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6398 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66117632->66121727] (req 66117632->66121727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.889256:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983451 is committed 00002000:00010000:2.0:1789760787.896101:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66117632 -> 66121728 00002000:00010000:2.0:1789760787.896111:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129160 -> 129168 00010000:00010000:2.0:1789760787.896124:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6398 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66117632->66121727] (req 66117632->66121727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.896138:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6398 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66117632->66121727] (req 66117632->66121727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.896147:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6398 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66117632->66121727] (req 66117632->66121727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.896161:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760787.896166:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6398 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66117632->66121727] (req 66117632->66121727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.896175:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6398 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66117632->66121727] (req 66117632->66121727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.896193:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6398 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66117632->66121727] (req 66117632->66121727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.896201:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6398 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66117632->66121727] (req 66117632->66121727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.896299:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554832896:12345-192.168.203.22@tcp:4:dd.0 Request processed in 15474us (15919us total) trans 4294983451 rc 0/0 00000100:00100000:2.0:1789760787.896309:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16153 00000100:00100000:2.0:1789760787.896319:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731096790252 time 731126428296 state 0x2 ref 0 00000100:00100000:0.0:1789760787.899882:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760787.900310:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554833024 00000100:00100000:2.0:1789760787.900321:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554833024 00000100:00100000:2.0:1789760787.900369:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731096790252 time 731126428296 state 0x0 ref 1 00000100:00100000:2.0:1789760787.900520:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731126688712 time 731126428296 state 0x1 ref 1 00000100:00100000:2.0:1789760787.900546:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731126688712 time 731146164907 state 0x0 ref 1 00000100:00100000:2.0:1789760787.900561:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554833024:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760787.900606:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66121728, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f890 00010000:00010000:2.0:1789760787.900615:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c639f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.900630:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760787.900634:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.900795:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c639f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66121728->66125823] (req 66121728->66125823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.900815:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c639f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66121728->66125823] (req 66121728->66125823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.906676:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983452 is committed 00002000:00010000:2.0:1789760787.906820:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66121728 -> 66125824 00002000:00010000:2.0:1789760787.906827:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129168 -> 129176 00010000:00010000:2.0:1789760787.906838:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c639f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66121728->66125823] (req 66121728->66125823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.906851:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c639f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66121728->66125823] (req 66121728->66125823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.906859:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c639f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66121728->66125823] (req 66121728->66125823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.906866:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760787.906871:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c639f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66121728->66125823] (req 66121728->66125823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.906879:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c639f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66121728->66125823] (req 66121728->66125823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.906898:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c639f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66121728->66125823] (req 66121728->66125823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.906905:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c639f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66121728->66125823] (req 66121728->66125823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.907044:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554833024:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6479us (7167us total) trans 4294983452 rc 0/0 00000100:00100000:2.0:1789760787.907054:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16154 00000100:00100000:2.0:1789760787.907063:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731126688712 time 731146164907 state 0x2 ref 0 00000100:00100000:0.0:1789760787.915255:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760787.915354:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554833152 00000100:00100000:3.0:1789760787.915365:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554833152 00000100:00100000:3.0:1789760787.916659:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731126688712 time 731146164907 state 0x0 ref 1 00000100:00100000:3.0:1789760787.916678:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731146425323 time 731146164907 state 0x1 ref 1 00000100:00100000:3.0:1789760787.916707:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731146425323 time 731162325223 state 0x0 ref 1 00000100:00100000:3.0:1789760787.916720:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554833152:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760787.916766:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66125824, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f898 00010000:00010000:3.0:1789760787.916775:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c63a6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.916791:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760787.916795:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760787.916801:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c63a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66125824->66129919] (req 66125824->66129919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760787.916814:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c63a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66125824->66129919] (req 66125824->66129919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.929174:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983453 is committed 00002000:00010000:2.0:1789760787.929312:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66125824 -> 66129920 00002000:00010000:2.0:1789760787.929317:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129176 -> 129184 00010000:00010000:2.0:1789760787.929330:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c63a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66125824->66129919] (req 66125824->66129919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.929506:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c63a6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66125824->66129919] (req 66125824->66129919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.929518:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c63a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66125824->66129919] (req 66125824->66129919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.929531:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760787.929536:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c63a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66125824->66129919] (req 66125824->66129919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.929546:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c63a6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66125824->66129919] (req 66125824->66129919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.929566:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c63a6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66125824->66129919] (req 66125824->66129919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.929574:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c63a6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66125824->66129919] (req 66125824->66129919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760787.929669:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554833152:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12947us (14417us total) trans 4294983453 rc 0/0 00000100:00100000:2.0:1789760787.929679:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16155 00000100:00100000:2.0:1789760787.929689:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731146425323 time 731162325223 state 0x2 ref 0 00000100:00100000:0.0:1789760787.933271:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760787.933631:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554833280 00000100:00100000:1.0:1789760787.933652:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554833280 00000100:00100000:1.0:1789760787.933714:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731146425323 time 731162325223 state 0x0 ref 1 00000100:00100000:1.0:1789760787.933731:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731162585639 time 731162325223 state 0x1 ref 1 00000100:00100000:1.0:1789760787.933762:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731162585639 time 731179380491 state 0x0 ref 1 00000100:00100000:1.0:1789760787.933774:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554833280:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760787.933823:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66129920, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f8a0 00010000:00010000:1.0:1789760787.933833:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c63ad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.933849:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760787.933853:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.933860:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c63ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66129920->66134015] (req 66129920->66134015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.933873:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c63ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66129920->66134015] (req 66129920->66134015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760787.943569:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983454 is committed 00002000:00010000:1.0:1789760787.943694:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66129920 -> 66134016 00002000:00010000:1.0:1789760787.943700:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129184 -> 129192 00010000:00010000:1.0:1789760787.943714:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c63ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66129920->66134015] (req 66129920->66134015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.943728:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c63ad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66129920->66134015] (req 66129920->66134015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.943737:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c63ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66129920->66134015] (req 66129920->66134015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.943744:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760787.943749:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c63ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66129920->66134015] (req 66129920->66134015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.943759:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c63ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66129920->66134015] (req 66129920->66134015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.943779:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c63ad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66129920->66134015] (req 66129920->66134015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.943787:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c63ad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66129920->66134015] (req 66129920->66134015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.943888:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554833280:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10111us (10622us total) trans 4294983454 rc 0/0 00000100:00100000:1.0:1789760787.943899:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16156 00000100:00100000:1.0:1789760787.943908:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731162585639 time 731179380491 state 0x2 ref 0 00000100:00100000:0.0:1789760787.951631:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760787.951871:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554833408 00000100:00100000:1.0:1789760787.951883:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554833408 00000100:00100000:1.0:1789760787.951939:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731162585639 time 731179380491 state 0x0 ref 1 00000100:00100000:1.0:1789760787.951956:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731179640907 time 731179380491 state 0x1 ref 1 00000100:00100000:1.0:1789760787.951987:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731179640907 time 731197606096 state 0x0 ref 1 00000100:00100000:1.0:1789760787.951999:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554833408:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760787.952053:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66134016, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f8a8 00010000:00010000:1.0:1789760787.952063:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c63b4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.952080:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760787.952084:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760787.952090:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c63b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66134016->66138111] (req 66134016->66138111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.952104:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c63b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66134016->66138111] (req 66134016->66138111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760787.964179:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983455 is committed 00002000:00010000:0.0:1789760787.964421:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66134016 -> 66138112 00002000:00010000:0.0:1789760787.964426:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129192 -> 129200 00010000:00010000:0.0:1789760787.964439:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c63b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66134016->66138111] (req 66134016->66138111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.964453:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c63b4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66134016->66138111] (req 66134016->66138111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.964462:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c63b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66134016->66138111] (req 66134016->66138111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.964470:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760787.964475:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c63b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66134016->66138111] (req 66134016->66138111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.964484:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c63b4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66134016->66138111] (req 66134016->66138111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.964504:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c63b4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66134016->66138111] (req 66134016->66138111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.964511:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c63b4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66134016->66138111] (req 66134016->66138111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.964665:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554833408:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12662us (13038us total) trans 4294983455 rc 0/0 00000100:00100000:0.0:1789760787.964676:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16157 00000100:00100000:0.0:1789760787.964685:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731179640907 time 731197606096 state 0x2 ref 0 00000100:00100000:0.0:1789760787.970197:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760787.970284:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554833536 00000100:00100000:2.0:1789760787.970295:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554833536 00000100:00100000:2.0:1789760787.970349:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731179640907 time 731197606096 state 0x0 ref 1 00000100:00100000:2.0:1789760787.970365:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731197866512 time 731197606096 state 0x1 ref 1 00000100:00100000:2.0:1789760787.970423:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731197866512 time 731216041134 state 0x0 ref 1 00000100:00100000:2.0:1789760787.970441:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554833536:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760787.970489:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66138112, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f8b0 00010000:00010000:2.0:1789760787.970497:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c63bb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.970512:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760787.970517:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760787.970523:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c63bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66138112->66142207] (req 66138112->66142207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760787.970537:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c63bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66138112->66142207] (req 66138112->66142207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760787.978838:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983456 is committed 00002000:00010000:1.0:1789760787.979144:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66138112 -> 66142208 00002000:00010000:1.0:1789760787.979149:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129200 -> 129208 00010000:00010000:1.0:1789760787.979160:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c63bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66138112->66142207] (req 66138112->66142207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.979178:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c63bb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66138112->66142207] (req 66138112->66142207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.979186:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c63bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66138112->66142207] (req 66138112->66142207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.979192:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760787.979196:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c63bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66138112->66142207] (req 66138112->66142207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.979204:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c63bb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66138112->66142207] (req 66138112->66142207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.979223:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c63bb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66138112->66142207] (req 66138112->66142207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760787.979230:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c63bb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66138112->66142207] (req 66138112->66142207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760787.979524:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554833536:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9080us (9331us total) trans 4294983456 rc 0/0 00000100:00100000:1.0:1789760787.979535:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16158 00000100:00100000:1.0:1789760787.979544:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731197866512 time 731216041134 state 0x2 ref 0 00000100:00100000:2.0:1789760787.984737:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760787.984989:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554833664 00000100:00100000:0.0:1789760787.985008:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554833664 00000100:00100000:0.0:1789760787.985069:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731197866512 time 731216041134 state 0x0 ref 1 00000100:00100000:0.0:1789760787.985085:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731216301550 time 731216041134 state 0x1 ref 1 00000100:00100000:0.0:1789760787.985118:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731216301550 time 731230736584 state 0x0 ref 1 00000100:00100000:0.0:1789760787.985131:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09859880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554833664:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760787.985174:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66142208, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f8b8 00010000:00010000:0.0:1789760787.985181:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c63c2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.985194:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760787.985197:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760787.985202:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c63c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66142208->66146303] (req 66142208->66146303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.985212:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c63c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66142208->66146303] (req 66142208->66146303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760787.998676:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983457 is committed 00002000:00010000:0.0:1789760787.998840:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66142208 -> 66146304 00002000:00010000:0.0:1789760787.998846:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129208 -> 129216 00010000:00010000:0.0:1789760787.998860:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c63c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66142208->66146303] (req 66142208->66146303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.998872:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c63c2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66142208->66146303] (req 66142208->66146303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.998881:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c63c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66142208->66146303] (req 66142208->66146303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.998889:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760787.998894:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c63c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66142208->66146303] (req 66142208->66146303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.998903:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c63c2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66142208->66146303] (req 66142208->66146303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.998922:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c63c2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66142208->66146303] (req 66142208->66146303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760787.998943:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c63c2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66142208->66146303] (req 66142208->66146303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760787.999307:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09859880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554833664:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14172us (14574us total) trans 4294983457 rc 0/0 00000100:00100000:0.0:1789760787.999319:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16159 00000100:00100000:0.0:1789760787.999329:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731216301550 time 731230736584 state 0x2 ref 0 00000100:00100000:0.0:1789760788.015650:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760788.015758:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554833792 00000100:00100000:1.0:1789760788.015770:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554833792 00000100:00100000:1.0:1789760788.015884:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731216301550 time 731230736584 state 0x0 ref 1 00000100:00100000:1.0:1789760788.015899:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731230997000 time 731230736584 state 0x1 ref 1 00000100:00100000:1.0:1789760788.015932:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731230997000 time 731261551312 state 0x0 ref 1 00000100:00100000:1.0:1789760788.015949:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554833792:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760788.015994:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66146304, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f8c0 00010000:00010000:1.0:1789760788.016004:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c63c9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.016020:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760788.016025:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.016031:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c63c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66146304->66150399] (req 66146304->66150399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.016045:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c63c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66146304->66150399] (req 66146304->66150399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.023479:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983458 is committed 00002000:00010000:1.0:1789760788.023758:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66146304 -> 66150400 00002000:00010000:1.0:1789760788.023764:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129216 -> 129224 00010000:00010000:1.0:1789760788.023779:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c63c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66146304->66150399] (req 66146304->66150399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.023791:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c63c9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66146304->66150399] (req 66146304->66150399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.023807:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c63c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66146304->66150399] (req 66146304->66150399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.023815:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760788.023821:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c63c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66146304->66150399] (req 66146304->66150399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.023829:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c63c9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66146304->66150399] (req 66146304->66150399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.023849:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c63c9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66146304->66150399] (req 66146304->66150399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.023858:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c63c9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66146304->66150399] (req 66146304->66150399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.024004:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554833792:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8054us (8366us total) trans 4294983458 rc 0/0 00000100:00100000:1.0:1789760788.024026:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16160 00000100:00100000:1.0:1789760788.024037:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731230997000 time 731261551312 state 0x2 ref 0 00000100:00100000:2.0:1789760788.028225:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760788.030031:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554833920 00000100:00100000:1.0:1789760788.030046:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554833920 00000100:00100000:1.0:1789760788.030103:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731230997000 time 731261551312 state 0x0 ref 1 00000100:00100000:1.0:1789760788.030118:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731261811728 time 731261551312 state 0x1 ref 1 00000100:00100000:1.0:1789760788.030135:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731261811728 time 731275754181 state 0x0 ref 1 00000100:00100000:1.0:1789760788.030148:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0985aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554833920:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760788.030198:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66150400, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f8c8 00010000:00010000:1.0:1789760788.030207:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c63d0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.030221:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760788.030225:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.030231:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c63d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66150400->66154495] (req 66150400->66154495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.030244:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c63d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66150400->66154495] (req 66150400->66154495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.036499:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983459 is committed 00002000:00010000:1.0:1789760788.036823:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66150400 -> 66154496 00002000:00010000:1.0:1789760788.036829:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129224 -> 129232 00010000:00010000:1.0:1789760788.036842:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c63d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66150400->66154495] (req 66150400->66154495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.036855:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c63d0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66150400->66154495] (req 66150400->66154495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.036864:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c63d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66150400->66154495] (req 66150400->66154495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.036878:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760788.036883:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c63d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66150400->66154495] (req 66150400->66154495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.036892:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c63d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66150400->66154495] (req 66150400->66154495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.036913:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c63d0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66150400->66154495] (req 66150400->66154495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.036920:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c63d0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66150400->66154495] (req 66150400->66154495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.037053:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0985aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554833920:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6902us (8832us total) trans 4294983459 rc 0/0 00000100:00100000:1.0:1789760788.037063:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16161 00000100:00100000:1.0:1789760788.037073:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731261811728 time 731275754181 state 0x2 ref 0 00000100:00100000:2.0:1789760788.041145:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760788.041195:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554834048 00000100:00100000:1.0:1789760788.041209:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554834048 00000100:00100000:1.0:1789760788.041259:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731261811728 time 731275754181 state 0x0 ref 1 00000100:00100000:1.0:1789760788.041274:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731276014597 time 731275754181 state 0x1 ref 1 00000100:00100000:1.0:1789760788.041306:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731276014597 time 731286924893 state 0x0 ref 1 00000100:00100000:1.0:1789760788.041323:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09858a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554834048:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760788.041367:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66154496, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f8d0 00010000:00010000:1.0:1789760788.041426:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c63d7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.041444:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760788.041449:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.041456:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c63d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66154496->66158591] (req 66154496->66158591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.041470:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c63d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66154496->66158591] (req 66154496->66158591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.051282:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983460 is committed 00010000:00010000:0.0:1789760788.052966:0:6509:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760788.053694:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66154496 -> 66158592 00002000:00010000:2.0:1789760788.053709:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129232 -> 129240 00010000:00010000:2.0:1789760788.053723:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c63d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66154496->66158591] (req 66154496->66158591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.053736:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c63d7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66154496->66158591] (req 66154496->66158591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.053751:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c63d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66154496->66158591] (req 66154496->66158591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.053759:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760788.053764:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c63d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66154496->66158591] (req 66154496->66158591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.053773:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c63d7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66154496->66158591] (req 66154496->66158591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.053792:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c63d7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66154496->66158591] (req 66154496->66158591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.053800:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c63d7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66154496->66158591] (req 66154496->66158591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.053973:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09858a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554834048:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12648us (12831us total) trans 4294983460 rc 0/0 00000100:00100000:2.0:1789760788.053985:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16162 00000100:00100000:2.0:1789760788.053994:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731276014597 time 731286924893 state 0x2 ref 0 00000100:00100000:0.0:1789760788.062959:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760788.063889:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554834176 00000100:00100000:2.0:1789760788.063905:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554834176 00000100:00100000:2.0:1789760788.063974:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731276014597 time 731286924893 state 0x0 ref 1 00000100:00100000:2.0:1789760788.063990:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731287185309 time 731286924893 state 0x1 ref 1 00000100:00100000:2.0:1789760788.064065:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731287185309 time 731309681748 state 0x0 ref 1 00000100:00100000:2.0:1789760788.064080:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554834176:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760788.064130:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66158592, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f8d8 00010000:00010000:2.0:1789760788.064139:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c63de lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.064155:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760788.064159:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.064166:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c63de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66158592->66162687] (req 66158592->66162687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.064180:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c63de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66158592->66162687] (req 66158592->66162687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.072277:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983461 is committed 00002000:00010000:2.0:1789760788.072573:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66158592 -> 66162688 00002000:00010000:2.0:1789760788.072579:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129240 -> 129248 00010000:00010000:2.0:1789760788.072591:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c63de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66158592->66162687] (req 66158592->66162687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.072605:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c63de lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66158592->66162687] (req 66158592->66162687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.072614:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c63de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66158592->66162687] (req 66158592->66162687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.072627:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760788.072632:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c63de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66158592->66162687] (req 66158592->66162687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.072653:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c63de lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66158592->66162687] (req 66158592->66162687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.072672:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c63de lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66158592->66162687] (req 66158592->66162687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.072679:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c63de lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66158592->66162687] (req 66158592->66162687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.072808:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554834176:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8726us (9852us total) trans 4294983461 rc 0/0 00000100:00100000:2.0:1789760788.072819:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16163 00000100:00100000:2.0:1789760788.072829:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731287185309 time 731309681748 state 0x2 ref 0 00000100:00100000:0.0:1789760788.079161:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760788.080623:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554834304 00000100:00100000:2.0:1789760788.080648:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554834304 00000100:00100000:2.0:1789760788.080705:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731287185309 time 731309681748 state 0x0 ref 1 00000100:00100000:2.0:1789760788.080720:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731309942164 time 731309681748 state 0x1 ref 1 00000100:00100000:2.0:1789760788.080757:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731309942164 time 731326375634 state 0x0 ref 1 00000100:00100000:2.0:1789760788.080773:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554834304:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760788.080818:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66162688, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f8e0 00010000:00010000:2.0:1789760788.080827:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c63e5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.080844:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760788.080848:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.080855:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c63e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66162688->66166783] (req 66162688->66166783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.080869:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c63e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66162688->66166783] (req 66162688->66166783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.094509:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983462 is committed 00002000:00010000:2.0:1789760788.094745:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66162688 -> 66166784 00002000:00010000:2.0:1789760788.094750:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129248 -> 129256 00010000:00010000:2.0:1789760788.094763:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c63e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66162688->66166783] (req 66162688->66166783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.094775:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c63e5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66162688->66166783] (req 66162688->66166783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.094785:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c63e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66162688->66166783] (req 66162688->66166783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.094792:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760788.094797:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c63e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66162688->66166783] (req 66162688->66166783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.094806:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c63e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66162688->66166783] (req 66162688->66166783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.094824:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c63e5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66162688->66166783] (req 66162688->66166783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.094831:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c63e5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66162688->66166783] (req 66162688->66166783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.094923:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554834304:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14146us (15775us total) trans 4294983462 rc 0/0 00000100:00100000:2.0:1789760788.094934:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16164 00000100:00100000:2.0:1789760788.094957:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731309942164 time 731326375634 state 0x2 ref 0 00000100:00100000:0.0:1789760788.108699:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760788.109011:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554834432 00000100:00100000:1.0:1789760788.109029:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554834432 00000100:00100000:1.0:1789760788.109079:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731309942164 time 731326375634 state 0x0 ref 1 00000100:00100000:1.0:1789760788.109094:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731326636050 time 731326375634 state 0x1 ref 1 00000100:00100000:1.0:1789760788.109125:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731326636050 time 731354743813 state 0x0 ref 1 00000100:00100000:1.0:1789760788.109136:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554834432:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760788.109181:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66166784, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f8e8 00010000:00010000:1.0:1789760788.109190:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c63ec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.109204:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760788.109209:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.109215:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c63ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66166784->66170879] (req 66166784->66170879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.109228:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c63ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66166784->66170879] (req 66166784->66170879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.124101:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983463 is committed 00002000:00010000:1.0:1789760788.124292:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66166784 -> 66170880 00002000:00010000:1.0:1789760788.124297:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129256 -> 129264 00010000:00010000:1.0:1789760788.124311:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c63ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66166784->66170879] (req 66166784->66170879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.124324:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c63ec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66166784->66170879] (req 66166784->66170879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.124334:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c63ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66166784->66170879] (req 66166784->66170879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.124341:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760788.124347:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c63ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66166784->66170879] (req 66166784->66170879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.124356:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c63ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66166784->66170879] (req 66166784->66170879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.124421:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c63ec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66166784->66170879] (req 66166784->66170879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.124431:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c63ec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66166784->66170879] (req 66166784->66170879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.124523:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554834432:12345-192.168.203.22@tcp:4:dd.0 Request processed in 15383us (15828us total) trans 4294983463 rc 0/0 00000100:00100000:1.0:1789760788.124533:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16165 00000100:00100000:1.0:1789760788.124542:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731326636050 time 731354743813 state 0x2 ref 0 00000100:00100000:0.0:1789760788.128368:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760788.128580:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554834560 00000100:00100000:1.0:1789760788.128592:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554834560 00000100:00100000:1.0:1789760788.128661:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731326636050 time 731354743813 state 0x0 ref 1 00000100:00100000:1.0:1789760788.128678:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731355004229 time 731354743813 state 0x1 ref 1 00000100:00100000:1.0:1789760788.128699:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731355004229 time 731374318192 state 0x0 ref 1 00000100:00100000:1.0:1789760788.128711:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e93800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554834560:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760788.128757:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66170880, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f8f0 00010000:00010000:1.0:1789760788.128766:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c63f3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.128782:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760788.128786:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.128793:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c63f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66170880->66174975] (req 66170880->66174975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.128807:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c63f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66170880->66174975] (req 66170880->66174975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.140282:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983464 is committed 00002000:00010000:1.0:1789760788.141101:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66170880 -> 66174976 00002000:00010000:1.0:1789760788.141108:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129264 -> 129272 00010000:00010000:1.0:1789760788.141121:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c63f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66170880->66174975] (req 66170880->66174975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.141134:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c63f3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66170880->66174975] (req 66170880->66174975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.141144:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c63f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66170880->66174975] (req 66170880->66174975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.141152:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760788.141157:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c63f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66170880->66174975] (req 66170880->66174975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.141167:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c63f3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66170880->66174975] (req 66170880->66174975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.141188:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c63f3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66170880->66174975] (req 66170880->66174975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.141201:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c63f3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66170880->66174975] (req 66170880->66174975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.141336:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e93800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554834560:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12621us (12973us total) trans 4294983464 rc 0/0 00000100:00100000:1.0:1789760788.141347:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16166 00000100:00100000:1.0:1789760788.141357:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731355004229 time 731374318192 state 0x2 ref 0 00000100:00100000:0.0:1789760788.146860:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760788.146944:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554834688 00000100:00100000:2.0:1789760788.146953:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554834688 00000100:00100000:2.0:1789760788.147007:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731355004229 time 731374318192 state 0x0 ref 1 00000100:00100000:2.0:1789760788.147035:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731374578608 time 731374318192 state 0x1 ref 1 00000100:00100000:2.0:1789760788.147065:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731374578608 time 731392683519 state 0x0 ref 1 00000100:00100000:2.0:1789760788.147077:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554834688:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760788.147123:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66174976, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f8f8 00010000:00010000:2.0:1789760788.147133:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c63fa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.147149:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760788.147153:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.147159:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c63fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66174976->66179071] (req 66174976->66179071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.147174:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c63fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66174976->66179071] (req 66174976->66179071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.155871:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983465 is committed 00002000:00010000:2.0:1789760788.156909:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66174976 -> 66179072 00002000:00010000:2.0:1789760788.156916:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129272 -> 129280 00010000:00010000:2.0:1789760788.156929:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c63fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66174976->66179071] (req 66174976->66179071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.156942:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c63fa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66174976->66179071] (req 66174976->66179071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.156951:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c63fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66174976->66179071] (req 66174976->66179071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.156959:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760788.156964:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c63fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66174976->66179071] (req 66174976->66179071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.156972:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c63fa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66174976->66179071] (req 66174976->66179071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.156992:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c63fa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66174976->66179071] (req 66174976->66179071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.156999:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c63fa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66174976->66179071] (req 66174976->66179071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.157579:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554834688:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10498us (10723us total) trans 4294983465 rc 0/0 00000100:00100000:2.0:1789760788.157589:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16167 00000100:00100000:2.0:1789760788.157598:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731374578608 time 731392683519 state 0x2 ref 0 00000100:00100000:0.0:1789760788.161626:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760788.161905:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554834816 00000100:00100000:2.0:1789760788.161914:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554834816 00000100:00100000:2.0:1789760788.161973:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731374578608 time 731392683519 state 0x0 ref 1 00000100:00100000:2.0:1789760788.161988:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731392943935 time 731392683519 state 0x1 ref 1 00000100:00100000:2.0:1789760788.162080:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731392943935 time 731407696776 state 0x0 ref 1 00000100:00100000:2.0:1789760788.162095:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554834816:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760788.162161:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66179072, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f900 00010000:00010000:2.0:1789760788.162171:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6401 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.162186:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760788.162191:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.162196:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6401 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66179072->66183167] (req 66179072->66183167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.162208:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6401 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66179072->66183167] (req 66179072->66183167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.173352:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983466 is committed 00002000:00010000:2.0:1789760788.173515:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66179072 -> 66183168 00002000:00010000:2.0:1789760788.173521:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129280 -> 129288 00010000:00010000:2.0:1789760788.173534:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6401 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66179072->66183167] (req 66179072->66183167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.173548:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6401 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66179072->66183167] (req 66179072->66183167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.173558:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6401 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66179072->66183167] (req 66179072->66183167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.173566:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760788.173571:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6401 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66179072->66183167] (req 66179072->66183167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.173580:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6401 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66179072->66183167] (req 66179072->66183167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.173601:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6401 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66179072->66183167] (req 66179072->66183167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.173614:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6401 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66179072->66183167] (req 66179072->66183167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.173719:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554834816:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11623us (12099us total) trans 4294983466 rc 0/0 00000100:00100000:2.0:1789760788.173729:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16168 00000100:00100000:2.0:1789760788.173739:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731392943935 time 731407696776 state 0x2 ref 0 00000100:00100000:0.0:1789760788.181597:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760788.181680:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554834944 00000100:00100000:1.0:1789760788.181691:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554834944 00000100:00100000:1.0:1789760788.181741:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731392943935 time 731407696776 state 0x0 ref 1 00000100:00100000:1.0:1789760788.181756:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731407957192 time 731407696776 state 0x1 ref 1 00000100:00100000:1.0:1789760788.181787:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731407957192 time 731427406445 state 0x0 ref 1 00000100:00100000:1.0:1789760788.181799:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554834944:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760788.181848:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66183168, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f908 00010000:00010000:1.0:1789760788.181857:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6408 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.181873:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760788.181877:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.181883:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6408 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66183168->66187263] (req 66183168->66187263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.181902:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6408 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66183168->66187263] (req 66183168->66187263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.188154:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983467 is committed 00002000:00010000:1.0:1789760788.188494:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66183168 -> 66187264 00002000:00010000:1.0:1789760788.188501:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129288 -> 129296 00010000:00010000:1.0:1789760788.188517:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6408 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66183168->66187263] (req 66183168->66187263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.188531:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6408 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66183168->66187263] (req 66183168->66187263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.188541:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6408 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66183168->66187263] (req 66183168->66187263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.188548:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760788.188553:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6408 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66183168->66187263] (req 66183168->66187263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.188561:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6408 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66183168->66187263] (req 66183168->66187263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.188580:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6408 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66183168->66187263] (req 66183168->66187263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.188587:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6408 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66183168->66187263] (req 66183168->66187263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.188681:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554834944:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6879us (7092us total) trans 4294983467 rc 0/0 00000100:00100000:1.0:1789760788.188692:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16169 00000100:00100000:1.0:1789760788.188703:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731407957192 time 731427406445 state 0x2 ref 0 00000100:00100000:0.0:1789760788.192924:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760788.193123:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554835072 00000100:00100000:1.0:1789760788.193133:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554835072 00000100:00100000:1.0:1789760788.193187:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731407957192 time 731427406445 state 0x0 ref 1 00000100:00100000:1.0:1789760788.193203:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731427666861 time 731427406445 state 0x1 ref 1 00000100:00100000:1.0:1789760788.193295:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731427666861 time 731438912004 state 0x0 ref 1 00000100:00100000:1.0:1789760788.193310:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554835072:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760788.193358:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66187264, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f910 00010000:00010000:1.0:1789760788.193368:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c640f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.193428:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760788.193433:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.193440:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c640f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66187264->66191359] (req 66187264->66191359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.193455:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c640f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66187264->66191359] (req 66187264->66191359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.201330:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983468 is committed 00002000:00010000:2.0:1789760788.201953:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66187264 -> 66191360 00002000:00010000:2.0:1789760788.201959:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129296 -> 129304 00010000:00010000:2.0:1789760788.201973:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c640f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66187264->66191359] (req 66187264->66191359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.201985:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c640f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66187264->66191359] (req 66187264->66191359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.201994:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c640f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66187264->66191359] (req 66187264->66191359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.202001:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760788.202005:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c640f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66187264->66191359] (req 66187264->66191359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.202014:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c640f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66187264->66191359] (req 66187264->66191359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.202039:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c640f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66187264->66191359] (req 66187264->66191359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.202047:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c640f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66187264->66191359] (req 66187264->66191359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.202146:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554835072:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8836us (9233us total) trans 4294983468 rc 0/0 00000100:00100000:2.0:1789760788.202156:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16170 00000100:00100000:2.0:1789760788.202166:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731427666861 time 731438912004 state 0x2 ref 0 00000100:00100000:0.0:1789760788.209798:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760788.209981:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554835200 00000100:00100000:2.0:1789760788.209992:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554835200 00000100:00100000:2.0:1789760788.210061:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731427666861 time 731438912004 state 0x0 ref 1 00000100:00100000:2.0:1789760788.210078:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731439172420 time 731438912004 state 0x1 ref 1 00000100:00100000:2.0:1789760788.210114:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731439172420 time 731455732116 state 0x0 ref 1 00000100:00100000:2.0:1789760788.210128:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e92a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554835200:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760788.210181:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66191360, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f918 00010000:00010000:2.0:1789760788.210190:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6416 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.210207:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760788.210213:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.210219:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6416 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66191360->66195455] (req 66191360->66195455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.210233:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6416 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66191360->66195455] (req 66191360->66195455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.216062:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983469 is committed 00002000:00010000:2.0:1789760788.216667:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66191360 -> 66195456 00002000:00010000:2.0:1789760788.216674:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129304 -> 129312 00010000:00010000:2.0:1789760788.216686:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6416 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66191360->66195455] (req 66191360->66195455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.216699:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6416 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66191360->66195455] (req 66191360->66195455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.216708:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6416 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66191360->66195455] (req 66191360->66195455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.216716:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760788.216721:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6416 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66191360->66195455] (req 66191360->66195455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.216731:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6416 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66191360->66195455] (req 66191360->66195455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.216753:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6416 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66191360->66195455] (req 66191360->66195455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.216767:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6416 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66191360->66195455] (req 66191360->66195455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.216864:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e92a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554835200:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6733us (7069us total) trans 4294983469 rc 0/0 00000100:00100000:2.0:1789760788.216876:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16171 00000100:00100000:2.0:1789760788.216886:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731439172420 time 731455732116 state 0x2 ref 0 00000100:00100000:0.0:1789760788.221850:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760788.221966:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554835328 00000100:00100000:2.0:1789760788.221975:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554835328 00000100:00100000:2.0:1789760788.222174:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731439172420 time 731455732116 state 0x0 ref 1 00000100:00100000:2.0:1789760788.222189:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731455992532 time 731455732116 state 0x1 ref 1 00000100:00100000:2.0:1789760788.222218:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731455992532 time 731467836226 state 0x0 ref 1 00000100:00100000:2.0:1789760788.222231:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554835328:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760788.222278:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66195456, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f920 00010000:00010000:2.0:1789760788.222288:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c641d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.222308:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760788.222313:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.222319:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c641d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66195456->66199551] (req 66195456->66199551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.222338:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c641d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66195456->66199551] (req 66195456->66199551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.229121:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983470 is committed 00002000:00010000:2.0:1789760788.229236:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66195456 -> 66199552 00002000:00010000:2.0:1789760788.229241:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129312 -> 129320 00010000:00010000:2.0:1789760788.229255:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c641d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66195456->66199551] (req 66195456->66199551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.229270:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c641d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66195456->66199551] (req 66195456->66199551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.229281:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c641d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66195456->66199551] (req 66195456->66199551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.229288:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760788.229294:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c641d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66195456->66199551] (req 66195456->66199551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.229303:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c641d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66195456->66199551] (req 66195456->66199551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.229324:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c641d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66195456->66199551] (req 66195456->66199551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.229333:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c641d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66195456->66199551] (req 66195456->66199551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.229734:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554835328:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7501us (7895us total) trans 4294983470 rc 0/0 00000100:00100000:2.0:1789760788.229745:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16172 00000100:00100000:2.0:1789760788.229754:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731455992532 time 731467836226 state 0x2 ref 0 00000100:00100000:0.0:1789760788.234226:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760788.234766:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554835456 00000100:00100000:1.0:1789760788.234777:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554835456 00000100:00100000:1.0:1789760788.234831:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731455992532 time 731467836226 state 0x0 ref 1 00000100:00100000:1.0:1789760788.234847:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731468096642 time 731467836226 state 0x1 ref 1 00000100:00100000:1.0:1789760788.234906:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731468096642 time 731480523588 state 0x0 ref 1 00000100:00100000:1.0:1789760788.234919:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554835456:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760788.234967:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66199552, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f928 00010000:00010000:1.0:1789760788.234976:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6424 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.234990:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760788.234994:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.235000:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6424 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66199552->66203647] (req 66199552->66203647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.235011:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6424 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66199552->66203647] (req 66199552->66203647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.240128:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983471 is committed 00002000:00010000:0.0:1789760788.240241:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66199552 -> 66203648 00002000:00010000:0.0:1789760788.240246:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129320 -> 129328 00010000:00010000:0.0:1789760788.240259:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6424 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66199552->66203647] (req 66199552->66203647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.240272:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6424 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66199552->66203647] (req 66199552->66203647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.240282:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6424 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66199552->66203647] (req 66199552->66203647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.240295:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760788.240301:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6424 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66199552->66203647] (req 66199552->66203647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.240311:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6424 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66199552->66203647] (req 66199552->66203647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.240330:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6424 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66199552->66203647] (req 66199552->66203647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.240338:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6424 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66199552->66203647] (req 66199552->66203647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.242876:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554835456:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7946us (8646us total) trans 4294983471 rc 0/0 00000100:00100000:0.0:1789760788.242897:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16173 00000100:00100000:0.0:1789760788.242908:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731468096642 time 731480523588 state 0x2 ref 0 00000100:00100000:0.0:1789760788.247823:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760788.247915:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554835584 00000100:00100000:1.0:1789760788.247925:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554835584 00000100:00100000:1.0:1789760788.247982:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731468096642 time 731480523588 state 0x0 ref 1 00000100:00100000:1.0:1789760788.247996:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731480784004 time 731480523588 state 0x1 ref 1 00000100:00100000:1.0:1789760788.248115:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731480784004 time 731493731919 state 0x0 ref 1 00000100:00100000:1.0:1789760788.248128:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554835584:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760788.248176:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66203648, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f930 00010000:00010000:1.0:1789760788.248184:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c642b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.248200:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760788.248204:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.248210:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c642b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66203648->66207743] (req 66203648->66207743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.248223:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c642b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66203648->66207743] (req 66203648->66207743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.253255:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983472 is committed 00002000:00010000:1.0:1789760788.253366:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66203648 -> 66207744 00002000:00010000:1.0:1789760788.253371:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129328 -> 129336 00010000:00010000:1.0:1789760788.253402:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c642b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66203648->66207743] (req 66203648->66207743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.253413:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c642b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66203648->66207743] (req 66203648->66207743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.253426:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c642b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66203648->66207743] (req 66203648->66207743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.253434:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760788.253439:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c642b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66203648->66207743] (req 66203648->66207743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.253447:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c642b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66203648->66207743] (req 66203648->66207743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.253466:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c642b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66203648->66207743] (req 66203648->66207743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.253473:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c642b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66203648->66207743] (req 66203648->66207743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.253572:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554835584:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5440us (5751us total) trans 4294983472 rc 0/0 00000100:00100000:1.0:1789760788.253585:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16174 00000100:00100000:1.0:1789760788.253596:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731480784004 time 731493731919 state 0x2 ref 0 00000100:00100000:2.0:1789760788.261493:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760788.263185:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554835712 00000100:00100000:0.0:1789760788.263198:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554835712 00000100:00100000:0.0:1789760788.263258:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731480784004 time 731493731919 state 0x0 ref 1 00000100:00100000:0.0:1789760788.263274:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731493992335 time 731493731919 state 0x1 ref 1 00000100:00100000:0.0:1789760788.263311:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731493992335 time 731508929488 state 0x0 ref 1 00000100:00100000:0.0:1789760788.263325:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09859c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554835712:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760788.263371:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66207744, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f938 00010000:00010000:0.0:1789760788.263559:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6432 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.263577:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760788.263582:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.263588:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6432 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66207744->66211839] (req 66207744->66211839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.263602:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6432 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66207744->66211839] (req 66207744->66211839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.274818:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983473 is committed 00002000:00010000:0.0:1789760788.276230:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66207744 -> 66211840 00002000:00010000:0.0:1789760788.276239:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129336 -> 129344 00010000:00010000:0.0:1789760788.276252:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6432 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66207744->66211839] (req 66207744->66211839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.276264:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6432 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66207744->66211839] (req 66207744->66211839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.276274:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6432 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66207744->66211839] (req 66207744->66211839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.276281:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760788.276286:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6432 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66207744->66211839] (req 66207744->66211839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.276295:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6432 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66207744->66211839] (req 66207744->66211839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.276314:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6432 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66207744->66211839] (req 66207744->66211839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.276321:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6432 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66207744->66211839] (req 66207744->66211839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.276871:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09859c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554835712:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13542us (15381us total) trans 4294983473 rc 0/0 00000100:00100000:0.0:1789760788.276884:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16175 00000100:00100000:0.0:1789760788.276894:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731493992335 time 731508929488 state 0x2 ref 0 00000100:00100000:0.0:1789760788.281900:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760788.281998:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554835840 00000100:00100000:2.0:1789760788.282007:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554835840 00000100:00100000:2.0:1789760788.282070:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731493992335 time 731508929488 state 0x0 ref 1 00000100:00100000:2.0:1789760788.282087:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731509189904 time 731508929488 state 0x1 ref 1 00000100:00100000:2.0:1789760788.282108:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731509189904 time 731527727291 state 0x0 ref 1 00000100:00100000:2.0:1789760788.282120:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554835840:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760788.282167:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66211840, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f940 00010000:00010000:2.0:1789760788.282177:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6439 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.282194:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760788.282199:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.282205:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6439 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66211840->66215935] (req 66211840->66215935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.282223:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6439 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66211840->66215935] (req 66211840->66215935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.287950:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983474 is committed 00002000:00010000:2.0:1789760788.288758:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66211840 -> 66215936 00002000:00010000:2.0:1789760788.288765:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129344 -> 129352 00010000:00010000:2.0:1789760788.288780:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6439 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66211840->66215935] (req 66211840->66215935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.288794:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6439 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66211840->66215935] (req 66211840->66215935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.288803:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6439 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66211840->66215935] (req 66211840->66215935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.288811:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760788.288816:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6439 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66211840->66215935] (req 66211840->66215935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.288826:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6439 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66211840->66215935] (req 66211840->66215935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.288845:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6439 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66211840->66215935] (req 66211840->66215935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.288853:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6439 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66211840->66215935] (req 66211840->66215935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.289909:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554835840:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7782us (8010us total) trans 4294983474 rc 0/0 00000100:00100000:2.0:1789760788.289922:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16176 00000100:00100000:2.0:1789760788.289932:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731509189904 time 731527727291 state 0x2 ref 0 00000100:00100000:2.0:1789760788.298072:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760788.299082:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554835968 00000100:00100000:1.0:1789760788.299093:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554835968 00000100:00100000:1.0:1789760788.299150:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731509189904 time 731527727291 state 0x0 ref 1 00000100:00100000:1.0:1789760788.299170:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731527987707 time 731527727291 state 0x1 ref 1 00000100:00100000:1.0:1789760788.299205:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731527987707 time 731544823426 state 0x0 ref 1 00000100:00100000:1.0:1789760788.299219:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09859180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554835968:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760788.299265:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66215936, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f948 00010000:00010000:1.0:1789760788.299275:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6440 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.299295:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760788.299299:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.299305:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6440 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66215936->66220031] (req 66215936->66220031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.299319:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6440 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66215936->66220031] (req 66215936->66220031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.304119:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983475 is committed 00002000:00010000:1.0:1789760788.304236:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66215936 -> 66220032 00002000:00010000:1.0:1789760788.304241:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129352 -> 129360 00010000:00010000:1.0:1789760788.304253:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6440 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66215936->66220031] (req 66215936->66220031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.304265:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6440 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66215936->66220031] (req 66215936->66220031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.304274:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6440 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66215936->66220031] (req 66215936->66220031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.304280:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760788.304285:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6440 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66215936->66220031] (req 66215936->66220031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.304293:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6440 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66215936->66220031] (req 66215936->66220031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.304312:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6440 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66215936->66220031] (req 66215936->66220031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.304320:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6440 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66215936->66220031] (req 66215936->66220031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.304611:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09859180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554835968:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5390us (6546us total) trans 4294983475 rc 0/0 00000100:00100000:1.0:1789760788.304621:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16177 00000100:00100000:1.0:1789760788.304630:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731527987707 time 731544823426 state 0x2 ref 0 00000100:00100000:0.0:1789760788.312232:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760788.312318:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554836096 00000100:00100000:1.0:1789760788.312328:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554836096 00000100:00100000:1.0:1789760788.312430:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731527987707 time 731544823426 state 0x0 ref 1 00000100:00100000:1.0:1789760788.312446:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731545083842 time 731544823426 state 0x1 ref 1 00000100:00100000:1.0:1789760788.312478:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731545083842 time 731558096647 state 0x0 ref 1 00000100:00100000:1.0:1789760788.312493:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554836096:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760788.312540:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66220032, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f950 00010000:00010000:1.0:1789760788.312549:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6447 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.312565:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760788.312569:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.312576:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6447 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66220032->66224127] (req 66220032->66224127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.312590:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6447 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66220032->66224127] (req 66220032->66224127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.316707:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983476 is committed 00002000:00010000:2.0:1789760788.316815:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66220032 -> 66224128 00002000:00010000:2.0:1789760788.316820:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129360 -> 129368 00010000:00010000:2.0:1789760788.316833:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6447 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66220032->66224127] (req 66220032->66224127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.316846:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6447 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66220032->66224127] (req 66220032->66224127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.316855:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6447 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66220032->66224127] (req 66220032->66224127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.316863:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760788.316868:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6447 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66220032->66224127] (req 66220032->66224127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.316877:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6447 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66220032->66224127] (req 66220032->66224127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.316902:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6447 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66220032->66224127] (req 66220032->66224127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.316911:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6447 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66220032->66224127] (req 66220032->66224127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.317055:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554836096:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4559us (4831us total) trans 4294983476 rc 0/0 00000100:00100000:2.0:1789760788.317066:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16178 00000100:00100000:2.0:1789760788.317076:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731545083842 time 731558096647 state 0x2 ref 0 00000100:00100000:0.0:1789760788.322700:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760788.322804:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554836224 00000100:00100000:2.0:1789760788.322814:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554836224 00000100:00100000:2.0:1789760788.322868:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731545083842 time 731558096647 state 0x0 ref 1 00000100:00100000:2.0:1789760788.322883:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731558357063 time 731558096647 state 0x1 ref 1 00000100:00100000:2.0:1789760788.322950:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731558357063 time 731568567931 state 0x0 ref 1 00000100:00100000:2.0:1789760788.322962:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554836224:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760788.323023:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66224128, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f958 00010000:00010000:2.0:1789760788.323033:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c644e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.323049:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760788.323055:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.323061:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c644e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66224128->66228223] (req 66224128->66228223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.323074:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c644e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66224128->66228223] (req 66224128->66228223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.328690:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983477 is committed 00002000:00010000:2.0:1789760788.328796:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66224128 -> 66228224 00002000:00010000:2.0:1789760788.328801:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129368 -> 129376 00010000:00010000:2.0:1789760788.328812:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c644e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66224128->66228223] (req 66224128->66228223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.328825:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c644e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66224128->66228223] (req 66224128->66228223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.328834:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c644e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66224128->66228223] (req 66224128->66228223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.328842:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760788.328847:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c644e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66224128->66228223] (req 66224128->66228223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.328856:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c644e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66224128->66228223] (req 66224128->66228223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.328875:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c644e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66224128->66228223] (req 66224128->66228223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.328887:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c644e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66224128->66228223] (req 66224128->66228223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.328979:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554836224:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6014us (6284us total) trans 4294983477 rc 0/0 00000100:00100000:2.0:1789760788.328989:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16179 00000100:00100000:2.0:1789760788.328998:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731558357063 time 731568567931 state 0x2 ref 0 00000100:00100000:0.0:1789760788.332619:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760788.332711:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554836352 00000100:00100000:2.0:1789760788.332720:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554836352 00000100:00100000:2.0:1789760788.332775:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731558357063 time 731568567931 state 0x0 ref 1 00000100:00100000:2.0:1789760788.332790:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731568828347 time 731568567931 state 0x1 ref 1 00000100:00100000:2.0:1789760788.332828:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731568828347 time 731578446087 state 0x0 ref 1 00000100:00100000:2.0:1789760788.332841:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554836352:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760788.332885:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66228224, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f960 00010000:00010000:2.0:1789760788.332894:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6455 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.332912:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760788.332917:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.332923:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6455 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66228224->66232319] (req 66228224->66232319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.332942:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6455 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66228224->66232319] (req 66228224->66232319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.336701:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983478 is committed 00002000:00010000:3.0:1789760788.345248:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66228224 -> 66232320 00002000:00010000:3.0:1789760788.345262:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129376 -> 129384 00010000:00010000:3.0:1789760788.345276:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6455 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66228224->66232319] (req 66228224->66232319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.345291:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6455 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66228224->66232319] (req 66228224->66232319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.345300:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6455 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66228224->66232319] (req 66228224->66232319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.345307:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760788.345313:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6455 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66228224->66232319] (req 66228224->66232319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.345323:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6455 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66228224->66232319] (req 66228224->66232319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.345344:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6455 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66228224->66232319] (req 66228224->66232319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.345353:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6455 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66228224->66232319] (req 66228224->66232319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.346424:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554836352:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13535us (13765us total) trans 4294983478 rc 0/0 00000100:00100000:3.0:1789760788.346438:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16180 00000100:00100000:3.0:1789760788.346449:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731568828347 time 731578446087 state 0x2 ref 0 00000100:00100000:0.0:1789760788.361976:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760788.362316:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554836480 00000100:00100000:1.0:1789760788.362327:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554836480 00000100:00100000:1.0:1789760788.362425:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731568828347 time 731578446087 state 0x0 ref 1 00000100:00100000:1.0:1789760788.362441:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731578706503 time 731578446087 state 0x1 ref 1 00000100:00100000:1.0:1789760788.362475:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731578706503 time 731608093027 state 0x0 ref 1 00000100:00100000:1.0:1789760788.362487:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554836480:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760788.362534:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66232320, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f968 00010000:00010000:1.0:1789760788.362544:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c645c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.362560:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760788.362565:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.362571:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c645c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66232320->66236415] (req 66232320->66236415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.362585:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c645c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66232320->66236415] (req 66232320->66236415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.367954:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983479 is committed 00002000:00010000:1.0:1789760788.368071:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66232320 -> 66236416 00002000:00010000:1.0:1789760788.368077:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129384 -> 129392 00010000:00010000:1.0:1789760788.368091:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c645c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66232320->66236415] (req 66232320->66236415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.368106:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c645c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66232320->66236415] (req 66232320->66236415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.368116:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c645c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66232320->66236415] (req 66232320->66236415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.368123:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760788.368128:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c645c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66232320->66236415] (req 66232320->66236415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.368138:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c645c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66232320->66236415] (req 66232320->66236415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.368165:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c645c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66232320->66236415] (req 66232320->66236415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.368173:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c645c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66232320->66236415] (req 66232320->66236415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.368274:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554836480:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5784us (6303us total) trans 4294983479 rc 0/0 00000100:00100000:1.0:1789760788.368285:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16181 00000100:00100000:1.0:1789760788.368294:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731578706503 time 731608093027 state 0x2 ref 0 00000100:00100000:0.0:1789760788.376130:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760788.376260:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554836608 00000100:00100000:1.0:1789760788.376270:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554836608 00000100:00100000:1.0:1789760788.376329:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731578706503 time 731608093027 state 0x0 ref 1 00000100:00100000:1.0:1789760788.376345:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731608353443 time 731608093027 state 0x1 ref 1 00000100:00100000:1.0:1789760788.376645:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731608353443 time 731622252873 state 0x0 ref 1 00000100:00100000:1.0:1789760788.376659:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554836608:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760788.376707:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66236416, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f970 00010000:00010000:1.0:1789760788.376718:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6463 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.376734:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760788.376739:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.376746:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6463 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66236416->66240511] (req 66236416->66240511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.376759:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6463 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66236416->66240511] (req 66236416->66240511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.380717:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983480 is committed 00002000:00010000:2.0:1789760788.380885:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66236416 -> 66240512 00002000:00010000:2.0:1789760788.380891:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129392 -> 129400 00010000:00010000:2.0:1789760788.380907:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6463 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66236416->66240511] (req 66236416->66240511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.380919:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6463 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66236416->66240511] (req 66236416->66240511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.380929:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6463 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66236416->66240511] (req 66236416->66240511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.380937:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760788.380944:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6463 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66236416->66240511] (req 66236416->66240511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.380952:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6463 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66236416->66240511] (req 66236416->66240511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.380973:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6463 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66236416->66240511] (req 66236416->66240511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.380980:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6463 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66236416->66240511] (req 66236416->66240511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.382808:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554836608:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6139us (6675us total) trans 4294983480 rc 0/0 00000100:00100000:2.0:1789760788.382823:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16182 00000100:00100000:2.0:1789760788.382832:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731608353443 time 731622252873 state 0x2 ref 0 00000100:00100000:0.0:1789760788.386988:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760788.387066:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554836736 00000100:00100000:2.0:1789760788.387087:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554836736 00000100:00100000:2.0:1789760788.387142:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731608353443 time 731622252873 state 0x0 ref 1 00000100:00100000:2.0:1789760788.387157:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731622513289 time 731622252873 state 0x1 ref 1 00000100:00100000:2.0:1789760788.387188:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731622513289 time 731632807007 state 0x0 ref 1 00000100:00100000:2.0:1789760788.387200:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554836736:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760788.387249:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66240512, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f978 00010000:00010000:2.0:1789760788.387258:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c646a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.387274:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760788.387279:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.387285:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c646a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66240512->66244607] (req 66240512->66244607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.387299:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c646a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66240512->66244607] (req 66240512->66244607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.392163:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983481 is committed 00002000:00010000:2.0:1789760788.394559:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66240512 -> 66244608 00002000:00010000:2.0:1789760788.394569:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129400 -> 129408 00010000:00010000:2.0:1789760788.394583:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c646a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66240512->66244607] (req 66240512->66244607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.394598:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c646a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66240512->66244607] (req 66240512->66244607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.394608:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c646a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66240512->66244607] (req 66240512->66244607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.394615:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760788.394621:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c646a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66240512->66244607] (req 66240512->66244607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.394631:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c646a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66240512->66244607] (req 66240512->66244607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.394660:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c646a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66240512->66244607] (req 66240512->66244607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.394668:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c646a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66240512->66244607] (req 66240512->66244607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.394826:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554836736:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7623us (7843us total) trans 4294983481 rc 0/0 00000100:00100000:2.0:1789760788.394838:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16183 00000100:00100000:2.0:1789760788.394847:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731622513289 time 731632807007 state 0x2 ref 0 00000100:00100000:0.0:1789760788.399844:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760788.399936:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554836864 00000100:00100000:3.0:1789760788.399947:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554836864 00000100:00100000:3.0:1789760788.400001:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731622513289 time 731632807007 state 0x0 ref 1 00000100:00100000:3.0:1789760788.400221:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731633067423 time 731632807007 state 0x1 ref 1 00000100:00100000:3.0:1789760788.400286:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731633067423 time 731645902823 state 0x0 ref 1 00000100:00100000:3.0:1789760788.400300:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554836864:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760788.400346:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66244608, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f980 00010000:00010000:3.0:1789760788.400355:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6471 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.400373:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760788.400406:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.400414:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6471 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66244608->66248703] (req 66244608->66248703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.400434:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6471 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66244608->66248703] (req 66244608->66248703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.404265:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983482 is committed 00002000:00010000:3.0:1789760788.404568:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66244608 -> 66248704 00002000:00010000:3.0:1789760788.404574:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129408 -> 129416 00010000:00010000:3.0:1789760788.404587:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6471 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66244608->66248703] (req 66244608->66248703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.404600:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6471 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66244608->66248703] (req 66244608->66248703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.404610:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6471 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66244608->66248703] (req 66244608->66248703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.404618:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760788.404623:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6471 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66244608->66248703] (req 66244608->66248703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.404632:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6471 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66244608->66248703] (req 66244608->66248703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.404660:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6471 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66244608->66248703] (req 66244608->66248703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.404668:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6471 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66244608->66248703] (req 66244608->66248703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.404820:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554836864:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4516us (4980us total) trans 4294983482 rc 0/0 00000100:00100000:3.0:1789760788.404831:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16184 00000100:00100000:3.0:1789760788.404841:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731633067423 time 731645902823 state 0x2 ref 0 00000100:00100000:1.0:1789760788.411352:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760788.411710:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554836992 00000100:00100000:0.0:1789760788.411720:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554836992 00000100:00100000:0.0:1789760788.411771:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731633067423 time 731645902823 state 0x0 ref 1 00000100:00100000:0.0:1789760788.411786:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731646163239 time 731645902823 state 0x1 ref 1 00000100:00100000:0.0:1789760788.411816:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731646163239 time 731657435240 state 0x0 ref 1 00000100:00100000:0.0:1789760788.411827:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554836992:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760788.411869:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66248704, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f988 00010000:00010000:0.0:1789760788.411879:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6478 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.411894:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760788.411898:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.411903:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6478 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66248704->66252799] (req 66248704->66252799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.411916:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6478 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66248704->66252799] (req 66248704->66252799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.415700:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983483 is committed 00002000:00010000:0.0:1789760788.416867:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66248704 -> 66252800 00002000:00010000:0.0:1789760788.416876:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129416 -> 129424 00010000:00010000:0.0:1789760788.416890:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6478 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66248704->66252799] (req 66248704->66252799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.416903:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6478 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66248704->66252799] (req 66248704->66252799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.416914:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6478 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66248704->66252799] (req 66248704->66252799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.416922:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760788.416926:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6478 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66248704->66252799] (req 66248704->66252799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.416936:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6478 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66248704->66252799] (req 66248704->66252799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.416955:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6478 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66248704->66252799] (req 66248704->66252799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.416962:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6478 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66248704->66252799] (req 66248704->66252799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.417161:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554836992:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5329us (5813us total) trans 4294983483 rc 0/0 00000100:00100000:0.0:1789760788.417172:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16185 00000100:00100000:0.0:1789760788.417181:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731646163239 time 731657435240 state 0x2 ref 0 00000100:00100000:0.0:1789760788.420932:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760788.421008:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554837120 00000100:00100000:2.0:1789760788.421026:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554837120 00000100:00100000:2.0:1789760788.421077:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731646163239 time 731657435240 state 0x0 ref 1 00000100:00100000:2.0:1789760788.421093:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731657695656 time 731657435240 state 0x1 ref 1 00000100:00100000:2.0:1789760788.421167:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731657695656 time 731666785694 state 0x0 ref 1 00000100:00100000:2.0:1789760788.421180:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554837120:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760788.421227:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66252800, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f990 00010000:00010000:2.0:1789760788.421237:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c647f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.421254:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760788.421258:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.421265:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c647f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66252800->66256895] (req 66252800->66256895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.421279:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c647f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66252800->66256895] (req 66252800->66256895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.424870:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983484 is committed 00002000:00010000:2.0:1789760788.424972:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66252800 -> 66256896 00002000:00010000:2.0:1789760788.424977:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129424 -> 129432 00010000:00010000:2.0:1789760788.424990:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c647f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66252800->66256895] (req 66252800->66256895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.425003:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c647f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66252800->66256895] (req 66252800->66256895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.425021:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c647f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66252800->66256895] (req 66252800->66256895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.425029:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760788.425034:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c647f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66252800->66256895] (req 66252800->66256895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.425043:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c647f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66252800->66256895] (req 66252800->66256895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.425062:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c647f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66252800->66256895] (req 66252800->66256895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.425069:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c647f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66252800->66256895] (req 66252800->66256895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.425170:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554837120:12345-192.168.203.22@tcp:4:dd.0 Request processed in 3986us (4242us total) trans 4294983484 rc 0/0 00000100:00100000:2.0:1789760788.425180:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16186 00000100:00100000:2.0:1789760788.425189:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731657695656 time 731666785694 state 0x2 ref 0 00000100:00100000:2.0:1789760788.433663:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760788.433811:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554837248 00000100:00100000:0.0:1789760788.433822:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554837248 00000100:00100000:0.0:1789760788.433878:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731657695656 time 731666785694 state 0x0 ref 1 00000100:00100000:0.0:1789760788.433899:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731667046110 time 731666785694 state 0x1 ref 1 00000100:00100000:0.0:1789760788.433934:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731667046110 time 731679552110 state 0x0 ref 1 00000100:00100000:0.0:1789760788.433948:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0985b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554837248:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760788.433995:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66256896, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f998 00010000:00010000:0.0:1789760788.434019:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6486 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.434036:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760788.434041:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.434047:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6486 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66256896->66260991] (req 66256896->66260991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.434061:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6486 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66256896->66260991] (req 66256896->66260991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.438336:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983485 is committed 00002000:00010000:1.0:1789760788.438749:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66256896 -> 66260992 00002000:00010000:1.0:1789760788.438755:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129432 -> 129440 00010000:00010000:1.0:1789760788.438768:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6486 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66256896->66260991] (req 66256896->66260991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.438781:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6486 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66256896->66260991] (req 66256896->66260991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.438790:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6486 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66256896->66260991] (req 66256896->66260991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.438798:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760788.438803:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6486 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66256896->66260991] (req 66256896->66260991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.438818:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6486 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66256896->66260991] (req 66256896->66260991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.438839:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6486 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66256896->66260991] (req 66256896->66260991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.438847:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6486 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66256896->66260991] (req 66256896->66260991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.438977:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0985b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554837248:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5027us (5319us total) trans 4294983485 rc 0/0 00000100:00100000:1.0:1789760788.438988:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16187 00000100:00100000:1.0:1789760788.438998:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731667046110 time 731679552110 state 0x2 ref 0 00000100:00100000:1.0:1789760788.444137:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760788.444231:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554837376 00000100:00100000:3.0:1789760788.444243:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554837376 00000100:00100000:3.0:1789760788.444293:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731667046110 time 731679552110 state 0x0 ref 1 00000100:00100000:3.0:1789760788.444307:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731679812526 time 731679552110 state 0x1 ref 1 00000100:00100000:3.0:1789760788.444374:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731679812526 time 731689992401 state 0x0 ref 1 00000100:00100000:3.0:1789760788.444502:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554837376:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760788.444543:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66260992, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f9a0 00010000:00010000:3.0:1789760788.444552:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c648d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.444568:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760788.444573:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.444579:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c648d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66260992->66265087] (req 66260992->66265087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.444591:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c648d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66260992->66265087] (req 66260992->66265087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.448093:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983486 is committed 00002000:00010000:3.0:1789760788.448205:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66260992 -> 66265088 00002000:00010000:3.0:1789760788.448210:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129440 -> 129448 00010000:00010000:3.0:1789760788.448224:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c648d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66260992->66265087] (req 66260992->66265087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.448236:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c648d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66260992->66265087] (req 66260992->66265087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.448245:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c648d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66260992->66265087] (req 66260992->66265087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.448251:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760788.448257:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c648d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66260992->66265087] (req 66260992->66265087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.448266:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c648d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66260992->66265087] (req 66260992->66265087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.448285:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c648d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66260992->66265087] (req 66260992->66265087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.448291:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c648d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66260992->66265087] (req 66260992->66265087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.448404:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554837376:12345-192.168.203.22@tcp:4:dd.0 Request processed in 3875us (4246us total) trans 4294983486 rc 0/0 00000100:00100000:3.0:1789760788.448416:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16188 00000100:00100000:3.0:1789760788.448426:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731679812526 time 731689992401 state 0x2 ref 0 00000100:00100000:1.0:1789760788.452180:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760788.452658:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554837504 00000100:00100000:3.0:1789760788.452668:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554837504 00000100:00100000:3.0:1789760788.452721:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731679812526 time 731689992401 state 0x0 ref 1 00000100:00100000:3.0:1789760788.452738:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731690252817 time 731689992401 state 0x1 ref 1 00000100:00100000:3.0:1789760788.452764:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731690252817 time 731698382442 state 0x0 ref 1 00000100:00100000:3.0:1789760788.452777:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554837504:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760788.452828:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66265088, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f9a8 00010000:00010000:3.0:1789760788.452838:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6494 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.452857:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760788.452861:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.452868:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6494 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66265088->66269183] (req 66265088->66269183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.452882:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6494 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66265088->66269183] (req 66265088->66269183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.456261:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983487 is committed 00002000:00010000:3.0:1789760788.456370:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66265088 -> 66269184 00002000:00010000:3.0:1789760788.456455:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129448 -> 129456 00010000:00010000:3.0:1789760788.456469:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6494 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66265088->66269183] (req 66265088->66269183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.456480:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6494 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66265088->66269183] (req 66265088->66269183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.456489:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6494 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66265088->66269183] (req 66265088->66269183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.456496:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760788.456503:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6494 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66265088->66269183] (req 66265088->66269183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.456512:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6494 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66265088->66269183] (req 66265088->66269183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.456529:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6494 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66265088->66269183] (req 66265088->66269183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.456535:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6494 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66265088->66269183] (req 66265088->66269183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.456667:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554837504:12345-192.168.203.22@tcp:4:dd.0 Request processed in 3888us (4492us total) trans 4294983487 rc 0/0 00000100:00100000:3.0:1789760788.456678:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16189 00000100:00100000:3.0:1789760788.456690:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731690252817 time 731698382442 state 0x2 ref 0 00000100:00100000:1.0:1789760788.460412:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760788.460503:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554837632 00000100:00100000:2.0:1789760788.460512:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554837632 00000100:00100000:2.0:1789760788.460563:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731690252817 time 731698382442 state 0x0 ref 1 00000100:00100000:2.0:1789760788.460579:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731698642858 time 731698382442 state 0x1 ref 1 00000100:00100000:2.0:1789760788.460605:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731698642858 time 731706223121 state 0x0 ref 1 00000100:00100000:2.0:1789760788.460616:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554837632:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760788.460680:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66269184, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f9b0 00010000:00010000:2.0:1789760788.460690:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c649b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.460705:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760788.460709:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.460716:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c649b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66269184->66273279] (req 66269184->66273279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.460729:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c649b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66269184->66273279] (req 66269184->66273279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.463999:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983488 is committed 00002000:00010000:2.0:1789760788.464160:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66269184 -> 66273280 00002000:00010000:2.0:1789760788.464165:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129456 -> 129464 00010000:00010000:2.0:1789760788.464178:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c649b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66269184->66273279] (req 66269184->66273279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.464197:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c649b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66269184->66273279] (req 66269184->66273279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.464206:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c649b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66269184->66273279] (req 66269184->66273279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.464214:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760788.464218:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c649b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66269184->66273279] (req 66269184->66273279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.464227:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c649b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66269184->66273279] (req 66269184->66273279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.464246:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c649b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66269184->66273279] (req 66269184->66273279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.464254:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c649b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66269184->66273279] (req 66269184->66273279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.464357:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554837632:12345-192.168.203.22@tcp:4:dd.0 Request processed in 3737us (3987us total) trans 4294983488 rc 0/0 00000100:00100000:2.0:1789760788.464368:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16190 00000100:00100000:2.0:1789760788.464414:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731698642858 time 731706223121 state 0x2 ref 0 00000100:00100000:1.0:1789760788.470069:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760788.470170:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554837760 00000100:00100000:0.0:1789760788.470180:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554837760 00000100:00100000:0.0:1789760788.470233:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731698642858 time 731706223121 state 0x0 ref 1 00000100:00100000:0.0:1789760788.470249:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731706483537 time 731706223121 state 0x1 ref 1 00000100:00100000:0.0:1789760788.470279:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731706483537 time 731715897761 state 0x0 ref 1 00000100:00100000:0.0:1789760788.470295:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554837760:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760788.470339:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66273280, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f9b8 00010000:00010000:0.0:1789760788.470348:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64a2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.470364:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760788.470368:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.470374:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66273280->66277375] (req 66273280->66277375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.470431:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66273280->66277375] (req 66273280->66277375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.474347:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983489 is committed 00002000:00010000:0.0:1789760788.474806:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66273280 -> 66277376 00002000:00010000:0.0:1789760788.474812:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129464 -> 129472 00010000:00010000:0.0:1789760788.474827:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66273280->66277375] (req 66273280->66277375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.474841:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66273280->66277375] (req 66273280->66277375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.474850:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66273280->66277375] (req 66273280->66277375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.474857:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760788.474863:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66273280->66277375] (req 66273280->66277375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.474872:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64a2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66273280->66277375] (req 66273280->66277375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.474891:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64a2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66273280->66277375] (req 66273280->66277375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.474899:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64a2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66273280->66277375] (req 66273280->66277375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.475146:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554837760:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4847us (5081us total) trans 4294983489 rc 0/0 00000100:00100000:0.0:1789760788.475158:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16191 00000100:00100000:0.0:1789760788.475168:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731706483537 time 731715897761 state 0x2 ref 0 00000100:00100000:1.0:1789760788.478442:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760788.478605:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554837888 00000100:00100000:3.0:1789760788.478615:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554837888 00000100:00100000:3.0:1789760788.478792:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731706483537 time 731715897761 state 0x0 ref 1 00000100:00100000:3.0:1789760788.478810:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731716158177 time 731715897761 state 0x1 ref 1 00000100:00100000:3.0:1789760788.478841:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731716158177 time 731724459351 state 0x0 ref 1 00000100:00100000:3.0:1789760788.478853:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554837888:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760788.478899:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66277376, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f9c0 00010000:00010000:3.0:1789760788.478908:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64a9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.478924:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760788.478938:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.478945:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66277376->66281471] (req 66277376->66281471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.478959:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66277376->66281471] (req 66277376->66281471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.482756:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983490 is committed 00002000:00010000:3.0:1789760788.482870:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66277376 -> 66281472 00002000:00010000:3.0:1789760788.482875:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129472 -> 129480 00010000:00010000:3.0:1789760788.482887:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66277376->66281471] (req 66277376->66281471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.482903:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66277376->66281471] (req 66277376->66281471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.482910:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66277376->66281471] (req 66277376->66281471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.482917:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760788.482920:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66277376->66281471] (req 66277376->66281471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.482999:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66277376->66281471] (req 66277376->66281471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.483030:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64a9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66277376->66281471] (req 66277376->66281471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.483037:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64a9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66277376->66281471] (req 66277376->66281471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760788.483164:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554837888:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4308us (4725us total) trans 4294983490 rc 0/0 00000100:00100000:3.0:1789760788.483174:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16192 00000100:00100000:3.0:1789760788.483183:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731716158177 time 731724459351 state 0x2 ref 0 00000100:00100000:1.0:1789760788.489831:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760788.490280:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554838016 00000100:00100000:3.0:1789760788.490290:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554838016 00000100:00100000:3.0:1789760788.490342:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731716158177 time 731724459351 state 0x0 ref 1 00000100:00100000:3.0:1789760788.490356:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731724719767 time 731724459351 state 0x1 ref 1 00000100:00100000:3.0:1789760788.490471:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731724719767 time 731736088155 state 0x0 ref 1 00000100:00100000:3.0:1789760788.490484:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554838016:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760788.490531:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66281472, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f9c8 00010000:00010000:3.0:1789760788.490541:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64b0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.490556:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760788.490560:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.490566:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66281472->66285567] (req 66281472->66285567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.490578:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66281472->66285567] (req 66281472->66285567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.495118:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983491 is committed 00002000:00010000:3.0:1789760788.495231:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66281472 -> 66285568 00002000:00010000:3.0:1789760788.495236:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129480 -> 129488 00010000:00010000:3.0:1789760788.495249:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66281472->66285567] (req 66281472->66285567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.495262:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64b0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66281472->66285567] (req 66281472->66285567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.495277:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66281472->66285567] (req 66281472->66285567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.495285:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760788.495290:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66281472->66285567] (req 66281472->66285567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.495298:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66281472->66285567] (req 66281472->66285567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.495317:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64b0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66281472->66285567] (req 66281472->66285567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.495325:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64b0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66281472->66285567] (req 66281472->66285567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.495711:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554838016:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5224us (5884us total) trans 4294983491 rc 0/0 00000100:00100000:0.0:1789760788.495723:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16193 00000100:00100000:0.0:1789760788.495732:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731724719767 time 731736088155 state 0x2 ref 0 00000100:00100000:1.0:1789760788.501205:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760788.501332:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554838144 00000100:00100000:2.0:1789760788.501341:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554838144 00000100:00100000:2.0:1789760788.501631:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731724719767 time 731736088155 state 0x0 ref 1 00000100:00100000:2.0:1789760788.501657:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731736348571 time 731736088155 state 0x1 ref 1 00000100:00100000:2.0:1789760788.501739:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731736348571 time 731747356115 state 0x0 ref 1 00000100:00100000:2.0:1789760788.501753:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554838144:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760788.501797:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66285568, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f9d0 00010000:00010000:2.0:1789760788.501805:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.501820:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760788.501824:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.501829:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66285568->66289663] (req 66285568->66289663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.501843:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66285568->66289663] (req 66285568->66289663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.506098:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983492 is committed 00002000:00010000:2.0:1789760788.506697:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66285568 -> 66289664 00002000:00010000:2.0:1789760788.506702:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129488 -> 129496 00010000:00010000:2.0:1789760788.506716:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66285568->66289663] (req 66285568->66289663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.506729:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66285568->66289663] (req 66285568->66289663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.506739:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66285568->66289663] (req 66285568->66289663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.506747:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760788.506751:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66285568->66289663] (req 66285568->66289663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.506760:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64b7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66285568->66289663] (req 66285568->66289663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.506779:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64b7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66285568->66289663] (req 66285568->66289663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.506788:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64b7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66285568->66289663] (req 66285568->66289663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.506875:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554838144:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5121us (5675us total) trans 4294983492 rc 0/0 00000100:00100000:2.0:1789760788.506885:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16194 00000100:00100000:2.0:1789760788.506896:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731736348571 time 731747356115 state 0x2 ref 0 00000100:00100000:1.0:1789760788.511696:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760788.511879:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554838272 00000100:00100000:0.0:1789760788.511888:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554838272 00000100:00100000:0.0:1789760788.511939:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731736348571 time 731747356115 state 0x0 ref 1 00000100:00100000:0.0:1789760788.511953:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731747616531 time 731747356115 state 0x1 ref 1 00000100:00100000:0.0:1789760788.511985:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731747616531 time 731757603346 state 0x0 ref 1 00000100:00100000:0.0:1789760788.511997:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a386a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554838272:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760788.512043:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66289664, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f9d8 00010000:00010000:0.0:1789760788.512052:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64be lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.512068:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760788.512071:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.512078:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66289664->66293759] (req 66289664->66293759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.512091:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66289664->66293759] (req 66289664->66293759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.517031:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983493 is committed 00002000:00010000:0.0:1789760788.517184:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66289664 -> 66293760 00002000:00010000:0.0:1789760788.517189:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129496 -> 129504 00010000:00010000:0.0:1789760788.517202:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66289664->66293759] (req 66289664->66293759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.517214:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64be lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66289664->66293759] (req 66289664->66293759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.517224:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66289664->66293759] (req 66289664->66293759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.517237:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760788.517242:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66289664->66293759] (req 66289664->66293759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.517250:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64be lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66289664->66293759] (req 66289664->66293759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.517269:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64be lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66289664->66293759] (req 66289664->66293759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.517277:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64be lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66289664->66293759] (req 66289664->66293759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.517369:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a386a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554838272:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5368us (5677us total) trans 4294983493 rc 0/0 00000100:00100000:0.0:1789760788.517767:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16195 00000100:00100000:0.0:1789760788.517776:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731747616531 time 731757603346 state 0x2 ref 0 00000100:00100000:1.0:1789760788.521359:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760788.521751:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554838400 00000100:00100000:3.0:1789760788.521761:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554838400 00000100:00100000:3.0:1789760788.521816:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731747616531 time 731757603346 state 0x0 ref 1 00000100:00100000:3.0:1789760788.521832:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731757863762 time 731757603346 state 0x1 ref 1 00000100:00100000:3.0:1789760788.521913:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731757863762 time 731767530630 state 0x0 ref 1 00000100:00100000:3.0:1789760788.521926:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a386300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554838400:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760788.521974:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66293760, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f9e0 00010000:00010000:3.0:1789760788.521983:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64c5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.522008:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:3.0:1789760788.522118:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760788.522125:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66293760->66297855] (req 66293760->66297855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760788.522149:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66293760->66297855] (req 66293760->66297855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.525973:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983494 is committed 00002000:00010000:0.0:1789760788.527815:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66293760 -> 66297856 00002000:00010000:0.0:1789760788.527824:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129504 -> 129512 00010000:00010000:0.0:1789760788.527836:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66293760->66297855] (req 66293760->66297855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.527850:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64c5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66293760->66297855] (req 66293760->66297855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.527864:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66293760->66297855] (req 66293760->66297855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.527871:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760788.527876:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66293760->66297855] (req 66293760->66297855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.527885:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64c5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66293760->66297855] (req 66293760->66297855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.527904:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64c5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66293760->66297855] (req 66293760->66297855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.527911:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c64c5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66293760->66297855] (req 66293760->66297855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.528160:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a386300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554838400:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6230us (6806us total) trans 4294983494 rc 0/0 00000100:00100000:0.0:1789760788.528171:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16196 00000100:00100000:0.0:1789760788.528181:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731757863762 time 731767530630 state 0x2 ref 0 00000100:00100000:1.0:1789760788.540195:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760788.540293:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554838528 00000100:00100000:0.0:1789760788.540304:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554838528 00000100:00100000:0.0:1789760788.540354:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731757863762 time 731767530630 state 0x0 ref 1 00000100:00100000:0.0:1789760788.540370:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731767791046 time 731767530630 state 0x1 ref 1 00000100:00100000:0.0:1789760788.540434:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731767791046 time 731786052351 state 0x0 ref 1 00000100:00100000:0.0:1789760788.540447:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe1a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554838528:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760788.540497:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66297856, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f9e8 00010000:00010000:0.0:1789760788.540507:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64cc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.540522:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760788.540526:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.540532:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66297856->66301951] (req 66297856->66301951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.540546:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66297856->66301951] (req 66297856->66301951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.543649:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983495 is committed 00002000:00010000:0.0:1789760788.543785:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66297856 -> 66301952 00002000:00010000:0.0:1789760788.543790:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129512 -> 129520 00010000:00010000:0.0:1789760788.543802:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66297856->66301951] (req 66297856->66301951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.543814:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64cc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66297856->66301951] (req 66297856->66301951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.543823:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66297856->66301951] (req 66297856->66301951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.543836:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760788.543842:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66297856->66301951] (req 66297856->66301951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.543851:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64cc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66297856->66301951] (req 66297856->66301951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.543870:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64cc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66297856->66301951] (req 66297856->66301951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.543878:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64cc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66297856->66301951] (req 66297856->66301951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.543952:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe1a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554838528:12345-192.168.203.22@tcp:4:dd.0 Request processed in 3502us (3760us total) trans 4294983495 rc 0/0 00000100:00100000:0.0:1789760788.543962:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16197 00000100:00100000:0.0:1789760788.543971:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731767791046 time 731786052351 state 0x2 ref 0 00000100:00100000:1.0:1789760788.547881:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760788.550304:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554838656 00000100:00100000:3.0:1789760788.550320:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554838656 00000100:00100000:2.0:1789760788.550767:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731767791046 time 731786052351 state 0x0 ref 1 00000100:00100000:2.0:1789760788.550784:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731786312767 time 731786052351 state 0x1 ref 1 00000100:00100000:2.0:1789760788.550834:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731786312767 time 731796442805 state 0x0 ref 1 00000100:00100000:2.0:1789760788.550849:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554838656:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760788.550897:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66301952, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f9f0 00010000:00010000:2.0:1789760788.550906:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64d3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.550922:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760788.550927:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.550933:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66301952->66306047] (req 66301952->66306047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.550947:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66301952->66306047] (req 66301952->66306047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.554742:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983496 is committed 00002000:00010000:2.0:1789760788.557524:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66301952 -> 66306048 00002000:00010000:2.0:1789760788.557533:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129520 -> 129528 00010000:00010000:2.0:1789760788.557546:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66301952->66306047] (req 66301952->66306047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.557560:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64d3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66301952->66306047] (req 66301952->66306047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.557569:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66301952->66306047] (req 66301952->66306047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.557583:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760788.557587:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66301952->66306047] (req 66301952->66306047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.557596:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64d3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66301952->66306047] (req 66301952->66306047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.557614:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64d3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66301952->66306047] (req 66301952->66306047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.557621:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64d3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66301952->66306047] (req 66301952->66306047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.557722:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554838656:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6872us (9845us total) trans 4294983496 rc 0/0 00000100:00100000:2.0:1789760788.557733:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16198 00000100:00100000:2.0:1789760788.557742:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731786312767 time 731796442805 state 0x2 ref 0 00000100:00100000:1.0:1789760788.562713:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760788.562790:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554838784 00000100:00100000:0.0:1789760788.562798:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554838784 00000100:00100000:0.0:1789760788.562846:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731786312767 time 731796442805 state 0x0 ref 1 00000100:00100000:0.0:1789760788.562860:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731796703221 time 731796442805 state 0x1 ref 1 00000100:00100000:0.0:1789760788.562888:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731796703221 time 731808507017 state 0x0 ref 1 00000100:00100000:0.0:1789760788.562902:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a387480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554838784:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760788.562943:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66306048, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1f9f8 00010000:00010000:0.0:1789760788.562952:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64da lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.562967:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760788.562971:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.562977:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66306048->66310143] (req 66306048->66310143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.562990:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66306048->66310143] (req 66306048->66310143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.566144:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983497 is committed 00002000:00010000:0.0:1789760788.566243:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66306048 -> 66310144 00002000:00010000:0.0:1789760788.566248:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129528 -> 129536 00010000:00010000:0.0:1789760788.566259:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66306048->66310143] (req 66306048->66310143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.566271:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64da lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66306048->66310143] (req 66306048->66310143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.566280:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66306048->66310143] (req 66306048->66310143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.566287:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760788.566292:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66306048->66310143] (req 66306048->66310143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.566301:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64da lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66306048->66310143] (req 66306048->66310143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.566319:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64da lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66306048->66310143] (req 66306048->66310143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.566327:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64da lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66306048->66310143] (req 66306048->66310143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.566455:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a387480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554838784:12345-192.168.203.22@tcp:4:dd.0 Request processed in 3549us (3745us total) trans 4294983497 rc 0/0 00000100:00100000:0.0:1789760788.566466:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16199 00000100:00100000:0.0:1789760788.566476:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731796703221 time 731808507017 state 0x2 ref 0 00000100:00100000:1.0:1789760788.572190:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760788.572286:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554838912 00000100:00100000:0.0:1789760788.572296:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554838912 00000100:00100000:0.0:1789760788.572350:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731796703221 time 731808507017 state 0x0 ref 1 00000100:00100000:0.0:1789760788.572364:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731808767433 time 731808507017 state 0x1 ref 1 00000100:00100000:0.0:1789760788.572419:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731808767433 time 731818036846 state 0x0 ref 1 00000100:00100000:0.0:1789760788.572432:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a387b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554838912:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760788.572478:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66310144, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1fa00 00010000:00010000:0.0:1789760788.572488:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64e1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.572503:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760788.572507:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.572513:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66310144->66314239] (req 66310144->66314239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.572527:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66310144->66314239] (req 66310144->66314239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.576626:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983498 is committed 00002000:00010000:0.0:1789760788.578555:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66310144 -> 66314240 00002000:00010000:0.0:1789760788.578564:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129536 -> 129544 00010000:00010000:0.0:1789760788.578578:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66310144->66314239] (req 66310144->66314239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.578590:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64e1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66310144->66314239] (req 66310144->66314239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.578598:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66310144->66314239] (req 66310144->66314239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.578606:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760788.578611:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66310144->66314239] (req 66310144->66314239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.578620:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64e1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66310144->66314239] (req 66310144->66314239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.578649:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64e1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66310144->66314239] (req 66310144->66314239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.578656:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64e1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66310144->66314239] (req 66310144->66314239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.578827:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a387b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554838912:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6392us (6641us total) trans 4294983498 rc 0/0 00000100:00100000:0.0:1789760788.578837:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16200 00000100:00100000:0.0:1789760788.578846:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731808767433 time 731818036846 state 0x2 ref 0 00000100:00100000:1.0:1789760788.586653:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760788.586751:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554839040 00000100:00100000:0.0:1789760788.586761:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554839040 00000100:00100000:0.0:1789760788.586814:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731808767433 time 731818036846 state 0x0 ref 1 00000100:00100000:0.0:1789760788.586830:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731818297262 time 731818036846 state 0x1 ref 1 00000100:00100000:0.0:1789760788.586860:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731818297262 time 731832478561 state 0x0 ref 1 00000100:00100000:0.0:1789760788.586872:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a384a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554839040:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760788.586918:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66314240, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1fa08 00010000:00010000:0.0:1789760788.586928:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64e8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.586943:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760788.586948:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.586953:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66314240->66318335] (req 66314240->66318335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.586968:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66314240->66318335] (req 66314240->66318335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.590338:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983499 is committed 00002000:00010000:0.0:1789760788.590469:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66314240 -> 66318336 00002000:00010000:0.0:1789760788.590474:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129544 -> 129552 00010000:00010000:0.0:1789760788.590486:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66314240->66318335] (req 66314240->66318335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.590499:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64e8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66314240->66318335] (req 66314240->66318335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.590508:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66314240->66318335] (req 66314240->66318335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.590515:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760788.590519:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66314240->66318335] (req 66314240->66318335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.590528:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64e8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66314240->66318335] (req 66314240->66318335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.590547:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64e8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66314240->66318335] (req 66314240->66318335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.590559:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64e8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66314240->66318335] (req 66314240->66318335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.590649:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a384a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554839040:12345-192.168.203.22@tcp:4:dd.0 Request processed in 3764us (4001us total) trans 4294983499 rc 0/0 00000100:00100000:0.0:1789760788.590659:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16201 00000100:00100000:0.0:1789760788.590668:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731818297262 time 731832478561 state 0x2 ref 0 00000100:00100000:1.0:1789760788.595212:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760788.596881:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554839168 00000100:00100000:2.0:1789760788.596894:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554839168 00000100:00100000:2.0:1789760788.596957:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731818297262 time 731832478561 state 0x0 ref 1 00000100:00100000:2.0:1789760788.596973:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731832738977 time 731832478561 state 0x1 ref 1 00000100:00100000:2.0:1789760788.597131:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731832738977 time 731842748480 state 0x0 ref 1 00000100:00100000:2.0:1789760788.597149:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554839168:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760788.597199:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66318336, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1fa10 00010000:00010000:2.0:1789760788.597598:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64ef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.597615:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760788.597619:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.597626:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66318336->66322431] (req 66318336->66322431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.597649:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66318336->66322431] (req 66318336->66322431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.600727:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983500 is committed 00002000:00010000:2.0:1789760788.600844:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66318336 -> 66322432 00002000:00010000:2.0:1789760788.600850:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129552 -> 129560 00010000:00010000:2.0:1789760788.600865:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66318336->66322431] (req 66318336->66322431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.600879:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64ef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66318336->66322431] (req 66318336->66322431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.600888:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66318336->66322431] (req 66318336->66322431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.600894:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760788.600900:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66318336->66322431] (req 66318336->66322431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.600908:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64ef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66318336->66322431] (req 66318336->66322431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.600929:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64ef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66318336->66322431] (req 66318336->66322431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.600937:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c64ef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66318336->66322431] (req 66318336->66322431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.601034:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554839168:12345-192.168.203.22@tcp:4:dd.0 Request processed in 3884us (5827us total) trans 4294983500 rc 0/0 00000100:00100000:2.0:1789760788.601043:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16202 00000100:00100000:2.0:1789760788.601053:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731832738977 time 731842748480 state 0x2 ref 0 00000100:00100000:1.0:1789760788.605004:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760788.605670:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554839296 00000100:00100000:0.0:1789760788.605681:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554839296 00000100:00100000:0.0:1789760788.605737:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731832738977 time 731842748480 state 0x0 ref 1 00000100:00100000:0.0:1789760788.605754:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731843008896 time 731842748480 state 0x1 ref 1 00000100:00100000:0.0:1789760788.605788:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731843008896 time 731851406491 state 0x0 ref 1 00000100:00100000:0.0:1789760788.605802:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a386680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554839296:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760788.605852:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66322432, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1fa18 00010000:00010000:0.0:1789760788.605862:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64f6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.605878:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760788.605882:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.605889:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66322432->66326527] (req 66322432->66326527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.605909:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66322432->66326527] (req 66322432->66326527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.614585:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983501 is committed 00002000:00010000:1.0:1789760788.614784:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66322432 -> 66326528 00002000:00010000:1.0:1789760788.614789:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129560 -> 129568 00010000:00010000:1.0:1789760788.614803:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66322432->66326527] (req 66322432->66326527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.614816:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64f6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66322432->66326527] (req 66322432->66326527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.614825:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66322432->66326527] (req 66322432->66326527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.614836:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760788.614842:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66322432->66326527] (req 66322432->66326527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.614850:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64f6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66322432->66326527] (req 66322432->66326527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.614869:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64f6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66322432->66326527] (req 66322432->66326527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.614876:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c64f6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66322432->66326527] (req 66322432->66326527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.615031:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a386680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554839296:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9226us (10030us total) trans 4294983501 rc 0/0 00000100:00100000:1.0:1789760788.615043:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16203 00000100:00100000:1.0:1789760788.615052:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731843008896 time 731851406491 state 0x2 ref 0 00000100:00100000:1.0:1789760788.621814:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760788.621940:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554839424 00000100:00100000:0.0:1789760788.621951:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554839424 00000100:00100000:0.0:1789760788.622006:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731843008896 time 731851406491 state 0x0 ref 1 00000100:00100000:0.0:1789760788.622034:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731851666907 time 731851406491 state 0x1 ref 1 00000100:00100000:0.0:1789760788.622065:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731851666907 time 731867684375 state 0x0 ref 1 00000100:00100000:0.0:1789760788.622078:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554839424:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760788.622125:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66326528, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1fa20 00010000:00010000:0.0:1789760788.622135:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c64fd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.622151:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760788.622155:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.622162:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c64fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66326528->66330623] (req 66326528->66330623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.622176:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c64fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66326528->66330623] (req 66326528->66330623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.629173:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983502 is committed 00002000:00010000:0.0:1789760788.630308:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66326528 -> 66330624 00002000:00010000:0.0:1789760788.630316:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129568 -> 129576 00010000:00010000:0.0:1789760788.630331:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c64fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66326528->66330623] (req 66326528->66330623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.630344:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c64fd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66326528->66330623] (req 66326528->66330623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.630353:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c64fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66326528->66330623] (req 66326528->66330623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.630360:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760788.630365:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c64fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66326528->66330623] (req 66326528->66330623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.630373:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c64fd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66326528->66330623] (req 66326528->66330623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.630440:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c64fd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66326528->66330623] (req 66326528->66330623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.630448:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c64fd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66326528->66330623] (req 66326528->66330623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.630540:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554839424:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8460us (8731us total) trans 4294983502 rc 0/0 00000100:00100000:0.0:1789760788.630550:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16204 00000100:00100000:0.0:1789760788.630559:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731851666907 time 731867684375 state 0x2 ref 0 00000100:00100000:2.0:1789760788.637421:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760788.637514:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554839552 00000100:00100000:1.0:1789760788.637524:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554839552 00000100:00100000:1.0:1789760788.637580:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731851666907 time 731867684375 state 0x0 ref 1 00000100:00100000:1.0:1789760788.637594:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731867944791 time 731867684375 state 0x1 ref 1 00000100:00100000:1.0:1789760788.637617:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731867944791 time 731883235387 state 0x0 ref 1 00000100:00100000:1.0:1789760788.637630:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0985a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554839552:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760788.637692:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66330624, mtime: 0x6aad9513, atime: 0x0, ctime: 0x6aad9513, blocks: 0x1fa28 00010000:00010000:1.0:1789760788.637701:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6504 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.637720:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760788.637725:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.637736:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6504 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66330624->66334719] (req 66330624->66334719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.637752:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6504 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66330624->66334719] (req 66330624->66334719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.644148:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983503 is committed 00002000:00010000:1.0:1789760788.645024:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66330624 -> 66334720 00002000:00010000:1.0:1789760788.645030:0:8442:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760787 -> 1789760788 00002000:00010000:1.0:1789760788.645033:0:8442:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760787 -> 1789760788 00002000:00010000:1.0:1789760788.645036:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129576 -> 129584 00010000:00010000:1.0:1789760788.645050:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6504 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66330624->66334719] (req 66330624->66334719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.645063:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6504 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66330624->66334719] (req 66330624->66334719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.645071:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6504 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66330624->66334719] (req 66330624->66334719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.645084:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760788.645088:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6504 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66330624->66334719] (req 66330624->66334719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.645097:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6504 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66330624->66334719] (req 66330624->66334719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.645116:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6504 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66330624->66334719] (req 66330624->66334719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.645124:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6504 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66330624->66334719] (req 66330624->66334719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.645223:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0985a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554839552:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7590us (7844us total) trans 4294983503 rc 0/0 00000100:00100000:1.0:1789760788.645232:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16205 00000100:00100000:1.0:1789760788.645241:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731867944791 time 731883235387 state 0x2 ref 0 00000100:00100000:1.0:1789760788.654475:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760788.654690:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554839680 00000100:00100000:2.0:1789760788.654701:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554839680 00000100:00100000:2.0:1789760788.654755:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731867944791 time 731883235387 state 0x0 ref 1 00000100:00100000:2.0:1789760788.654770:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731883495803 time 731883235387 state 0x1 ref 1 00000100:00100000:2.0:1789760788.654808:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731883495803 time 731900426516 state 0x0 ref 1 00000100:00100000:2.0:1789760788.654825:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554839680:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760788.654874:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66334720, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fa30 00010000:00010000:2.0:1789760788.654884:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c650b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.654901:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760788.654906:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.654912:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c650b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66334720->66338815] (req 66334720->66338815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.654927:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c650b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66334720->66338815] (req 66334720->66338815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.668506:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983504 is committed 00002000:00010000:2.0:1789760788.668755:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66334720 -> 66338816 00002000:00010000:2.0:1789760788.668760:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129584 -> 129592 00010000:00010000:2.0:1789760788.668773:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c650b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66334720->66338815] (req 66334720->66338815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.668786:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c650b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66334720->66338815] (req 66334720->66338815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.668795:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c650b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66334720->66338815] (req 66334720->66338815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.668802:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760788.668808:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c650b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66334720->66338815] (req 66334720->66338815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.668816:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c650b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66334720->66338815] (req 66334720->66338815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.668835:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c650b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66334720->66338815] (req 66334720->66338815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.668842:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c650b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66334720->66338815] (req 66334720->66338815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.668949:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554839680:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14121us (14477us total) trans 4294983504 rc 0/0 00000100:00100000:2.0:1789760788.668960:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16206 00000100:00100000:2.0:1789760788.668970:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731883495803 time 731900426516 state 0x2 ref 0 00000100:00100000:0.0:1789760788.676713:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760788.677028:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554839808 00000100:00100000:1.0:1789760788.677038:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554839808 00000100:00100000:1.0:1789760788.677091:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731883495803 time 731900426516 state 0x0 ref 1 00000100:00100000:1.0:1789760788.677107:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731900686932 time 731900426516 state 0x1 ref 1 00000100:00100000:1.0:1789760788.677179:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731900686932 time 731922796633 state 0x0 ref 1 00000100:00100000:1.0:1789760788.677191:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554839808:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760788.677235:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66338816, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fa38 00010000:00010000:1.0:1789760788.677243:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6512 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.677259:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:1.0:1789760788.677263:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.677270:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6512 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66338816->66342911] (req 66338816->66342911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.677283:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6512 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66338816->66342911] (req 66338816->66342911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.682802:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983505 is committed 00002000:00010000:1.0:1789760788.682905:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66338816 -> 66342912 00002000:00010000:1.0:1789760788.682910:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129592 -> 129600 00010000:00010000:1.0:1789760788.682922:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6512 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66338816->66342911] (req 66338816->66342911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.682934:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6512 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66338816->66342911] (req 66338816->66342911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.682942:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6512 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66338816->66342911] (req 66338816->66342911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.682954:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655de00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760788.682959:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6512 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66338816->66342911] (req 66338816->66342911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.682969:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6512 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66338816->66342911] (req 66338816->66342911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.682988:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6512 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66338816->66342911] (req 66338816->66342911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.682995:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6512 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66338816->66342911] (req 66338816->66342911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.683101:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554839808:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5907us (6393us total) trans 4294983505 rc 0/0 00000100:00100000:1.0:1789760788.683112:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16207 00000100:00100000:1.0:1789760788.683121:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731900686932 time 731922796633 state 0x2 ref 0 00000100:00100000:1.0:1789760788.686628:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760788.686760:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554839936 00000100:00100000:0.0:1789760788.686770:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554839936 00000100:00100000:0.0:1789760788.686822:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731900686932 time 731922796633 state 0x0 ref 1 00000100:00100000:0.0:1789760788.686837:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731923057049 time 731922796633 state 0x1 ref 1 00000100:00100000:0.0:1789760788.686870:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731923057049 time 731932488404 state 0x0 ref 1 00000100:00100000:0.0:1789760788.686880:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a384e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554839936:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760788.686925:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66342912, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fa40 00010000:00010000:0.0:1789760788.686935:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6519 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.686950:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760788.686954:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.686961:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6519 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66342912->66347007] (req 66342912->66347007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.686975:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6519 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66342912->66347007] (req 66342912->66347007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.691924:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983506 is committed 00002000:00010000:0.0:1789760788.692045:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66342912 -> 66347008 00002000:00010000:0.0:1789760788.692051:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129600 -> 129608 00010000:00010000:0.0:1789760788.692064:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6519 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66342912->66347007] (req 66342912->66347007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.692076:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6519 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66342912->66347007] (req 66342912->66347007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.692085:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6519 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66342912->66347007] (req 66342912->66347007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.692092:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760788.692096:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6519 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66342912->66347007] (req 66342912->66347007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.692105:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6519 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66342912->66347007] (req 66342912->66347007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.692124:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6519 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66342912->66347007] (req 66342912->66347007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.692132:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6519 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66342912->66347007] (req 66342912->66347007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.692224:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a384e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554839936:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5340us (5601us total) trans 4294983506 rc 0/0 00000100:00100000:0.0:1789760788.692235:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16208 00000100:00100000:0.0:1789760788.692246:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731923057049 time 731932488404 state 0x2 ref 0 00000100:00100000:3.0:1789760788.695772:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760788.696766:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554840064 00000100:00100000:0.0:1789760788.696778:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554840064 00000100:00100000:0.0:1789760788.696833:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731923057049 time 731932488404 state 0x0 ref 1 00000100:00100000:0.0:1789760788.696849:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731932748820 time 731932488404 state 0x1 ref 1 00000100:00100000:0.0:1789760788.696872:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731932748820 time 731942490593 state 0x0 ref 1 00000100:00100000:0.0:1789760788.696884:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb020b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554840064:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760788.696928:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66347008, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fa48 00010000:00010000:0.0:1789760788.696937:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6520 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.696952:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760788.696957:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.696964:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6520 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66347008->66351103] (req 66347008->66351103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.696977:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6520 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66347008->66351103] (req 66347008->66351103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.702685:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983507 is committed 00002000:00010000:0.0:1789760788.703272:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66347008 -> 66351104 00002000:00010000:0.0:1789760788.703278:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129608 -> 129616 00010000:00010000:0.0:1789760788.703293:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6520 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66347008->66351103] (req 66347008->66351103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.703311:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6520 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66347008->66351103] (req 66347008->66351103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.703320:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6520 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66347008->66351103] (req 66347008->66351103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.703327:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760788.703331:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6520 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66347008->66351103] (req 66347008->66351103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.703340:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6520 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66347008->66351103] (req 66347008->66351103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.703359:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6520 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66347008->66351103] (req 66347008->66351103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.703366:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6520 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66347008->66351103] (req 66347008->66351103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.703475:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb020b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554840064:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6588us (7709us total) trans 4294983507 rc 0/0 00000100:00100000:0.0:1789760788.703486:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16209 00000100:00100000:0.0:1789760788.703495:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731932748820 time 731942490593 state 0x2 ref 0 00000100:00100000:1.0:1789760788.712293:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760788.712430:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554840192 00000100:00100000:2.0:1789760788.712440:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554840192 00000100:00100000:2.0:1789760788.712494:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731932748820 time 731942490593 state 0x0 ref 1 00000100:00100000:2.0:1789760788.712513:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731942751009 time 731942490593 state 0x1 ref 1 00000100:00100000:2.0:1789760788.712537:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731942751009 time 731958156338 state 0x0 ref 1 00000100:00100000:2.0:1789760788.712552:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a386d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554840192:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760788.712605:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66351104, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fa50 00010000:00010000:2.0:1789760788.712615:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6527 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.712640:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:2.0:1789760788.712645:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.712652:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6527 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66351104->66355199] (req 66351104->66355199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.712666:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6527 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66351104->66355199] (req 66351104->66355199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.719575:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983508 is committed 00002000:00010000:2.0:1789760788.719760:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66351104 -> 66355200 00002000:00010000:2.0:1789760788.719766:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129616 -> 129624 00010000:00010000:2.0:1789760788.719779:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6527 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66351104->66355199] (req 66351104->66355199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.719793:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6527 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66351104->66355199] (req 66351104->66355199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.719802:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6527 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66351104->66355199] (req 66351104->66355199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.719810:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760788.719815:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6527 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66351104->66355199] (req 66351104->66355199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.719824:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6527 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66351104->66355199] (req 66351104->66355199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.719842:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6527 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66351104->66355199] (req 66351104->66355199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.719850:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6527 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66351104->66355199] (req 66351104->66355199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.719996:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a386d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554840192:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7440us (7706us total) trans 4294983508 rc 0/0 00000100:00100000:2.0:1789760788.720017:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16210 00000100:00100000:2.0:1789760788.720028:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731942751009 time 731958156338 state 0x2 ref 0 00000100:00100000:1.0:1789760788.724372:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760788.724620:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554840320 00000100:00100000:0.0:1789760788.724640:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554840320 00000100:00100000:0.0:1789760788.724691:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731942751009 time 731958156338 state 0x0 ref 1 00000100:00100000:0.0:1789760788.724705:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731958416754 time 731958156338 state 0x1 ref 1 00000100:00100000:0.0:1789760788.724733:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731958416754 time 731970352207 state 0x0 ref 1 00000100:00100000:0.0:1789760788.724744:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554840320:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760788.724786:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66355200, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fa58 00010000:00010000:0.0:1789760788.724794:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c652e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.724810:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760788.724814:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.724820:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c652e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66355200->66359295] (req 66355200->66359295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.724835:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c652e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66355200->66359295] (req 66355200->66359295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.728991:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983509 is committed 00002000:00010000:0.0:1789760788.729120:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66355200 -> 66359296 00002000:00010000:0.0:1789760788.729125:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129624 -> 129632 00010000:00010000:0.0:1789760788.729138:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c652e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66355200->66359295] (req 66355200->66359295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.729150:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c652e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66355200->66359295] (req 66355200->66359295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.729165:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c652e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66355200->66359295] (req 66355200->66359295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.729172:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760788.729177:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c652e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66355200->66359295] (req 66355200->66359295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.729186:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c652e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66355200->66359295] (req 66355200->66359295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.729206:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c652e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66355200->66359295] (req 66355200->66359295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.729214:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c652e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66355200->66359295] (req 66355200->66359295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.729317:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554840320:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4566us (4947us total) trans 4294983509 rc 0/0 00000100:00100000:0.0:1789760788.729328:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16211 00000100:00100000:0.0:1789760788.729338:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731958416754 time 731970352207 state 0x2 ref 0 00000100:00100000:3.0:1789760788.735598:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760788.735654:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760788.735696:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760788.735714:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760788.735736:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760788.735755:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760788.735780:0:11089:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554840832 00000100:00100000:3.0:1789760788.735790:0:11089:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554840832 00000100:00100000:3.0:1789760788.735847:0:11089:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554841088 00000100:00100000:3.0:1789760788.735850:0:11089:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554841088 00000100:00100000:3.0:1789760788.735864:0:11089:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554840704 00000100:00100000:3.0:1789760788.735867:0:11089:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554840704 00000100:00100000:3.0:1789760788.735880:0:11089:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554840960 00000100:00100000:3.0:1789760788.735883:0:11089:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554840960 00000100:00100000:3.0:1789760788.735900:0:11089:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 895 00000100:00100000:3.0:1789760788.735910:0:11089:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554840832:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:3.0:1789760788.735978:0:11089:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554840832:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 69us (281us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760788.735985:0:11089:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 895 00000100:00100000:3.0:1789760788.736001:0:11089:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 896 00000100:00100000:3.0:1789760788.736008:0:11089:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2414:x1876699554841088:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:2.0:1789760788.736112:0:6515:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554840576 00000100:00100000:2.0:1789760788.736117:0:6515:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554840576 00000100:00100000:2.0:1789760788.736145:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 474 00000100:00100000:2.0:1789760788.736153:0:6515:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc320b6850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9765eb0b-55b9-40b6-bfdf-ffee156c3226+13:2414:x1876699554840576:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:2.0:1789760788.736181:0:6515:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc320b6850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9765eb0b-55b9-40b6-bfdf-ffee156c3226+13:2414:x1876699554840576:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 29us (526us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760788.736187:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 474 00000100:00100000:3.0:1789760788.736265:0:11089:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2414:x1876699554841088:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 257us (549us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760788.736273:0:11089:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 896 00000100:00100000:3.0:1789760788.736286:0:11089:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 897 00000100:00100000:3.0:1789760788.736291:0:11089:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+8:2414:x1876699554840704:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:3.0:1789760788.736312:0:11089:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+8:2414:x1876699554840704:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 21us (575us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760788.736319:0:11089:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 897 00000100:00100000:3.0:1789760788.736330:0:11089:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 898 00000100:00100000:3.0:1789760788.736335:0:11089:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2414:x1876699554840960:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:3.0:1789760788.736358:0:11089:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2414:x1876699554840960:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 23us (603us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760788.736364:0:11089:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 898 00000100:00080000:3.0:1789760788.736428:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789760788.736433:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789760788.736516:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554840448 00000100:00100000:0.0:1789760788.736521:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554840448 00000100:00100000:0.0:1789760788.736566:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731958416754 time 731970352207 state 0x0 ref 1 00000100:00100000:0.0:1789760788.736580:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731970612623 time 731970352207 state 0x1 ref 1 00000100:00100000:0.0:1789760788.736597:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731970612623 time 731982216423 state 0x0 ref 1 00000100:00100000:0.0:1789760788.736607:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb020b0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554840448:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760788.736659:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66359296, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fa60 00010000:00010000:0.0:1789760788.736670:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6535 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.736685:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=731s v=(0 0) 00010000:00010000:0.0:1789760788.736690:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.736696:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6535 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66359296->66363391] (req 66359296->66363391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.736709:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6535 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66359296->66363391] (req 66359296->66363391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.745253:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983510 is committed 00002000:00010000:0.0:1789760788.745369:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66359296 -> 66363392 00002000:00010000:0.0:1789760788.745374:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129632 -> 129640 00010000:00010000:0.0:1789760788.745408:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6535 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66359296->66363391] (req 66359296->66363391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.745422:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6535 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66359296->66363391] (req 66359296->66363391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.745431:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6535 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66359296->66363391] (req 66359296->66363391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.745444:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760788.745449:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6535 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66359296->66363391] (req 66359296->66363391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.745459:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6535 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66359296->66363391] (req 66359296->66363391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.745479:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6535 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66359296->66363391] (req 66359296->66363391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.745488:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6535 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66359296->66363391] (req 66359296->66363391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.745578:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb020b0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554840448:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8966us (9983us total) trans 4294983510 rc 0/0 00000100:00100000:0.0:1789760788.745589:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16212 00000100:00100000:0.0:1789760788.745598:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731970612623 time 731982216423 state 0x2 ref 0 00000100:00100000:2.0:1789760788.761029:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760788.761130:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554841216 00000100:00100000:1.0:1789760788.761141:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554841216 00000100:00100000:1.0:1789760788.761196:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731970612623 time 731982216423 state 0x0 ref 1 00000100:00100000:1.0:1789760788.761212:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731982476839 time 731982216423 state 0x1 ref 1 00000100:00100000:1.0:1789760788.761246:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731982476839 time 732006864563 state 0x0 ref 1 00000100:00100000:1.0:1789760788.761259:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09858380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554841216:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760788.761306:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66363392, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fa68 00010000:00010000:1.0:1789760788.761316:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c653c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.761333:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760788.761337:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.761343:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c653c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66363392->66367487] (req 66363392->66367487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.761358:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c653c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66363392->66367487] (req 66363392->66367487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.764933:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983511 is committed 00002000:00010000:1.0:1789760788.765098:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66363392 -> 66367488 00002000:00010000:1.0:1789760788.765103:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129640 -> 129648 00010000:00010000:1.0:1789760788.765116:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c653c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66363392->66367487] (req 66363392->66367487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.765129:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c653c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66363392->66367487] (req 66363392->66367487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.765144:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c653c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66363392->66367487] (req 66363392->66367487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.765152:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760788.765157:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c653c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66363392->66367487] (req 66363392->66367487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.765166:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c653c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66363392->66367487] (req 66363392->66367487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.765189:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c653c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66363392->66367487] (req 66363392->66367487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.765196:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c653c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66363392->66367487] (req 66363392->66367487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.765291:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09858380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554841216:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4030us (4268us total) trans 4294983511 rc 0/0 00000100:00100000:1.0:1789760788.765303:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16213 00000100:00100000:1.0:1789760788.765312:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731982476839 time 732006864563 state 0x2 ref 0 00000100:00100000:3.0:1789760788.768798:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760788.769468:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554841344 00000100:00100000:2.0:1789760788.769478:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554841344 00000100:00100000:2.0:1789760788.769531:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 731982476839 time 732006864563 state 0x0 ref 1 00000100:00100000:2.0:1789760788.769547:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732007124979 time 732006864563 state 0x1 ref 1 00000100:00100000:2.0:1789760788.769578:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732007124979 time 732015196059 state 0x0 ref 1 00000100:00100000:2.0:1789760788.769590:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554841344:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760788.769646:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66367488, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fa70 00010000:00010000:2.0:1789760788.769655:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6543 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.769671:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760788.769676:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760788.769682:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6543 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66367488->66371583] (req 66367488->66371583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.769696:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6543 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66367488->66371583] (req 66367488->66371583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760788.774205:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983512 is committed 00002000:00010000:2.0:1789760788.774328:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66367488 -> 66371584 00002000:00010000:2.0:1789760788.774334:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129648 -> 129656 00010000:00010000:2.0:1789760788.774347:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6543 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66367488->66371583] (req 66367488->66371583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.774360:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6543 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66367488->66371583] (req 66367488->66371583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.774374:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6543 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66367488->66371583] (req 66367488->66371583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.774721:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760788.774727:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6543 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66367488->66371583] (req 66367488->66371583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.774737:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6543 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66367488->66371583] (req 66367488->66371583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.774756:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6543 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66367488->66371583] (req 66367488->66371583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760788.774764:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6543 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66367488->66371583] (req 66367488->66371583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760788.774976:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554841344:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5383us (6183us total) trans 4294983512 rc 0/0 00000100:00100000:2.0:1789760788.774987:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16214 00000100:00100000:2.0:1789760788.774995:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732007124979 time 732015196059 state 0x2 ref 0 00000100:00100000:2.0:1789760788.779308:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760788.779693:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554841472 00000100:00100000:0.0:1789760788.779703:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554841472 00000100:00100000:0.0:1789760788.779758:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732007124979 time 732015196059 state 0x0 ref 1 00000100:00100000:0.0:1789760788.779772:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732015456475 time 732015196059 state 0x1 ref 1 00000100:00100000:0.0:1789760788.779893:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732015456475 time 732025509811 state 0x0 ref 1 00000100:00100000:0.0:1789760788.779911:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0985a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554841472:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760788.779959:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66371584, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fa78 00010000:00010000:0.0:1789760788.779968:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c654a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.779983:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760788.779987:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.779992:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c654a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66371584->66375679] (req 66371584->66375679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.780006:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c654a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66371584->66375679] (req 66371584->66375679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.788762:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983513 is committed 00002000:00010000:0.0:1789760788.788943:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66371584 -> 66375680 00002000:00010000:0.0:1789760788.788949:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129656 -> 129664 00010000:00010000:0.0:1789760788.788963:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c654a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66371584->66375679] (req 66371584->66375679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.788976:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c654a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66371584->66375679] (req 66371584->66375679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.788986:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c654a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66371584->66375679] (req 66371584->66375679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.788993:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760788.788999:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c654a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66371584->66375679] (req 66371584->66375679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.789019:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c654a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66371584->66375679] (req 66371584->66375679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.789041:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c654a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66371584->66375679] (req 66371584->66375679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.789049:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c654a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66371584->66375679] (req 66371584->66375679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.789147:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0985a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554841472:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9233us (9842us total) trans 4294983513 rc 0/0 00000100:00100000:0.0:1789760788.789157:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16215 00000100:00100000:0.0:1789760788.789168:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732015456475 time 732025509811 state 0x2 ref 0 00000100:00100000:0.0:1789760788.798031:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760788.798176:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554841600 00000100:00100000:1.0:1789760788.798187:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554841600 00000100:00100000:1.0:1789760788.798244:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732015456475 time 732025509811 state 0x0 ref 1 00000100:00100000:1.0:1789760788.798263:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732025770227 time 732025509811 state 0x1 ref 1 00000100:00100000:1.0:1789760788.798643:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732025770227 time 732044249239 state 0x0 ref 1 00000100:00100000:1.0:1789760788.798660:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554841600:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760788.798721:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66375680, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fa80 00010000:00010000:1.0:1789760788.798730:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6551 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.798748:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760788.798752:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.798758:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6551 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66375680->66379775] (req 66375680->66379775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.798772:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6551 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66375680->66379775] (req 66375680->66379775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.809014:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983514 is committed 00002000:00010000:1.0:1789760788.813083:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66375680 -> 66379776 00002000:00010000:1.0:1789760788.813092:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129664 -> 129672 00010000:00010000:1.0:1789760788.813104:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6551 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66375680->66379775] (req 66375680->66379775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.813117:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6551 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66375680->66379775] (req 66375680->66379775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.813125:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6551 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66375680->66379775] (req 66375680->66379775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.813132:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760788.813137:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6551 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66375680->66379775] (req 66375680->66379775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.813146:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6551 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66375680->66379775] (req 66375680->66379775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.813165:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6551 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66375680->66379775] (req 66375680->66379775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.813173:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6551 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66375680->66379775] (req 66375680->66379775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.813330:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554841600:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14670us (15303us total) trans 4294983514 rc 0/0 00000100:00100000:1.0:1789760788.813341:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16216 00000100:00100000:1.0:1789760788.813350:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732025770227 time 732044249239 state 0x2 ref 0 00000100:00100000:3.0:1789760788.820507:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760788.820599:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554841728 00000100:00100000:1.0:1789760788.820613:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554841728 00000100:00100000:1.0:1789760788.820676:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732025770227 time 732044249239 state 0x0 ref 1 00000100:00100000:1.0:1789760788.820689:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732044509655 time 732044249239 state 0x1 ref 1 00000100:00100000:1.0:1789760788.820726:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732044509655 time 732066345032 state 0x0 ref 1 00000100:00100000:1.0:1789760788.820741:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554841728:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760788.820786:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66379776, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fa88 00010000:00010000:1.0:1789760788.820795:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6558 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.820810:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760788.820815:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.820821:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6558 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66379776->66383871] (req 66379776->66383871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.820833:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6558 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66379776->66383871] (req 66379776->66383871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.833034:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983515 is committed 00002000:00010000:1.0:1789760788.833440:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66379776 -> 66383872 00002000:00010000:1.0:1789760788.833445:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129672 -> 129680 00010000:00010000:1.0:1789760788.833458:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6558 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66379776->66383871] (req 66379776->66383871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.833472:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6558 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66379776->66383871] (req 66379776->66383871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.833480:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6558 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66379776->66383871] (req 66379776->66383871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.833490:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760788.833495:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6558 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66379776->66383871] (req 66379776->66383871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.833504:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6558 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66379776->66383871] (req 66379776->66383871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.833524:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6558 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66379776->66383871] (req 66379776->66383871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.833531:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6558 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66379776->66383871] (req 66379776->66383871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.833716:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554841728:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12970us (13213us total) trans 4294983515 rc 0/0 00000100:00100000:0.0:1789760788.833727:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16217 00000100:00100000:0.0:1789760788.833737:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732044509655 time 732066345032 state 0x2 ref 0 00000100:00100000:2.0:1789760788.838860:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760788.841830:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554841856 00000100:00100000:0.0:1789760788.841847:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554841856 00000100:00100000:0.0:1789760788.841912:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732044509655 time 732066345032 state 0x0 ref 1 00000100:00100000:0.0:1789760788.841928:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732066605448 time 732066345032 state 0x1 ref 1 00000100:00100000:0.0:1789760788.842026:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732066605448 time 732087644177 state 0x0 ref 1 00000100:00100000:0.0:1789760788.842042:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09858700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554841856:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760788.842095:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66383872, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fa90 00010000:00010000:0.0:1789760788.842104:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c655f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.842122:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760788.842127:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.842135:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c655f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66383872->66387967] (req 66383872->66387967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.842149:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c655f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66383872->66387967] (req 66383872->66387967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.849288:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983516 is committed 00002000:00010000:0.0:1789760788.849544:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66383872 -> 66387968 00002000:00010000:0.0:1789760788.849549:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129680 -> 129688 00010000:00010000:0.0:1789760788.849563:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c655f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66383872->66387967] (req 66383872->66387967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.849576:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c655f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66383872->66387967] (req 66383872->66387967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.849585:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c655f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66383872->66387967] (req 66383872->66387967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.849598:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760788.849603:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c655f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66383872->66387967] (req 66383872->66387967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.849612:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c655f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66383872->66387967] (req 66383872->66387967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.849641:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c655f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66383872->66387967] (req 66383872->66387967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.849649:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c655f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66383872->66387967] (req 66383872->66387967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.849737:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09858700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554841856:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7694us (10881us total) trans 4294983516 rc 0/0 00000100:00100000:0.0:1789760788.849747:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16218 00000100:00100000:0.0:1789760788.849757:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732066605448 time 732087644177 state 0x2 ref 0 00000100:00100000:2.0:1789760788.860252:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760788.860509:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554841984 00000100:00100000:0.0:1789760788.860519:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554841984 00000100:00100000:0.0:1789760788.860567:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732066605448 time 732087644177 state 0x0 ref 1 00000100:00100000:0.0:1789760788.860582:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732087904593 time 732087644177 state 0x1 ref 1 00000100:00100000:0.0:1789760788.860612:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732087904593 time 732106231036 state 0x0 ref 1 00000100:00100000:0.0:1789760788.860627:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0985bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554841984:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760788.860679:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66387968, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fa98 00010000:00010000:0.0:1789760788.860688:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6566 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.860703:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760788.860708:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.860714:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6566 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66387968->66392063] (req 66387968->66392063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.860727:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6566 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66387968->66392063] (req 66387968->66392063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.867280:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983517 is committed 00002000:00010000:0.0:1789760788.867481:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66387968 -> 66392064 00002000:00010000:0.0:1789760788.867486:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129688 -> 129696 00010000:00010000:0.0:1789760788.867500:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6566 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66387968->66392063] (req 66387968->66392063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.867513:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6566 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66387968->66392063] (req 66387968->66392063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.867522:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6566 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66387968->66392063] (req 66387968->66392063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.867528:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760788.867534:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6566 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66387968->66392063] (req 66387968->66392063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.867542:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6566 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66387968->66392063] (req 66387968->66392063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.867562:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6566 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66387968->66392063] (req 66387968->66392063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.867570:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6566 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66387968->66392063] (req 66387968->66392063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.867661:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0985bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554841984:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7031us (7415us total) trans 4294983517 rc 0/0 00000100:00100000:0.0:1789760788.867671:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16219 00000100:00100000:0.0:1789760788.867680:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732087904593 time 732106231036 state 0x2 ref 0 00000100:00100000:3.0:1789760788.881126:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760788.881218:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554842112 00000100:00100000:1.0:1789760788.881230:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554842112 00000100:00100000:1.0:1789760788.881290:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732087904593 time 732106231036 state 0x0 ref 1 00000100:00100000:1.0:1789760788.881305:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732106491452 time 732106231036 state 0x1 ref 1 00000100:00100000:1.0:1789760788.881337:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732106491452 time 732126955528 state 0x0 ref 1 00000100:00100000:1.0:1789760788.881349:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554842112:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760788.881691:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66392064, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1faa0 00010000:00010000:1.0:1789760788.881701:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c656d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.881718:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760788.881722:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.881729:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c656d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66392064->66396159] (req 66392064->66396159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.881743:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c656d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66392064->66396159] (req 66392064->66396159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.886914:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983518 is committed 00002000:00010000:1.0:1789760788.887026:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66392064 -> 66396160 00002000:00010000:1.0:1789760788.887032:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129696 -> 129704 00010000:00010000:1.0:1789760788.887045:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c656d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66392064->66396159] (req 66392064->66396159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.887060:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c656d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66392064->66396159] (req 66392064->66396159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.887069:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c656d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66392064->66396159] (req 66392064->66396159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.887076:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760788.887081:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c656d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66392064->66396159] (req 66392064->66396159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.887091:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c656d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66392064->66396159] (req 66392064->66396159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.887109:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c656d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66392064->66396159] (req 66392064->66396159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.887120:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c656d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66392064->66396159] (req 66392064->66396159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.887222:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554842112:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5869us (6101us total) trans 4294983518 rc 0/0 00000100:00100000:1.0:1789760788.887232:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16220 00000100:00100000:1.0:1789760788.887243:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732106491452 time 732126955528 state 0x2 ref 0 00000100:00100000:2.0:1789760788.891339:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760788.891652:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554842240 00000100:00100000:0.0:1789760788.891662:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554842240 00000100:00100000:0.0:1789760788.891712:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732106491452 time 732126955528 state 0x0 ref 1 00000100:00100000:0.0:1789760788.891727:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732127215944 time 732126955528 state 0x1 ref 1 00000100:00100000:0.0:1789760788.891839:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732127215944 time 732137456428 state 0x0 ref 1 00000100:00100000:0.0:1789760788.891852:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0985b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554842240:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760788.891901:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66396160, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1faa8 00010000:00010000:0.0:1789760788.891911:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6574 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.891925:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760788.891929:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.891935:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6574 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66396160->66400255] (req 66396160->66400255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.891947:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6574 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66396160->66400255] (req 66396160->66400255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.897544:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983519 is committed 00002000:00010000:0.0:1789760788.897657:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66396160 -> 66400256 00002000:00010000:0.0:1789760788.897663:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129704 -> 129712 00010000:00010000:0.0:1789760788.897677:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6574 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66396160->66400255] (req 66396160->66400255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.897690:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6574 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66396160->66400255] (req 66396160->66400255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.897699:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6574 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66396160->66400255] (req 66396160->66400255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.897706:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760788.897711:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6574 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66396160->66400255] (req 66396160->66400255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.897719:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6574 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66396160->66400255] (req 66396160->66400255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.897738:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6574 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66396160->66400255] (req 66396160->66400255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.897745:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6574 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66396160->66400255] (req 66396160->66400255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.897830:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0985b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554842240:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5975us (6495us total) trans 4294983519 rc 0/0 00000100:00100000:0.0:1789760788.897840:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16221 00000100:00100000:0.0:1789760788.897849:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732127215944 time 732137456428 state 0x2 ref 0 00000100:00100000:2.0:1789760788.900420:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760788.900733:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554842368 00000100:00100000:0.0:1789760788.900742:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554842368 00000100:00100000:0.0:1789760788.900789:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732127215944 time 732137456428 state 0x0 ref 1 00000100:00100000:0.0:1789760788.900803:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732137716844 time 732137456428 state 0x1 ref 1 00000100:00100000:0.0:1789760788.900866:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732137716844 time 732146484343 state 0x0 ref 1 00000100:00100000:0.0:1789760788.900878:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09859880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554842368:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760788.900922:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66400256, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fab0 00010000:00010000:0.0:1789760788.900930:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c657b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.900945:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760788.900950:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.900956:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c657b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66400256->66404351] (req 66400256->66404351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.900969:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c657b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66400256->66404351] (req 66400256->66404351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.905932:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983520 is committed 00002000:00010000:0.0:1789760788.906029:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66400256 -> 66404352 00002000:00010000:0.0:1789760788.906035:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129712 -> 129720 00010000:00010000:0.0:1789760788.906047:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c657b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66400256->66404351] (req 66400256->66404351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.906059:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c657b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66400256->66404351] (req 66400256->66404351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.906068:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c657b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66400256->66404351] (req 66400256->66404351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.906074:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760788.906079:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c657b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66400256->66404351] (req 66400256->66404351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.906089:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c657b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66400256->66404351] (req 66400256->66404351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.906108:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c657b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66400256->66404351] (req 66400256->66404351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.906120:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c657b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66400256->66404351] (req 66400256->66404351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.906208:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09859880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554842368:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5326us (5829us total) trans 4294983520 rc 0/0 00000100:00100000:0.0:1789760788.906218:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16222 00000100:00100000:0.0:1789760788.906227:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732137716844 time 732146484343 state 0x2 ref 0 00000100:00100000:3.0:1789760788.915797:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760788.915907:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554842496 00000100:00100000:1.0:1789760788.915919:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554842496 00000100:00100000:1.0:1789760788.915995:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732137716844 time 732146484343 state 0x0 ref 1 00000100:00100000:1.0:1789760788.916011:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732146744759 time 732146484343 state 0x1 ref 1 00000100:00100000:1.0:1789760788.916044:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732146744759 time 732161661433 state 0x0 ref 1 00000100:00100000:1.0:1789760788.916057:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554842496:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760788.916104:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66404352, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fab8 00010000:00010000:1.0:1789760788.916114:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6582 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.916130:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760788.916135:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.916141:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6582 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66404352->66408447] (req 66404352->66408447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.916159:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6582 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66404352->66408447] (req 66404352->66408447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760788.922987:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983521 is committed 00002000:00010000:1.0:1789760788.923686:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66404352 -> 66408448 00002000:00010000:1.0:1789760788.924290:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129720 -> 129728 00010000:00010000:1.0:1789760788.924315:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6582 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66404352->66408447] (req 66404352->66408447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.924332:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6582 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66404352->66408447] (req 66404352->66408447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.924342:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6582 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66404352->66408447] (req 66404352->66408447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.924350:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760788.924355:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6582 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66404352->66408447] (req 66404352->66408447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.924365:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6582 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66404352->66408447] (req 66404352->66408447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.924526:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6582 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66404352->66408447] (req 66404352->66408447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.924536:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6582 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66404352->66408447] (req 66404352->66408447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.924681:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554842496:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8617us (8884us total) trans 4294983521 rc 0/0 00000100:00100000:1.0:1789760788.925054:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16223 00000100:00100000:1.0:1789760788.925071:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732146744759 time 732161661433 state 0x2 ref 0 00000100:00100000:1.0:1789760788.930793:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760788.930955:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554842624 00000100:00100000:0.0:1789760788.930970:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554842624 00000100:00100000:0.0:1789760788.931041:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732146744759 time 732161661433 state 0x0 ref 1 00000100:00100000:0.0:1789760788.931060:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732161921849 time 732161661433 state 0x1 ref 1 00000100:00100000:0.0:1789760788.931098:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732161921849 time 732176715633 state 0x0 ref 1 00000100:00100000:0.0:1789760788.931120:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a386a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554842624:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760788.931182:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66408448, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fac0 00010000:00010000:0.0:1789760788.931193:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6589 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.931212:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760788.931218:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.931226:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6589 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66408448->66412543] (req 66408448->66412543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.931248:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6589 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66408448->66412543] (req 66408448->66412543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.948869:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983522 is committed 00002000:00010000:1.0:1789760788.949071:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66408448 -> 66412544 00002000:00010000:1.0:1789760788.949077:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129728 -> 129736 00010000:00010000:1.0:1789760788.949096:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6589 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66408448->66412543] (req 66408448->66412543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.949168:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6589 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66408448->66412543] (req 66408448->66412543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.949178:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6589 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66408448->66412543] (req 66408448->66412543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.949187:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760788.949193:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6589 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66408448->66412543] (req 66408448->66412543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.949203:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6589 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66408448->66412543] (req 66408448->66412543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.949228:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6589 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66408448->66412543] (req 66408448->66412543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.949238:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6589 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66408448->66412543] (req 66408448->66412543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.949475:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a386a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554842624:12345-192.168.203.22@tcp:4:dd.0 Request processed in 18357us (18686us total) trans 4294983522 rc 0/0 00000100:00100000:1.0:1789760788.949488:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16224 00000100:00100000:1.0:1789760788.949500:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732161921849 time 732176715633 state 0x2 ref 0 00000100:00100000:2.0:1789760788.960808:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760788.961261:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554842752 00000100:00100000:0.0:1789760788.961275:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554842752 00000100:00100000:0.0:1789760788.961353:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732161921849 time 732176715633 state 0x0 ref 1 00000100:00100000:0.0:1789760788.961373:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732176976049 time 732176715633 state 0x1 ref 1 00000100:00100000:0.0:1789760788.961449:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732176976049 time 732207066339 state 0x0 ref 1 00000100:00100000:0.0:1789760788.961465:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0985aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554842752:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760788.961534:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66412544, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fac8 00010000:00010000:0.0:1789760788.961545:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6590 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.961565:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760788.961570:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.961588:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6590 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66412544->66416639] (req 66412544->66416639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.961607:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6590 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66412544->66416639] (req 66412544->66416639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760788.972931:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983523 is committed 00002000:00010000:0.0:1789760788.973143:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66412544 -> 66416640 00002000:00010000:0.0:1789760788.973150:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129736 -> 129744 00010000:00010000:0.0:1789760788.973167:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6590 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66412544->66416639] (req 66412544->66416639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.973183:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6590 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66412544->66416639] (req 66412544->66416639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.973193:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6590 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66412544->66416639] (req 66412544->66416639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.973201:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760788.973207:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6590 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66412544->66416639] (req 66412544->66416639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.973216:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6590 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66412544->66416639] (req 66412544->66416639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.973244:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6590 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66412544->66416639] (req 66412544->66416639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.973252:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6590 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66412544->66416639] (req 66412544->66416639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760788.973476:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0985aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554842752:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12006us (12674us total) trans 4294983523 rc 0/0 00000100:00100000:0.0:1789760788.973491:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16225 00000100:00100000:0.0:1789760788.973504:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732176976049 time 732207066339 state 0x2 ref 0 00000100:00100000:3.0:1789760788.977102:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760788.977335:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554842880 00000100:00100000:1.0:1789760788.977348:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554842880 00000100:00100000:1.0:1789760788.977806:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732176976049 time 732207066339 state 0x0 ref 1 00000100:00100000:1.0:1789760788.977828:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732207326755 time 732207066339 state 0x1 ref 1 00000100:00100000:1.0:1789760788.977864:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732207326755 time 732223482126 state 0x0 ref 1 00000100:00100000:1.0:1789760788.977880:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554842880:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760788.977943:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66416640, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fad0 00010000:00010000:1.0:1789760788.977956:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6597 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.977975:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760788.977980:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760788.977987:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6597 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66416640->66420735] (req 66416640->66420735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.978102:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6597 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66416640->66420735] (req 66416640->66420735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760788.989301:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983524 is committed 00002000:00010000:1.0:1789760788.989473:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66416640 -> 66420736 00002000:00010000:1.0:1789760788.989479:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129744 -> 129752 00010000:00010000:1.0:1789760788.989499:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6597 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66416640->66420735] (req 66416640->66420735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.989523:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6597 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66416640->66420735] (req 66416640->66420735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.989533:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6597 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66416640->66420735] (req 66416640->66420735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.989542:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760788.989547:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6597 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66416640->66420735] (req 66416640->66420735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.989558:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6597 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66416640->66420735] (req 66416640->66420735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.989584:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6597 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66416640->66420735] (req 66416640->66420735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760788.989591:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6597 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66416640->66420735] (req 66416640->66420735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760788.989741:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554842880:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11858us (12648us total) trans 4294983524 rc 0/0 00000100:00100000:1.0:1789760788.989755:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16226 00000100:00100000:1.0:1789760788.989767:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732207326755 time 732223482126 state 0x2 ref 0 00000100:00100000:1.0:1789760788.997353:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760788.997642:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554843008 00000100:00100000:0.0:1789760788.997656:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554843008 00000100:00100000:0.0:1789760788.997793:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732207326755 time 732223482126 state 0x0 ref 1 00000100:00100000:0.0:1789760788.997813:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732223742542 time 732223482126 state 0x1 ref 1 00000100:00100000:0.0:1789760788.997904:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732223742542 time 732243521044 state 0x0 ref 1 00000100:00100000:0.0:1789760788.997920:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a386300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554843008:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760788.997987:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66420736, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fad8 00010000:00010000:0.0:1789760788.997998:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c659e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.998173:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760788.998178:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760788.998186:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c659e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66420736->66424831] (req 66420736->66424831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760788.998204:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c659e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66420736->66424831] (req 66420736->66424831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.012268:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983525 is committed 00002000:00010000:0.0:1789760789.012684:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66420736 -> 66424832 00002000:00010000:0.0:1789760789.013131:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129752 -> 129760 00010000:00010000:0.0:1789760789.013152:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c659e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66420736->66424831] (req 66420736->66424831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.013167:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c659e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66420736->66424831] (req 66420736->66424831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.013177:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c659e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66420736->66424831] (req 66420736->66424831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.013186:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760789.013192:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c659e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66420736->66424831] (req 66420736->66424831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.013202:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c659e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66420736->66424831] (req 66420736->66424831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.013226:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c659e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66420736->66424831] (req 66420736->66424831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.013234:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c659e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66420736->66424831] (req 66420736->66424831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.013368:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a386300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554843008:12345-192.168.203.22@tcp:4:dd.0 Request processed in 15444us (16023us total) trans 4294983525 rc 0/0 00000100:00100000:0.0:1789760789.016917:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16227 00000100:00100000:0.0:1789760789.016937:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732223742542 time 732243521044 state 0x2 ref 0 00000100:00100000:0.0:1789760789.019566:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760789.019742:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554843136 00000100:00100000:1.0:1789760789.019756:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554843136 00000100:00100000:1.0:1789760789.019850:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732223742542 time 732243521044 state 0x0 ref 1 00000100:00100000:1.0:1789760789.019871:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732243781460 time 732243521044 state 0x1 ref 1 00000100:00100000:1.0:1789760789.019908:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732243781460 time 732265525936 state 0x0 ref 1 00000100:00100000:1.0:1789760789.019923:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554843136:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760789.020003:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66424832, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fae0 00010000:00010000:1.0:1789760789.020024:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c65a5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.020045:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760789.020050:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.020058:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c65a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66424832->66428927] (req 66424832->66428927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.020076:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c65a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66424832->66428927] (req 66424832->66428927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789760789.020757:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760789.028574:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983526 is committed 00002000:00010000:1.0:1789760789.028845:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66424832 -> 66428928 00002000:00010000:1.0:1789760789.028855:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129760 -> 129768 00010000:00010000:1.0:1789760789.028873:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c65a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66424832->66428927] (req 66424832->66428927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.028889:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c65a5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66424832->66428927] (req 66424832->66428927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.028901:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c65a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66424832->66428927] (req 66424832->66428927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.028909:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760789.028914:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c65a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66424832->66428927] (req 66424832->66428927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.028924:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c65a5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66424832->66428927] (req 66424832->66428927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.028950:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c65a5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66424832->66428927] (req 66424832->66428927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.028959:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c65a5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66424832->66428927] (req 66424832->66428927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.030013:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554843136:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10074us (10443us total) trans 4294983526 rc 0/0 00000100:00100000:1.0:1789760789.030029:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16228 00000100:00100000:1.0:1789760789.030041:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732243781460 time 732265525936 state 0x2 ref 0 00000100:00100000:3.0:1789760789.040877:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760789.041144:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554843264 00000100:00100000:2.0:1789760789.041158:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554843264 00000100:00100000:2.0:1789760789.041236:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732243781460 time 732265525936 state 0x0 ref 1 00000100:00100000:2.0:1789760789.041257:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732265786352 time 732265525936 state 0x1 ref 1 00000100:00100000:2.0:1789760789.041294:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732265786352 time 732286911770 state 0x0 ref 1 00000100:00100000:2.0:1789760789.041308:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554843264:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760789.041370:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66428928, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fae8 00010000:00010000:2.0:1789760789.041582:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c65ac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.041605:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760789.041610:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.041619:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c65ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66428928->66433023] (req 66428928->66433023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.041644:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c65ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66428928->66433023] (req 66428928->66433023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.058179:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983527 is committed 00002000:00010000:2.0:1789760789.058334:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66428928 -> 66433024 00002000:00010000:2.0:1789760789.058340:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129768 -> 129776 00010000:00010000:2.0:1789760789.058356:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c65ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66428928->66433023] (req 66428928->66433023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.058370:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c65ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66428928->66433023] (req 66428928->66433023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.058428:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c65ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66428928->66433023] (req 66428928->66433023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.058436:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760789.058441:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c65ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66428928->66433023] (req 66428928->66433023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.058451:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c65ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66428928->66433023] (req 66428928->66433023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.058475:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c65ac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66428928->66433023] (req 66428928->66433023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.058483:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c65ac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66428928->66433023] (req 66428928->66433023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.058980:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554843264:12345-192.168.203.22@tcp:4:dd.0 Request processed in 17665us (18292us total) trans 4294983527 rc 0/0 00000100:00100000:3.0:1789760789.058996:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16229 00000100:00100000:3.0:1789760789.059016:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732265786352 time 732286911770 state 0x2 ref 0 00000100:00100000:3.0:1789760789.065142:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760789.066330:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554843392 00000100:00100000:0.0:1789760789.066345:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554843392 00000100:00100000:0.0:1789760789.066448:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732265786352 time 732286911770 state 0x0 ref 1 00000100:00100000:0.0:1789760789.066469:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732287172186 time 732286911770 state 0x1 ref 1 00000100:00100000:0.0:1789760789.066500:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732287172186 time 732312119263 state 0x0 ref 1 00000100:00100000:0.0:1789760789.066520:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554843392:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760789.066587:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66433024, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1faf0 00010000:00010000:0.0:1789760789.066600:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c65b3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.066620:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760789.066625:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.067011:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c65b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66433024->66437119] (req 66433024->66437119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.067030:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c65b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66433024->66437119] (req 66433024->66437119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.073283:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983528 is committed 00002000:00010000:0.0:1789760789.073470:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66433024 -> 66437120 00002000:00010000:0.0:1789760789.073477:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129776 -> 129784 00010000:00010000:0.0:1789760789.073496:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c65b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66433024->66437119] (req 66433024->66437119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.073531:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c65b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66433024->66437119] (req 66433024->66437119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.073541:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c65b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66433024->66437119] (req 66433024->66437119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.073550:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760789.073556:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c65b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66433024->66437119] (req 66433024->66437119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.073566:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c65b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66433024->66437119] (req 66433024->66437119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.073590:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c65b3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66433024->66437119] (req 66433024->66437119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.073598:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c65b3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66433024->66437119] (req 66433024->66437119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.073849:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554843392:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7328us (8714us total) trans 4294983528 rc 0/0 00000100:00100000:0.0:1789760789.073863:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16230 00000100:00100000:0.0:1789760789.073876:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732287172186 time 732312119263 state 0x2 ref 0 00010000:00010000:3.0:1789760789.074022:0:6508:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789760789.078099:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760789.078281:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554843520 00000100:00100000:1.0:1789760789.078295:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554843520 00000100:00100000:1.0:1789760789.078372:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732287172186 time 732312119263 state 0x0 ref 1 00000100:00100000:1.0:1789760789.078424:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732312379679 time 732312119263 state 0x1 ref 1 00000100:00100000:1.0:1789760789.078462:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732312379679 time 732324079731 state 0x0 ref 1 00000100:00100000:1.0:1789760789.078476:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554843520:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760789.078663:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66437120, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1faf8 00010000:00010000:1.0:1789760789.078675:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c65ba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.078807:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760789.078813:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.078822:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c65ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66437120->66441215] (req 66437120->66441215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.078841:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c65ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66437120->66441215] (req 66437120->66441215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.091275:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983529 is committed 00002000:00010000:0.0:1789760789.097822:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66437120 -> 66441216 00002000:00010000:0.0:1789760789.097836:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129784 -> 129792 00010000:00010000:0.0:1789760789.097854:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c65ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66437120->66441215] (req 66437120->66441215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.097871:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c65ba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66437120->66441215] (req 66437120->66441215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.097882:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c65ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66437120->66441215] (req 66437120->66441215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.097892:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760789.097899:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c65ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66437120->66441215] (req 66437120->66441215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.097910:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c65ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66437120->66441215] (req 66437120->66441215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.097935:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c65ba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66437120->66441215] (req 66437120->66441215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.097945:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c65ba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66437120->66441215] (req 66437120->66441215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.099142:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554843520:12345-192.168.203.22@tcp:4:dd.0 Request processed in 20655us (21046us total) trans 4294983529 rc 0/0 00000100:00100000:0.0:1789760789.099160:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16231 00000100:00100000:0.0:1789760789.099175:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732312379679 time 732324079731 state 0x2 ref 0 00000100:00100000:1.0:1789760789.104047:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760789.104150:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554843648 00000100:00100000:0.0:1789760789.104164:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554843648 00000100:00100000:0.0:1789760789.104245:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732312379679 time 732324079731 state 0x0 ref 1 00000100:00100000:0.0:1789760789.104266:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732324340147 time 732324079731 state 0x1 ref 1 00000100:00100000:0.0:1789760789.104303:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732324340147 time 732349921628 state 0x0 ref 1 00000100:00100000:0.0:1789760789.104317:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554843648:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760789.104426:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66441216, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fb00 00010000:00010000:0.0:1789760789.104438:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c65c1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.104461:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760789.104467:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.104475:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c65c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66441216->66445311] (req 66441216->66445311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.104493:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c65c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66441216->66445311] (req 66441216->66445311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.120845:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983530 is committed 00002000:00010000:2.0:1789760789.134336:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66441216 -> 66445312 00002000:00010000:2.0:1789760789.134351:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129792 -> 129800 00010000:00010000:2.0:1789760789.134371:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c65c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66441216->66445311] (req 66441216->66445311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.134398:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c65c1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66441216->66445311] (req 66441216->66445311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.134408:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c65c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66441216->66445311] (req 66441216->66445311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.134415:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760789.134421:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c65c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66441216->66445311] (req 66441216->66445311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.134432:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c65c1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66441216->66445311] (req 66441216->66445311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.134456:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c65c1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66441216->66445311] (req 66441216->66445311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.134465:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c65c1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66441216->66445311] (req 66441216->66445311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.135262:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554843648:12345-192.168.203.22@tcp:4:dd.0 Request processed in 30937us (31221us total) trans 4294983530 rc 0/0 00000100:00100000:2.0:1789760789.135277:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16232 00000100:00100000:2.0:1789760789.135291:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732324340147 time 732349921628 state 0x2 ref 0 00000100:00100000:3.0:1789760789.151117:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760789.151246:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554843776 00000100:00100000:0.0:1789760789.151261:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554843776 00000100:00100000:0.0:1789760789.151336:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732324340147 time 732349921628 state 0x0 ref 1 00000100:00100000:0.0:1789760789.151355:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732350182044 time 732349921628 state 0x1 ref 1 00000100:00100000:0.0:1789760789.151447:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732350182044 time 732397064628 state 0x0 ref 1 00000100:00100000:0.0:1789760789.151463:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554843776:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760789.151539:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66445312, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fb08 00010000:00010000:0.0:1789760789.151550:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c65c8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.151570:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760789.151575:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.151582:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c65c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66445312->66449407] (req 66445312->66449407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.151599:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c65c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66445312->66449407] (req 66445312->66449407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.164115:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983531 is committed 00002000:00010000:1.0:1789760789.168699:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66445312 -> 66449408 00002000:00010000:1.0:1789760789.168715:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129800 -> 129808 00010000:00010000:1.0:1789760789.168733:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c65c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66445312->66449407] (req 66445312->66449407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.168749:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c65c8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66445312->66449407] (req 66445312->66449407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.168758:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c65c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66445312->66449407] (req 66445312->66449407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.168766:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760789.168772:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c65c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66445312->66449407] (req 66445312->66449407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.168783:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c65c8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66445312->66449407] (req 66445312->66449407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.168809:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c65c8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66445312->66449407] (req 66445312->66449407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.168818:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c65c8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66445312->66449407] (req 66445312->66449407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.168965:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554843776:12345-192.168.203.22@tcp:4:dd.0 Request processed in 17496us (17854us total) trans 4294983531 rc 0/0 00000100:00100000:1.0:1789760789.168983:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16233 00000100:00100000:1.0:1789760789.168996:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732350182044 time 732397064628 state 0x2 ref 0 00000100:00100000:2.0:1789760789.182529:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760789.182900:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554843904 00000100:00100000:1.0:1789760789.182916:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554843904 00000100:00100000:1.0:1789760789.182997:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732350182044 time 732397064628 state 0x0 ref 1 00000100:00100000:1.0:1789760789.183028:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732397325044 time 732397064628 state 0x1 ref 1 00000100:00100000:1.0:1789760789.183114:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732397325044 time 732428730513 state 0x0 ref 1 00000100:00100000:1.0:1789760789.183130:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09858a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554843904:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760789.183197:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66449408, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fb10 00010000:00010000:1.0:1789760789.183209:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c65cf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.183228:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760789.183234:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.183242:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c65cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66449408->66453503] (req 66449408->66453503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.183260:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c65cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66449408->66453503] (req 66449408->66453503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.192160:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983532 is committed 00002000:00010000:0.0:1789760789.192305:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66449408 -> 66453504 00002000:00010000:0.0:1789760789.192311:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129808 -> 129816 00010000:00010000:0.0:1789760789.192328:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c65cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66449408->66453503] (req 66449408->66453503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.192342:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c65cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66449408->66453503] (req 66449408->66453503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.192351:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c65cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66449408->66453503] (req 66449408->66453503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.192364:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760789.192371:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c65cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66449408->66453503] (req 66449408->66453503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.192412:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c65cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66449408->66453503] (req 66449408->66453503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.192437:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c65cf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66449408->66453503] (req 66449408->66453503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.192444:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c65cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66449408->66453503] (req 66449408->66453503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.192567:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09858a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554843904:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9434us (10046us total) trans 4294983532 rc 0/0 00000100:00100000:0.0:1789760789.192589:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16234 00000100:00100000:0.0:1789760789.192602:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732397325044 time 732428730513 state 0x2 ref 0 00000100:00100000:3.0:1789760789.200562:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760789.201112:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554844032 00000100:00100000:1.0:1789760789.201127:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554844032 00000100:00100000:1.0:1789760789.201203:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732397325044 time 732428730513 state 0x0 ref 1 00000100:00100000:1.0:1789760789.201222:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732428990929 time 732428730513 state 0x1 ref 1 00000100:00100000:1.0:1789760789.201265:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732428990929 time 732446881976 state 0x0 ref 1 00000100:00100000:1.0:1789760789.201279:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554844032:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760789.201344:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66453504, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fb18 00010000:00010000:1.0:1789760789.201355:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c65d6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.201416:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760789.201422:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.201431:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c65d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66453504->66457599] (req 66453504->66457599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.201448:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c65d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66453504->66457599] (req 66453504->66457599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.211943:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983533 is committed 00002000:00010000:0.0:1789760789.215611:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66453504 -> 66457600 00002000:00010000:0.0:1789760789.215625:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129816 -> 129824 00010000:00010000:0.0:1789760789.215653:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c65d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66453504->66457599] (req 66453504->66457599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.215668:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c65d6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66453504->66457599] (req 66453504->66457599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.215685:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c65d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66453504->66457599] (req 66453504->66457599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.215693:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760789.215699:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c65d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66453504->66457599] (req 66453504->66457599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.215710:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c65d6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66453504->66457599] (req 66453504->66457599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.215810:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c65d6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66453504->66457599] (req 66453504->66457599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.215819:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c65d6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66453504->66457599] (req 66453504->66457599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.216096:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554844032:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14811us (15543us total) trans 4294983533 rc 0/0 00000100:00100000:0.0:1789760789.216110:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16235 00000100:00100000:0.0:1789760789.216122:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732428990929 time 732446881976 state 0x2 ref 0 00000100:00100000:3.0:1789760789.220797:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760789.221813:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554844160 00000100:00100000:2.0:1789760789.221831:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554844160 00000100:00100000:2.0:1789760789.221917:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732428990929 time 732446881976 state 0x0 ref 1 00000100:00100000:2.0:1789760789.221938:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732447142392 time 732446881976 state 0x1 ref 1 00000100:00100000:2.0:1789760789.222076:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732447142392 time 732467692315 state 0x0 ref 1 00000100:00100000:2.0:1789760789.222092:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554844160:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760789.222152:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66457600, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fb20 00010000:00010000:2.0:1789760789.222163:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c65dd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.222182:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760789.222187:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.222195:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c65dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66457600->66461695] (req 66457600->66461695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.222212:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c65dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66457600->66461695] (req 66457600->66461695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.229756:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983534 is committed 00002000:00010000:2.0:1789760789.229914:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66457600 -> 66461696 00002000:00010000:2.0:1789760789.229921:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129824 -> 129832 00010000:00010000:2.0:1789760789.229938:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c65dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66457600->66461695] (req 66457600->66461695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.229959:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c65dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66457600->66461695] (req 66457600->66461695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.229969:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c65dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66457600->66461695] (req 66457600->66461695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.229977:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760789.229983:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c65dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66457600->66461695] (req 66457600->66461695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.229994:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c65dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66457600->66461695] (req 66457600->66461695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.230021:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c65dd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66457600->66461695] (req 66457600->66461695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.230030:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c65dd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66457600->66461695] (req 66457600->66461695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.230157:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554844160:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8061us (9368us total) trans 4294983534 rc 0/0 00000100:00100000:2.0:1789760789.230169:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16236 00000100:00100000:2.0:1789760789.230181:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732447142392 time 732467692315 state 0x2 ref 0 00000100:00100000:2.0:1789760789.235489:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760789.235588:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554844288 00000100:00100000:3.0:1789760789.235602:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554844288 00000100:00100000:3.0:1789760789.235681:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732447142392 time 732467692315 state 0x0 ref 1 00000100:00100000:3.0:1789760789.235705:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732467952731 time 732467692315 state 0x1 ref 1 00000100:00100000:3.0:1789760789.235732:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732467952731 time 732481349040 state 0x0 ref 1 00000100:00100000:3.0:1789760789.235746:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09859c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554844288:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760789.235806:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66461696, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fb28 00010000:00010000:3.0:1789760789.235818:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c65e4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.235837:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760789.235842:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.235850:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c65e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66461696->66465791] (req 66461696->66465791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.235865:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c65e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66461696->66465791] (req 66461696->66465791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.242713:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983535 is committed 00002000:00010000:1.0:1789760789.252558:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66461696 -> 66465792 00002000:00010000:1.0:1789760789.252570:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129832 -> 129840 00010000:00010000:1.0:1789760789.252588:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c65e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66461696->66465791] (req 66461696->66465791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.252602:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c65e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66461696->66465791] (req 66461696->66465791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.252613:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c65e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66461696->66465791] (req 66461696->66465791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.252757:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760789.252764:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c65e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66461696->66465791] (req 66461696->66465791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.252775:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c65e4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66461696->66465791] (req 66461696->66465791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.252799:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c65e4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66461696->66465791] (req 66461696->66465791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.252807:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c65e4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66461696->66465791] (req 66461696->66465791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.253094:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09859c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554844288:12345-192.168.203.22@tcp:4:dd.0 Request processed in 17341us (17610us total) trans 4294983535 rc 0/0 00000100:00100000:1.0:1789760789.253107:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16237 00000100:00100000:1.0:1789760789.253119:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732467952731 time 732481349040 state 0x2 ref 0 00000100:00100000:3.0:1789760789.262641:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760789.262740:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554844416 00000100:00100000:0.0:1789760789.262754:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554844416 00000100:00100000:0.0:1789760789.262840:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732467952731 time 732481349040 state 0x0 ref 1 00000100:00100000:0.0:1789760789.262861:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732481609456 time 732481349040 state 0x1 ref 1 00000100:00100000:0.0:1789760789.262900:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732481609456 time 732508518074 state 0x0 ref 1 00000100:00100000:0.0:1789760789.262915:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554844416:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760789.262983:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66465792, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fb30 00010000:00010000:0.0:1789760789.262994:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c65eb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.263015:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760789.263019:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.263027:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c65eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66465792->66469887] (req 66465792->66469887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.263043:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c65eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66465792->66469887] (req 66465792->66469887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.274721:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983536 is committed 00002000:00010000:2.0:1789760789.284222:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66465792 -> 66469888 00002000:00010000:2.0:1789760789.284234:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129840 -> 129848 00010000:00010000:2.0:1789760789.284252:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c65eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66465792->66469887] (req 66465792->66469887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.284267:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c65eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66465792->66469887] (req 66465792->66469887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.284278:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c65eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66465792->66469887] (req 66465792->66469887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.284286:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760789.284293:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c65eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66465792->66469887] (req 66465792->66469887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.284303:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c65eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66465792->66469887] (req 66465792->66469887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.284330:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c65eb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66465792->66469887] (req 66465792->66469887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.284340:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c65eb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66465792->66469887] (req 66465792->66469887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.285614:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554844416:12345-192.168.203.22@tcp:4:dd.0 Request processed in 22687us (22983us total) trans 4294983536 rc 0/0 00000100:00100000:2.0:1789760789.285649:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16238 00000100:00100000:2.0:1789760789.285663:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732481609456 time 732508518074 state 0x2 ref 0 00000100:00100000:3.0:1789760789.298256:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760789.302701:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554844544 00000100:00100000:0.0:1789760789.302723:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554844544 00000100:00100000:0.0:1789760789.303119:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732481609456 time 732508518074 state 0x0 ref 1 00000100:00100000:0.0:1789760789.303144:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732508778490 time 732508518074 state 0x1 ref 1 00000100:00100000:0.0:1789760789.303260:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732508778490 time 732548876189 state 0x0 ref 1 00000100:00100000:0.0:1789760789.303278:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554844544:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760789.303370:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66469888, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fb38 00010000:00010000:0.0:1789760789.303404:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c65f2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.303425:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760789.303430:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.303438:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c65f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66469888->66473983] (req 66469888->66473983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.303456:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c65f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66469888->66473983] (req 66469888->66473983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.313511:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983537 is committed 00002000:00010000:2.0:1789760789.318960:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66469888 -> 66473984 00002000:00010000:2.0:1789760789.318973:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129848 -> 129856 00010000:00010000:2.0:1789760789.318991:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c65f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66469888->66473983] (req 66469888->66473983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.319156:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c65f2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66469888->66473983] (req 66469888->66473983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.319168:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c65f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66469888->66473983] (req 66469888->66473983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.319177:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760789.319183:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c65f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66469888->66473983] (req 66469888->66473983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.319194:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c65f2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66469888->66473983] (req 66469888->66473983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.319218:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c65f2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66469888->66473983] (req 66469888->66473983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.319227:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c65f2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66469888->66473983] (req 66469888->66473983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.319485:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554844544:12345-192.168.203.22@tcp:4:dd.0 Request processed in 16206us (21237us total) trans 4294983537 rc 0/0 00000100:00100000:2.0:1789760789.319499:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16239 00000100:00100000:2.0:1789760789.319512:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732508778490 time 732548876189 state 0x2 ref 0 00000100:00100000:2.0:1789760789.334203:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760789.334312:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554844672 00000100:00100000:1.0:1789760789.334326:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554844672 00000100:00100000:1.0:1789760789.334442:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732508778490 time 732548876189 state 0x0 ref 1 00000100:00100000:1.0:1789760789.334462:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732549136605 time 732548876189 state 0x1 ref 1 00000100:00100000:1.0:1789760789.334496:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732549136605 time 732580113867 state 0x0 ref 1 00000100:00100000:1.0:1789760789.334509:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09859180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554844672:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760789.334569:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66473984, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fb40 00010000:00010000:1.0:1789760789.334590:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c65f9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.334610:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760789.334614:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.334623:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c65f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66473984->66478079] (req 66473984->66478079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.334648:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c65f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66473984->66478079] (req 66473984->66478079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.352312:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983538 is committed 00002000:00010000:1.0:1789760789.352723:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66473984 -> 66478080 00002000:00010000:1.0:1789760789.352731:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129856 -> 129864 00010000:00010000:1.0:1789760789.352748:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c65f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66473984->66478079] (req 66473984->66478079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.352762:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c65f9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66473984->66478079] (req 66473984->66478079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.352772:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c65f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66473984->66478079] (req 66473984->66478079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.352785:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760789.352791:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c65f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66473984->66478079] (req 66473984->66478079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.352803:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c65f9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66473984->66478079] (req 66473984->66478079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.352838:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c65f9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66473984->66478079] (req 66473984->66478079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.352848:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c65f9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66473984->66478079] (req 66473984->66478079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.352980:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09859180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554844672:12345-192.168.203.22@tcp:4:dd.0 Request processed in 18466us (18786us total) trans 4294983538 rc 0/0 00000100:00100000:1.0:1789760789.352993:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16240 00000100:00100000:1.0:1789760789.353013:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732549136605 time 732580113867 state 0x2 ref 0 00000100:00100000:3.0:1789760789.358713:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760789.358848:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554844800 00000100:00100000:1.0:1789760789.358863:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554844800 00000100:00100000:1.0:1789760789.358943:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732549136605 time 732580113867 state 0x0 ref 1 00000100:00100000:1.0:1789760789.358963:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732580374283 time 732580113867 state 0x1 ref 1 00000100:00100000:1.0:1789760789.359015:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732580374283 time 732604624037 state 0x0 ref 1 00000100:00100000:1.0:1789760789.359033:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554844800:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760789.359100:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66478080, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fb48 00010000:00010000:1.0:1789760789.359112:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6600 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.359132:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760789.359138:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.359146:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6600 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66478080->66482175] (req 66478080->66482175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.359163:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6600 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66478080->66482175] (req 66478080->66482175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.372846:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983539 is committed 00002000:00010000:1.0:1789760789.372989:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66478080 -> 66482176 00002000:00010000:1.0:1789760789.372995:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129864 -> 129872 00010000:00010000:1.0:1789760789.373021:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6600 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66478080->66482175] (req 66478080->66482175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.373036:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6600 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66478080->66482175] (req 66478080->66482175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.373047:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6600 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66478080->66482175] (req 66478080->66482175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.373056:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760789.373063:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6600 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66478080->66482175] (req 66478080->66482175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.373074:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6600 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66478080->66482175] (req 66478080->66482175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.373101:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6600 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66478080->66482175] (req 66478080->66482175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.373110:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6600 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66478080->66482175] (req 66478080->66482175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.373248:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554844800:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14211us (14770us total) trans 4294983539 rc 0/0 00000100:00100000:1.0:1789760789.373261:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16241 00000100:00100000:1.0:1789760789.373274:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732580374283 time 732604624037 state 0x2 ref 0 00000100:00100000:2.0:1789760789.378551:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760789.378678:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554844928 00000100:00100000:0.0:1789760789.378692:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554844928 00000100:00100000:0.0:1789760789.378776:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732580374283 time 732604624037 state 0x0 ref 1 00000100:00100000:0.0:1789760789.378796:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732604884453 time 732604624037 state 0x1 ref 1 00000100:00100000:0.0:1789760789.378842:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732604884453 time 732624460231 state 0x0 ref 1 00000100:00100000:0.0:1789760789.378856:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0985b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554844928:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760789.378924:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66482176, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fb50 00010000:00010000:0.0:1789760789.378935:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6607 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.378955:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760789.378961:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.378968:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6607 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66482176->66486271] (req 66482176->66486271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.378985:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6607 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66482176->66486271] (req 66482176->66486271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.391126:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983540 is committed 00002000:00010000:0.0:1789760789.391345:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66482176 -> 66486272 00002000:00010000:0.0:1789760789.391352:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129872 -> 129880 00010000:00010000:0.0:1789760789.391369:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6607 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66482176->66486271] (req 66482176->66486271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.391515:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6607 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66482176->66486271] (req 66482176->66486271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.391525:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6607 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66482176->66486271] (req 66482176->66486271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.391534:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760789.391540:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6607 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66482176->66486271] (req 66482176->66486271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.391564:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6607 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66482176->66486271] (req 66482176->66486271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.391613:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6607 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66482176->66486271] (req 66482176->66486271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.391756:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6607 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66482176->66486271] (req 66482176->66486271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.392088:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0985b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554844928:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13226us (13545us total) trans 4294983540 rc 0/0 00000100:00100000:0.0:1789760789.392102:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16242 00000100:00100000:0.0:1789760789.392114:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732604884453 time 732624460231 state 0x2 ref 0 00000100:00100000:2.0:1789760789.399019:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760789.399542:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554845056 00000100:00100000:3.0:1789760789.399559:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554845056 00000100:00100000:3.0:1789760789.399651:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732604884453 time 732624460231 state 0x0 ref 1 00000100:00100000:3.0:1789760789.399669:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732624720647 time 732624460231 state 0x1 ref 1 00000100:00100000:3.0:1789760789.399706:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732624720647 time 732645324419 state 0x0 ref 1 00000100:00100000:3.0:1789760789.399720:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09859500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554845056:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760789.399786:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66486272, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fb58 00010000:00010000:3.0:1789760789.399797:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c660e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.399816:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760789.399821:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.399829:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c660e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66486272->66490367] (req 66486272->66490367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.399846:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c660e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66486272->66490367] (req 66486272->66490367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.412072:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983541 is committed 00002000:00010000:3.0:1789760789.412245:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66486272 -> 66490368 00002000:00010000:3.0:1789760789.412251:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129880 -> 129888 00010000:00010000:3.0:1789760789.412267:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c660e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66486272->66490367] (req 66486272->66490367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.412282:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c660e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66486272->66490367] (req 66486272->66490367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.412297:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c660e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66486272->66490367] (req 66486272->66490367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.412305:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760789.412310:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c660e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66486272->66490367] (req 66486272->66490367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.412320:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c660e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66486272->66490367] (req 66486272->66490367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.412342:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c660e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66486272->66490367] (req 66486272->66490367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.412349:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c660e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66486272->66490367] (req 66486272->66490367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.412564:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09859500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554845056:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12840us (13564us total) trans 4294983541 rc 0/0 00000100:00100000:3.0:1789760789.412576:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16243 00000100:00100000:3.0:1789760789.412588:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732624720647 time 732645324419 state 0x2 ref 0 00000100:00100000:2.0:1789760789.419754:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760789.420194:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554845184 00000100:00100000:1.0:1789760789.420208:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554845184 00000100:00100000:1.0:1789760789.420281:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732624720647 time 732645324419 state 0x0 ref 1 00000100:00100000:1.0:1789760789.420300:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732645584835 time 732645324419 state 0x1 ref 1 00000100:00100000:1.0:1789760789.420414:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732645584835 time 732666032265 state 0x0 ref 1 00000100:00100000:1.0:1789760789.420429:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09858e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554845184:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760789.420495:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66490368, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fb60 00010000:00010000:1.0:1789760789.420505:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6615 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.420525:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760789.420530:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.420538:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6615 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66490368->66494463] (req 66490368->66494463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.420554:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6615 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66490368->66494463] (req 66490368->66494463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.435595:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983542 is committed 00002000:00010000:2.0:1789760789.435872:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66490368 -> 66494464 00002000:00010000:2.0:1789760789.435878:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129888 -> 129896 00010000:00010000:2.0:1789760789.435897:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6615 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66490368->66494463] (req 66490368->66494463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.435912:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6615 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66490368->66494463] (req 66490368->66494463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.435922:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6615 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66490368->66494463] (req 66490368->66494463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.435930:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760789.435937:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6615 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66490368->66494463] (req 66490368->66494463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.435948:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6615 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66490368->66494463] (req 66490368->66494463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.435979:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6615 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66490368->66494463] (req 66490368->66494463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.435988:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6615 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66490368->66494463] (req 66490368->66494463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.436840:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09858e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554845184:12345-192.168.203.22@tcp:4:dd.0 Request processed in 16403us (17089us total) trans 4294983542 rc 0/0 00000100:00100000:2.0:1789760789.436855:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16244 00000100:00100000:2.0:1789760789.436866:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732645584835 time 732666032265 state 0x2 ref 0 00000100:00100000:2.0:1789760789.447216:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760789.448778:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554845312 00000100:00100000:1.0:1789760789.448795:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554845312 00000100:00100000:1.0:1789760789.448878:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732645584835 time 732666032265 state 0x0 ref 1 00000100:00100000:1.0:1789760789.448898:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732666292681 time 732666032265 state 0x1 ref 1 00000100:00100000:1.0:1789760789.448938:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732666292681 time 732694556484 state 0x0 ref 1 00000100:00100000:1.0:1789760789.448954:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0985b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554845312:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760789.449019:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66494464, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fb68 00010000:00010000:1.0:1789760789.449031:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c661c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.449058:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760789.449063:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.449071:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c661c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66494464->66498559] (req 66494464->66498559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.449087:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c661c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66494464->66498559] (req 66494464->66498559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.468908:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983543 is committed 00002000:00010000:1.0:1789760789.469454:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66494464 -> 66498560 00002000:00010000:1.0:1789760789.469462:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129896 -> 129904 00010000:00010000:1.0:1789760789.469481:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c661c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66494464->66498559] (req 66494464->66498559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.469496:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c661c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66494464->66498559] (req 66494464->66498559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.469507:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c661c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66494464->66498559] (req 66494464->66498559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.469516:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760789.469522:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c661c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66494464->66498559] (req 66494464->66498559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.469532:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c661c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66494464->66498559] (req 66494464->66498559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.469563:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c661c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66494464->66498559] (req 66494464->66498559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.469572:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c661c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66494464->66498559] (req 66494464->66498559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.469757:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0985b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554845312:12345-192.168.203.22@tcp:4:dd.0 Request processed in 20800us (22552us total) trans 4294983543 rc 0/0 00000100:00100000:1.0:1789760789.469771:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16245 00000100:00100000:1.0:1789760789.469784:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732666292681 time 732694556484 state 0x2 ref 0 00000100:00100000:0.0:1789760789.479072:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760789.479641:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554845440 00000100:00100000:2.0:1789760789.479654:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554845440 00000100:00100000:2.0:1789760789.479730:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732666292681 time 732694556484 state 0x0 ref 1 00000100:00100000:2.0:1789760789.479750:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732694816900 time 732694556484 state 0x1 ref 1 00000100:00100000:2.0:1789760789.479789:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732694816900 time 732725407065 state 0x0 ref 1 00000100:00100000:2.0:1789760789.479804:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554845440:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760789.479888:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66498560, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fb70 00010000:00010000:2.0:1789760789.479900:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6623 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.479954:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760789.479959:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.479976:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6623 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66498560->66502655] (req 66498560->66502655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.480012:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6623 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66498560->66502655] (req 66498560->66502655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.491672:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983544 is committed 00002000:00010000:3.0:1789760789.491812:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66498560 -> 66502656 00002000:00010000:3.0:1789760789.491819:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129904 -> 129912 00010000:00010000:3.0:1789760789.491837:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6623 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66498560->66502655] (req 66498560->66502655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.491859:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6623 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66498560->66502655] (req 66498560->66502655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.491869:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6623 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66498560->66502655] (req 66498560->66502655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.491877:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760789.491883:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6623 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66498560->66502655] (req 66498560->66502655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.491893:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6623 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66498560->66502655] (req 66498560->66502655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.491915:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6623 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66498560->66502655] (req 66498560->66502655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.491924:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6623 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66498560->66502655] (req 66498560->66502655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.492046:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554845440:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12238us (12983us total) trans 4294983544 rc 0/0 00000100:00100000:3.0:1789760789.492057:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16246 00000100:00100000:3.0:1789760789.492068:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732694816900 time 732725407065 state 0x2 ref 0 00000100:00100000:3.0:1789760789.495060:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760789.495247:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554845568 00000100:00100000:2.0:1789760789.495260:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554845568 00000100:00100000:2.0:1789760789.495327:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732694816900 time 732725407065 state 0x0 ref 1 00000100:00100000:2.0:1789760789.495345:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732725667481 time 732725407065 state 0x1 ref 1 00000100:00100000:2.0:1789760789.495410:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732725667481 time 732741027839 state 0x0 ref 1 00000100:00100000:2.0:1789760789.495423:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554845568:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760789.495488:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66502656, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fb78 00010000:00010000:2.0:1789760789.495499:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c662a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.495520:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760789.495524:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.495531:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c662a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66502656->66506751] (req 66502656->66506751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.495548:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c662a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66502656->66506751] (req 66502656->66506751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.504091:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983545 is committed 00002000:00010000:2.0:1789760789.504260:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66502656 -> 66506752 00002000:00010000:2.0:1789760789.504267:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129912 -> 129920 00010000:00010000:2.0:1789760789.504284:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c662a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66502656->66506751] (req 66502656->66506751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.504299:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c662a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66502656->66506751] (req 66502656->66506751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.504310:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c662a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66502656->66506751] (req 66502656->66506751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.504319:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760789.504325:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c662a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66502656->66506751] (req 66502656->66506751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.504336:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c662a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66502656->66506751] (req 66502656->66506751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.504361:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c662a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66502656->66506751] (req 66502656->66506751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.504371:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c662a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66502656->66506751] (req 66502656->66506751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.504799:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554845568:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9369us (9743us total) trans 4294983545 rc 0/0 00000100:00100000:3.0:1789760789.504813:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16247 00000100:00100000:3.0:1789760789.504825:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732725667481 time 732741027839 state 0x2 ref 0 00000100:00100000:2.0:1789760789.508352:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760789.508915:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554845696 00000100:00100000:1.0:1789760789.508930:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554845696 00000100:00100000:1.0:1789760789.509020:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732725667481 time 732741027839 state 0x0 ref 1 00000100:00100000:1.0:1789760789.509042:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732741288255 time 732741027839 state 0x1 ref 1 00000100:00100000:1.0:1789760789.509153:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732741288255 time 732754769542 state 0x0 ref 1 00000100:00100000:1.0:1789760789.509170:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0985ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554845696:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760789.509235:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66506752, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fb80 00010000:00010000:1.0:1789760789.509248:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6631 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.509274:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760789.509280:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.509289:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6631 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66506752->66510847] (req 66506752->66510847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.509304:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6631 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66506752->66510847] (req 66506752->66510847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.516199:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983546 is committed 00002000:00010000:2.0:1789760789.516455:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66506752 -> 66510848 00002000:00010000:2.0:1789760789.516460:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129920 -> 129928 00010000:00010000:2.0:1789760789.516477:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6631 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66506752->66510847] (req 66506752->66510847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.516492:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6631 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66506752->66510847] (req 66506752->66510847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.516502:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6631 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66506752->66510847] (req 66506752->66510847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.516510:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760789.516516:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6631 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66506752->66510847] (req 66506752->66510847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.516526:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6631 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66506752->66510847] (req 66506752->66510847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.516550:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6631 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66506752->66510847] (req 66506752->66510847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.516572:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6631 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66506752->66510847] (req 66506752->66510847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.516714:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0985ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554845696:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7540us (8369us total) trans 4294983546 rc 0/0 00000100:00100000:2.0:1789760789.516727:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16248 00000100:00100000:2.0:1789760789.516739:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732741288255 time 732754769542 state 0x2 ref 0 00000100:00100000:0.0:1789760789.520683:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760789.520830:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554845824 00000100:00100000:1.0:1789760789.520843:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554845824 00000100:00100000:1.0:1789760789.520917:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732741288255 time 732754769542 state 0x0 ref 1 00000100:00100000:1.0:1789760789.520937:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732755029958 time 732754769542 state 0x1 ref 1 00000100:00100000:1.0:1789760789.520972:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732755029958 time 732766590057 state 0x0 ref 1 00000100:00100000:1.0:1789760789.520987:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554845824:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760789.521062:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66510848, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fb88 00010000:00010000:1.0:1789760789.521075:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6638 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.521094:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760789.521100:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.521107:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6638 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66510848->66514943] (req 66510848->66514943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.521125:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6638 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66510848->66514943] (req 66510848->66514943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760789.533661:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983547 is committed 00002000:00010000:3.0:1789760789.533821:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66510848 -> 66514944 00002000:00010000:3.0:1789760789.533828:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129928 -> 129936 00010000:00010000:3.0:1789760789.533847:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6638 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66510848->66514943] (req 66510848->66514943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.533863:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6638 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66510848->66514943] (req 66510848->66514943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.533875:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6638 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66510848->66514943] (req 66510848->66514943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.533884:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760789.533891:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6638 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66510848->66514943] (req 66510848->66514943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.533902:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6638 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66510848->66514943] (req 66510848->66514943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.533930:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6638 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66510848->66514943] (req 66510848->66514943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.533939:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6638 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66510848->66514943] (req 66510848->66514943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.534187:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554845824:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13193us (13510us total) trans 4294983547 rc 0/0 00000100:00100000:3.0:1789760789.534201:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16249 00000100:00100000:3.0:1789760789.534213:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732755029958 time 732766590057 state 0x2 ref 0 00000100:00100000:3.0:1789760789.542643:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760789.543024:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554845952 00000100:00100000:1.0:1789760789.543038:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554845952 00000100:00100000:1.0:1789760789.543128:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732755029958 time 732766590057 state 0x0 ref 1 00000100:00100000:1.0:1789760789.543150:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732766850473 time 732766590057 state 0x1 ref 1 00000100:00100000:1.0:1789760789.543193:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732766850473 time 732788810731 state 0x0 ref 1 00000100:00100000:1.0:1789760789.543210:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554845952:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760789.543282:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66514944, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fb90 00010000:00010000:1.0:1789760789.543294:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c663f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.543313:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760789.543319:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.543327:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c663f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66514944->66519039] (req 66514944->66519039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.543343:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c663f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66514944->66519039] (req 66514944->66519039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.560462:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983548 is committed 00002000:00010000:1.0:1789760789.562481:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66514944 -> 66519040 00002000:00010000:1.0:1789760789.562495:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129936 -> 129944 00010000:00010000:1.0:1789760789.562515:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c663f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66514944->66519039] (req 66514944->66519039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.562530:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c663f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66514944->66519039] (req 66514944->66519039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.562540:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c663f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66514944->66519039] (req 66514944->66519039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.562548:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760789.562554:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c663f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66514944->66519039] (req 66514944->66519039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.562564:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c663f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66514944->66519039] (req 66514944->66519039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.562592:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c663f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66514944->66519039] (req 66514944->66519039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.562600:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c663f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66514944->66519039] (req 66514944->66519039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.562986:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554845952:12345-192.168.203.22@tcp:4:dd.0 Request processed in 19763us (20367us total) trans 4294983548 rc 0/0 00000100:00100000:1.0:1789760789.563008:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16250 00000100:00100000:1.0:1789760789.563021:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732766850473 time 732788810731 state 0x2 ref 0 00000100:00100000:2.0:1789760789.565900:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760789.566070:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554846080 00000100:00100000:3.0:1789760789.566081:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554846080 00000100:00100000:3.0:1789760789.566150:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732766850473 time 732788810731 state 0x0 ref 1 00000100:00100000:3.0:1789760789.566170:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732789071147 time 732788810731 state 0x1 ref 1 00000100:00100000:3.0:1789760789.566209:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732789071147 time 732811826781 state 0x0 ref 1 00000100:00100000:3.0:1789760789.566223:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09858000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554846080:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760789.566284:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66519040, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fb98 00010000:00010000:3.0:1789760789.566294:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6646 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.566315:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760789.566320:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.566327:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6646 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66519040->66523135] (req 66519040->66523135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.566364:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6646 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66519040->66523135] (req 66519040->66523135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.576875:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983549 is committed 00002000:00010000:3.0:1789760789.577024:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66519040 -> 66523136 00002000:00010000:3.0:1789760789.577033:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129944 -> 129952 00010000:00010000:3.0:1789760789.577057:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6646 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66519040->66523135] (req 66519040->66523135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.577077:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6646 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66519040->66523135] (req 66519040->66523135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.577088:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6646 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66519040->66523135] (req 66519040->66523135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.577097:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760789.577103:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6646 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66519040->66523135] (req 66519040->66523135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.577115:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6646 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66519040->66523135] (req 66519040->66523135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.577161:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6646 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66519040->66523135] (req 66519040->66523135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.577171:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6646 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66519040->66523135] (req 66519040->66523135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.577352:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09858000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554846080:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11122us (11457us total) trans 4294983549 rc 0/0 00000100:00100000:3.0:1789760789.577368:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16251 00000100:00100000:3.0:1789760789.577425:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732789071147 time 732811826781 state 0x2 ref 0 00000100:00100000:0.0:1789760789.591187:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760789.593775:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554846208 00000100:00100000:2.0:1789760789.593799:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554846208 00000100:00100000:2.0:1789760789.593896:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732789071147 time 732811826781 state 0x0 ref 1 00000100:00100000:2.0:1789760789.593916:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732812087197 time 732811826781 state 0x1 ref 1 00000100:00100000:2.0:1789760789.593960:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732812087197 time 732839577925 state 0x0 ref 1 00000100:00100000:2.0:1789760789.593977:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554846208:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760789.594051:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66523136, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fba0 00010000:00010000:2.0:1789760789.594065:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c664d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.594091:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760789.594097:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.594106:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c664d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66523136->66527231] (req 66523136->66527231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.594127:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c664d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66523136->66527231] (req 66523136->66527231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.607107:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983550 is committed 00002000:00010000:2.0:1789760789.607355:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66523136 -> 66527232 00002000:00010000:2.0:1789760789.607362:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129952 -> 129960 00010000:00010000:2.0:1789760789.607979:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c664d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66523136->66527231] (req 66523136->66527231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.607997:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c664d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66523136->66527231] (req 66523136->66527231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.608007:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c664d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66523136->66527231] (req 66523136->66527231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.608015:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760789.608020:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c664d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66523136->66527231] (req 66523136->66527231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.608031:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c664d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66523136->66527231] (req 66523136->66527231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.608060:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c664d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66523136->66527231] (req 66523136->66527231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.608070:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c664d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66523136->66527231] (req 66523136->66527231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.608581:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554846208:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14598us (17400us total) trans 4294983550 rc 0/0 00000100:00100000:3.0:1789760789.608596:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16252 00000100:00100000:3.0:1789760789.608609:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732812087197 time 732839577925 state 0x2 ref 0 00000100:00100000:1.0:1789760789.614710:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760789.614828:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554846336 00000100:00100000:3.0:1789760789.614844:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554846336 00000100:00100000:3.0:1789760789.614922:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732812087197 time 732839577925 state 0x0 ref 1 00000100:00100000:3.0:1789760789.614943:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732839838341 time 732839577925 state 0x1 ref 1 00000100:00100000:3.0:1789760789.614985:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732839838341 time 732860602491 state 0x0 ref 1 00000100:00100000:3.0:1789760789.615001:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554846336:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760789.615080:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66527232, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fba8 00010000:00010000:3.0:1789760789.615092:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6654 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.615114:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760789.615119:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.615129:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6654 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66527232->66531327] (req 66527232->66531327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.615146:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6654 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66527232->66531327] (req 66527232->66531327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.623797:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983551 is committed 00002000:00010000:3.0:1789760789.624311:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66527232 -> 66531328 00002000:00010000:3.0:1789760789.624318:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129960 -> 129968 00010000:00010000:3.0:1789760789.624336:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6654 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66527232->66531327] (req 66527232->66531327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.624353:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6654 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66527232->66531327] (req 66527232->66531327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.624364:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6654 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66527232->66531327] (req 66527232->66531327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.624372:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760789.624402:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6654 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66527232->66531327] (req 66527232->66531327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.624413:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6654 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66527232->66531327] (req 66527232->66531327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.624438:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6654 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66527232->66531327] (req 66527232->66531327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.624452:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c6654 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66527232->66531327] (req 66527232->66531327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.625305:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554846336:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10296us (10598us total) trans 4294983551 rc 0/0 00000100:00100000:3.0:1789760789.625320:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16253 00000100:00100000:3.0:1789760789.625330:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732839838341 time 732860602491 state 0x2 ref 0 00000100:00100000:3.0:1789760789.632914:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760789.633142:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554846464 00000100:00100000:1.0:1789760789.633156:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554846464 00000100:00100000:1.0:1789760789.633244:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732839838341 time 732860602491 state 0x0 ref 1 00000100:00100000:1.0:1789760789.633264:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732860862907 time 732860602491 state 0x1 ref 1 00000100:00100000:1.0:1789760789.633346:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732860862907 time 732878962679 state 0x0 ref 1 00000100:00100000:1.0:1789760789.633362:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe19050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554846464:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760789.633504:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66531328, mtime: 0x6aad9514, atime: 0x0, ctime: 0x6aad9514, blocks: 0x1fbb0 00010000:00010000:1.0:1789760789.633528:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c665b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.633552:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760789.633557:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.633565:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c665b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66531328->66535423] (req 66531328->66535423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.633583:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c665b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66531328->66535423] (req 66531328->66535423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.643248:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983552 is committed 00002000:00010000:1.0:1789760789.643399:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66531328 -> 66535424 00002000:00010000:1.0:1789760789.643405:0:8443:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760788 -> 1789760789 00002000:00010000:1.0:1789760789.643407:0:8443:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760788 -> 1789760789 00002000:00010000:1.0:1789760789.643410:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129968 -> 129976 00010000:00010000:1.0:1789760789.643423:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c665b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66531328->66535423] (req 66531328->66535423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.643437:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c665b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66531328->66535423] (req 66531328->66535423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.643447:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c665b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66531328->66535423] (req 66531328->66535423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.643454:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760789.643458:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c665b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66531328->66535423] (req 66531328->66535423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.643467:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c665b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66531328->66535423] (req 66531328->66535423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.643488:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c665b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66531328->66535423] (req 66531328->66535423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.643495:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c665b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66531328->66535423] (req 66531328->66535423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.643646:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe19050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554846464:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10282us (10755us total) trans 4294983552 rc 0/0 00000100:00100000:1.0:1789760789.643660:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16254 00000100:00100000:1.0:1789760789.643669:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732860862907 time 732878962679 state 0x2 ref 0 00000100:00100000:0.0:1789760789.651280:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760789.651416:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554846592 00000100:00100000:3.0:1789760789.651426:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554846592 00000100:00100000:3.0:1789760789.651486:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732860862907 time 732878962679 state 0x0 ref 1 00000100:00100000:3.0:1789760789.651501:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732879223095 time 732878962679 state 0x1 ref 1 00000100:00100000:3.0:1789760789.651527:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732879223095 time 732897145733 state 0x0 ref 1 00000100:00100000:3.0:1789760789.651538:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554846592:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760789.651583:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66535424, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fbb8 00010000:00010000:3.0:1789760789.651591:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6662 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.651607:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760789.651611:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.651618:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6662 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66535424->66539519] (req 66535424->66539519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.651646:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6662 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66535424->66539519] (req 66535424->66539519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.660301:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983553 is committed 00002000:00010000:3.0:1789760789.660503:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66535424 -> 66539520 00002000:00010000:3.0:1789760789.660509:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129976 -> 129984 00010000:00010000:3.0:1789760789.660522:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6662 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66535424->66539519] (req 66535424->66539519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.660535:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6662 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66535424->66539519] (req 66535424->66539519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.660544:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6662 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66535424->66539519] (req 66535424->66539519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.660551:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760789.660557:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6662 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66535424->66539519] (req 66535424->66539519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.660566:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6662 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66535424->66539519] (req 66535424->66539519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.660585:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6662 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66535424->66539519] (req 66535424->66539519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.660599:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6662 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66535424->66539519] (req 66535424->66539519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.660770:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554846592:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9226us (9492us total) trans 4294983553 rc 0/0 00000100:00100000:3.0:1789760789.660783:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16255 00000100:00100000:3.0:1789760789.660793:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732879223095 time 732897145733 state 0x2 ref 0 00000100:00100000:3.0:1789760789.671823:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760789.672759:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554846720 00000100:00100000:1.0:1789760789.672770:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554846720 00000100:00100000:1.0:1789760789.672831:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732879223095 time 732897145733 state 0x0 ref 1 00000100:00100000:1.0:1789760789.672849:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732897406149 time 732897145733 state 0x1 ref 1 00000100:00100000:1.0:1789760789.672886:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732897406149 time 732918504537 state 0x0 ref 1 00000100:00100000:1.0:1789760789.672898:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554846720:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760789.672948:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66539520, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fbc0 00010000:00010000:1.0:1789760789.672958:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6669 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.672974:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:1.0:1789760789.672978:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.672984:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6669 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66539520->66543615] (req 66539520->66543615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.672998:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6669 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66539520->66543615] (req 66539520->66543615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.681791:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983554 is committed 00002000:00010000:1.0:1789760789.682249:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66539520 -> 66543616 00002000:00010000:1.0:1789760789.682254:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129984 -> 129992 00010000:00010000:1.0:1789760789.682268:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6669 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66539520->66543615] (req 66539520->66543615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.682280:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6669 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66539520->66543615] (req 66539520->66543615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.682289:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6669 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66539520->66543615] (req 66539520->66543615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.682297:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760789.682302:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6669 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66539520->66543615] (req 66539520->66543615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.682311:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6669 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66539520->66543615] (req 66539520->66543615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.682330:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6669 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66539520->66543615] (req 66539520->66543615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.682337:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6669 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66539520->66543615] (req 66539520->66543615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.682669:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554846720:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9768us (10851us total) trans 4294983554 rc 0/0 00000100:00100000:1.0:1789760789.682680:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16256 00000100:00100000:1.0:1789760789.682690:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732897406149 time 732918504537 state 0x2 ref 0 00000100:00100000:1.0:1789760789.689800:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760789.689898:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554846848 00000100:00100000:3.0:1789760789.689909:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554846848 00000100:00100000:3.0:1789760789.689959:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732897406149 time 732918504537 state 0x0 ref 1 00000100:00100000:3.0:1789760789.689974:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732918764953 time 732918504537 state 0x1 ref 1 00000100:00100000:3.0:1789760789.690009:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732918764953 time 732935627267 state 0x0 ref 1 00000100:00100000:3.0:1789760789.690022:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554846848:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760789.690069:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66543616, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fbc8 00010000:00010000:3.0:1789760789.690078:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6670 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.690093:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760789.690098:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.690104:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6670 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66543616->66547711] (req 66543616->66547711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.690118:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6670 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66543616->66547711] (req 66543616->66547711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.699885:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983555 is committed 00002000:00010000:3.0:1789760789.700068:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66543616 -> 66547712 00002000:00010000:3.0:1789760789.700074:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 129992 -> 130000 00010000:00010000:3.0:1789760789.700087:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6670 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66543616->66547711] (req 66543616->66547711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.700100:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6670 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66543616->66547711] (req 66543616->66547711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.700109:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6670 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66543616->66547711] (req 66543616->66547711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.700117:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760789.700122:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6670 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66543616->66547711] (req 66543616->66547711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.700131:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6670 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66543616->66547711] (req 66543616->66547711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.700150:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6670 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66543616->66547711] (req 66543616->66547711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.700158:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6670 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66543616->66547711] (req 66543616->66547711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.700256:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554846848:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10232us (10461us total) trans 4294983555 rc 0/0 00000100:00100000:3.0:1789760789.700267:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16257 00000100:00100000:3.0:1789760789.700278:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732918764953 time 732935627267 state 0x2 ref 0 00000100:00100000:0.0:1789760789.705299:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760789.705659:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554846976 00000100:00100000:3.0:1789760789.705669:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554846976 00000100:00100000:3.0:1789760789.705724:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732918764953 time 732935627267 state 0x0 ref 1 00000100:00100000:3.0:1789760789.705740:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732935887683 time 732935627267 state 0x1 ref 1 00000100:00100000:3.0:1789760789.705774:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732935887683 time 732951392619 state 0x0 ref 1 00000100:00100000:3.0:1789760789.705788:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554846976:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760789.705996:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66547712, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fbd0 00010000:00010000:2.0:1789760789.706006:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6677 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.706023:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:2.0:1789760789.706027:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.706033:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6677 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66547712->66551807] (req 66547712->66551807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.706046:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6677 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66547712->66551807] (req 66547712->66551807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.713971:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983556 is committed 00002000:00010000:2.0:1789760789.714140:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66547712 -> 66551808 00002000:00010000:2.0:1789760789.714146:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130000 -> 130008 00010000:00010000:2.0:1789760789.714158:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6677 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66547712->66551807] (req 66547712->66551807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.714171:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6677 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66547712->66551807] (req 66547712->66551807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.714179:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6677 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66547712->66551807] (req 66547712->66551807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.714187:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760789.714192:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6677 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66547712->66551807] (req 66547712->66551807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.714202:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6677 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66547712->66551807] (req 66547712->66551807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.714222:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6677 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66547712->66551807] (req 66547712->66551807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.714230:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6677 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66547712->66551807] (req 66547712->66551807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.714322:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554846976:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8533us (9028us total) trans 4294983556 rc 0/0 00000100:00100000:2.0:1789760789.714332:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16258 00000100:00100000:2.0:1789760789.714341:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732935887683 time 732951392619 state 0x2 ref 0 00000100:00100000:1.0:1789760789.719221:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760789.719428:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554847104 00000100:00100000:0.0:1789760789.719780:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554847104 00000100:00100000:0.0:1789760789.719841:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732935887683 time 732951392619 state 0x0 ref 1 00000100:00100000:0.0:1789760789.719856:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732951653035 time 732951392619 state 0x1 ref 1 00000100:00100000:0.0:1789760789.719887:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732951653035 time 732965506032 state 0x0 ref 1 00000100:00100000:0.0:1789760789.719900:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554847104:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760789.720197:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66551808, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fbd8 00010000:00010000:0.0:1789760789.720207:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c667e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.720223:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:0.0:1789760789.720228:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.720235:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c667e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66551808->66555903] (req 66551808->66555903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.720248:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c667e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66551808->66555903] (req 66551808->66555903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.726843:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983557 is committed 00002000:00010000:3.0:1789760789.726946:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66551808 -> 66555904 00002000:00010000:3.0:1789760789.726951:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130008 -> 130016 00010000:00010000:3.0:1789760789.726965:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c667e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66551808->66555903] (req 66551808->66555903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.726977:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c667e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66551808->66555903] (req 66551808->66555903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.726987:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c667e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66551808->66555903] (req 66551808->66555903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.727060:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760789.727066:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c667e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66551808->66555903] (req 66551808->66555903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.727076:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c667e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66551808->66555903] (req 66551808->66555903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.727095:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c667e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66551808->66555903] (req 66551808->66555903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.727102:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c667e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66551808->66555903] (req 66551808->66555903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.727189:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554847104:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7286us (7971us total) trans 4294983557 rc 0/0 00000100:00100000:3.0:1789760789.727200:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16259 00000100:00100000:3.0:1789760789.727210:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732951653035 time 732965506032 state 0x2 ref 0 00000100:00100000:1.0:1789760789.734903:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760789.735330:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554847232 00000100:00100000:3.0:1789760789.735341:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554847232 00000100:00100000:3.0:1789760789.735416:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732951653035 time 732965506032 state 0x0 ref 1 00000100:00100000:3.0:1789760789.735431:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732965766448 time 732965506032 state 0x1 ref 1 00000100:00100000:3.0:1789760789.735514:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732965766448 time 732981132086 state 0x0 ref 1 00000100:00100000:3.0:1789760789.735541:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554847232:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760789.735588:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66555904, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fbe0 00010000:00010000:3.0:1789760789.735597:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6685 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.735613:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760789.735618:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.735635:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6685 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66555904->66559999] (req 66555904->66559999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.735650:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6685 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66555904->66559999] (req 66555904->66559999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.743576:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983558 is committed 00002000:00010000:3.0:1789760789.743705:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66555904 -> 66560000 00002000:00010000:3.0:1789760789.743710:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130016 -> 130024 00010000:00010000:3.0:1789760789.743723:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6685 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66555904->66559999] (req 66555904->66559999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.743736:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6685 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66555904->66559999] (req 66555904->66559999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.743745:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6685 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66555904->66559999] (req 66555904->66559999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.743752:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760789.743757:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6685 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66555904->66559999] (req 66555904->66559999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.743766:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6685 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66555904->66559999] (req 66555904->66559999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.743786:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6685 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66555904->66559999] (req 66555904->66559999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.743794:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c6685 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66555904->66559999] (req 66555904->66559999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.743889:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554847232:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8346us (8992us total) trans 4294983558 rc 0/0 00000100:00100000:3.0:1789760789.743899:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16260 00000100:00100000:3.0:1789760789.743909:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732965766448 time 732981132086 state 0x2 ref 0 00000100:00100000:0.0:1789760789.751932:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760789.752029:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554847360 00000100:00100000:3.0:1789760789.752039:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554847360 00000100:00100000:3.0:1789760789.752094:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732965766448 time 732981132086 state 0x0 ref 1 00000100:00100000:3.0:1789760789.752111:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732981392502 time 732981132086 state 0x1 ref 1 00000100:00100000:3.0:1789760789.752146:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732981392502 time 732997764031 state 0x0 ref 1 00000100:00100000:3.0:1789760789.752159:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554847360:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760789.752206:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66560000, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fbe8 00010000:00010000:3.0:1789760789.752216:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c668c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.752233:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=732s v=(0 0) 00010000:00010000:3.0:1789760789.752237:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.752244:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c668c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66560000->66564095] (req 66560000->66564095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.752259:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c668c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66560000->66564095] (req 66560000->66564095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.758513:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983559 is committed 00002000:00010000:3.0:1789760789.758650:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66560000 -> 66564096 00002000:00010000:3.0:1789760789.758656:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130024 -> 130032 00010000:00010000:3.0:1789760789.758669:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c668c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66560000->66564095] (req 66560000->66564095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.758683:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c668c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66560000->66564095] (req 66560000->66564095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.758692:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c668c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66560000->66564095] (req 66560000->66564095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.758794:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760789.758799:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c668c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66560000->66564095] (req 66560000->66564095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.758810:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c668c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66560000->66564095] (req 66560000->66564095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.758831:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c668c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66560000->66564095] (req 66560000->66564095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.758839:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c668c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66560000->66564095] (req 66560000->66564095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.758992:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554847360:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6829us (7063us total) trans 4294983559 rc 0/0 00000100:00100000:3.0:1789760789.759014:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16261 00000100:00100000:3.0:1789760789.759025:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732981392502 time 732997764031 state 0x2 ref 0 00000100:00100000:1.0:1789760789.766115:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760789.766252:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554847488 00000100:00100000:2.0:1789760789.766266:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554847488 00000100:00100000:2.0:1789760789.766670:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732981392502 time 732997764031 state 0x0 ref 1 00000100:00100000:2.0:1789760789.766689:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732998024447 time 732997764031 state 0x1 ref 1 00000100:00100000:2.0:1789760789.766726:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732998024447 time 733012343697 state 0x0 ref 1 00000100:00100000:2.0:1789760789.766741:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554847488:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760789.766805:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66564096, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fbf0 00010000:00010000:2.0:1789760789.766818:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6693 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.766837:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760789.766843:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.766850:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6693 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66564096->66568191] (req 66564096->66568191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.766866:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6693 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66564096->66568191] (req 66564096->66568191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.772593:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983560 is committed 00002000:00010000:2.0:1789760789.772795:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66564096 -> 66568192 00002000:00010000:2.0:1789760789.772801:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130032 -> 130040 00010000:00010000:2.0:1789760789.772814:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6693 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66564096->66568191] (req 66564096->66568191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.772827:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6693 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66564096->66568191] (req 66564096->66568191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.772836:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6693 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66564096->66568191] (req 66564096->66568191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.772844:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760789.772849:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6693 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66564096->66568191] (req 66564096->66568191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.772857:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6693 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66564096->66568191] (req 66564096->66568191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.772877:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6693 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66564096->66568191] (req 66564096->66568191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.772891:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6693 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66564096->66568191] (req 66564096->66568191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.772988:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554847488:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6247us (6885us total) trans 4294983560 rc 0/0 00000100:00100000:2.0:1789760789.773007:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16262 00000100:00100000:2.0:1789760789.773017:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732998024447 time 733012343697 state 0x2 ref 0 00000100:00100000:1.0:1789760789.779015:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760789.779141:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554847616 00000100:00100000:0.0:1789760789.779154:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554847616 00000100:00100000:0.0:1789760789.779229:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 732998024447 time 733012343697 state 0x0 ref 1 00000100:00100000:0.0:1789760789.779251:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733012604113 time 733012343697 state 0x1 ref 1 00000100:00100000:0.0:1789760789.779300:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733012604113 time 733024917749 state 0x0 ref 1 00000100:00100000:0.0:1789760789.779315:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554847616:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760789.779418:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66568192, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fbf8 00010000:00010000:0.0:1789760789.779431:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c669a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.779451:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760789.779457:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760789.779466:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c669a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66568192->66572287] (req 66568192->66572287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.779484:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c669a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66568192->66572287] (req 66568192->66572287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.785878:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983561 is committed 00002000:00010000:0.0:1789760789.787971:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66568192 -> 66572288 00002000:00010000:0.0:1789760789.787985:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130040 -> 130048 00010000:00010000:0.0:1789760789.788002:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c669a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66568192->66572287] (req 66568192->66572287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.788018:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c669a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66568192->66572287] (req 66568192->66572287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.788028:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c669a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66568192->66572287] (req 66568192->66572287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.788036:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760789.788042:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c669a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66568192->66572287] (req 66568192->66572287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.788058:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c669a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66568192->66572287] (req 66568192->66572287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.788083:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c669a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66568192->66572287] (req 66568192->66572287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.788092:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c669a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66568192->66572287] (req 66568192->66572287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.788238:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554847616:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8918us (9231us total) trans 4294983561 rc 0/0 00000100:00100000:0.0:1789760789.788252:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16263 00000100:00100000:0.0:1789760789.788264:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733012604113 time 733024917749 state 0x2 ref 0 00000100:00100000:0.0:1789760789.797148:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760789.797468:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554847744 00000100:00100000:3.0:1789760789.797482:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554847744 00000100:00100000:3.0:1789760789.797563:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733012604113 time 733024917749 state 0x0 ref 1 00000100:00100000:3.0:1789760789.797583:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733025178165 time 733024917749 state 0x1 ref 1 00000100:00100000:3.0:1789760789.797859:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733025178165 time 733043474942 state 0x0 ref 1 00000100:00100000:3.0:1789760789.797878:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554847744:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760789.797953:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66572288, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fc00 00010000:00010000:3.0:1789760789.797966:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c66a1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.797992:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760789.797997:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.798006:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c66a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66572288->66576383] (req 66572288->66576383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.798022:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c66a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66572288->66576383] (req 66572288->66576383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.812283:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983562 is committed 00002000:00010000:2.0:1789760789.812877:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66572288 -> 66576384 00002000:00010000:2.0:1789760789.812882:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130048 -> 130056 00010000:00010000:2.0:1789760789.812897:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c66a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66572288->66576383] (req 66572288->66576383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.812911:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c66a1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66572288->66576383] (req 66572288->66576383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.812921:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c66a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66572288->66576383] (req 66572288->66576383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.812928:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760789.812933:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c66a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66572288->66576383] (req 66572288->66576383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.812942:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c66a1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66572288->66576383] (req 66572288->66576383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.812965:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c66a1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66572288->66576383] (req 66572288->66576383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.812973:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c66a1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66572288->66576383] (req 66572288->66576383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.813370:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554847744:12345-192.168.203.22@tcp:4:dd.0 Request processed in 15491us (16231us total) trans 4294983562 rc 0/0 00000100:00100000:2.0:1789760789.815242:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16264 00000100:00100000:2.0:1789760789.815256:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733025178165 time 733043474942 state 0x2 ref 0 00000100:00100000:1.0:1789760789.824504:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760789.824652:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554847872 00000100:00100000:3.0:1789760789.824663:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554847872 00000100:00100000:3.0:1789760789.824727:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733025178165 time 733043474942 state 0x0 ref 1 00000100:00100000:3.0:1789760789.824914:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733043735358 time 733043474942 state 0x1 ref 1 00000100:00100000:3.0:1789760789.824948:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733043735358 time 733070565533 state 0x0 ref 1 00000100:00100000:3.0:1789760789.824960:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554847872:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760789.825184:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66576384, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fc08 00010000:00010000:3.0:1789760789.825194:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66a8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.825209:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760789.825213:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.825219:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66576384->66580479] (req 66576384->66580479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.825238:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66576384->66580479] (req 66576384->66580479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.833758:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983563 is committed 00002000:00010000:3.0:1789760789.834274:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66576384 -> 66580480 00002000:00010000:3.0:1789760789.834279:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130056 -> 130064 00010000:00010000:3.0:1789760789.834293:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66576384->66580479] (req 66576384->66580479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.834306:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66a8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66576384->66580479] (req 66576384->66580479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.834315:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66576384->66580479] (req 66576384->66580479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.834323:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760789.834328:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66576384->66580479] (req 66576384->66580479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.834337:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66a8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66576384->66580479] (req 66576384->66580479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.834357:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66a8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66576384->66580479] (req 66576384->66580479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.834365:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66a8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66576384->66580479] (req 66576384->66580479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.834684:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554847872:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9717us (10181us total) trans 4294983563 rc 0/0 00000100:00100000:2.0:1789760789.834696:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16265 00000100:00100000:2.0:1789760789.834707:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733043735358 time 733070565533 state 0x2 ref 0 00000100:00100000:1.0:1789760789.847928:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760789.847998:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554848000 00000100:00100000:2.0:1789760789.848008:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554848000 00000100:00100000:2.0:1789760789.848062:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733043735358 time 733070565533 state 0x0 ref 1 00000100:00100000:2.0:1789760789.848078:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733070825949 time 733070565533 state 0x1 ref 1 00000100:00100000:2.0:1789760789.848116:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733070825949 time 733093735081 state 0x0 ref 1 00000100:00100000:2.0:1789760789.848138:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554848000:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760789.848193:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66580480, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fc10 00010000:00010000:2.0:1789760789.848203:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66af lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.848218:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760789.848223:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.848229:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66580480->66584575] (req 66580480->66584575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.848244:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66580480->66584575] (req 66580480->66584575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.856441:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983564 is committed 00002000:00010000:2.0:1789760789.856834:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66580480 -> 66584576 00002000:00010000:2.0:1789760789.856841:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130064 -> 130072 00010000:00010000:2.0:1789760789.856855:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66580480->66584575] (req 66580480->66584575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.856868:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66af lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66580480->66584575] (req 66580480->66584575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.856877:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66580480->66584575] (req 66580480->66584575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.856884:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760789.856889:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66580480->66584575] (req 66580480->66584575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.856903:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66af lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66580480->66584575] (req 66580480->66584575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.856923:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66af lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66580480->66584575] (req 66580480->66584575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.856930:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66af lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66580480->66584575] (req 66580480->66584575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.857025:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554848000:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8893us (9104us total) trans 4294983564 rc 0/0 00000100:00100000:2.0:1789760789.857035:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16266 00000100:00100000:2.0:1789760789.857045:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733070825949 time 733093735081 state 0x2 ref 0 00000100:00100000:0.0:1789760789.862561:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760789.862656:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554848128 00000100:00100000:3.0:1789760789.862666:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554848128 00000100:00100000:3.0:1789760789.862726:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733070825949 time 733093735081 state 0x0 ref 1 00000100:00100000:3.0:1789760789.862857:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733093995497 time 733093735081 state 0x1 ref 1 00000100:00100000:3.0:1789760789.862890:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733093995497 time 733108508087 state 0x0 ref 1 00000100:00100000:3.0:1789760789.862903:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554848128:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760789.862948:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66584576, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fc18 00010000:00010000:3.0:1789760789.862957:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66b6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.862973:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760789.862977:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.862984:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66584576->66588671] (req 66584576->66588671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.862998:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66584576->66588671] (req 66584576->66588671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.878590:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983565 is committed 00002000:00010000:3.0:1789760789.878737:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66584576 -> 66588672 00002000:00010000:3.0:1789760789.878743:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130072 -> 130080 00010000:00010000:3.0:1789760789.878757:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66584576->66588671] (req 66584576->66588671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.878773:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66b6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66584576->66588671] (req 66584576->66588671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.878782:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66584576->66588671] (req 66584576->66588671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.878789:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760789.878794:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66584576->66588671] (req 66584576->66588671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.878804:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66b6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66584576->66588671] (req 66584576->66588671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.878823:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66b6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66584576->66588671] (req 66584576->66588671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.878831:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66b6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66584576->66588671] (req 66584576->66588671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.878928:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554848128:12345-192.168.203.22@tcp:4:dd.0 Request processed in 16022us (16371us total) trans 4294983565 rc 0/0 00000100:00100000:3.0:1789760789.878939:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16267 00000100:00100000:3.0:1789760789.878948:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733093995497 time 733108508087 state 0x2 ref 0 00000100:00100000:1.0:1789760789.884993:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760789.885108:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554848256 00000100:00100000:2.0:1789760789.885119:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554848256 00000100:00100000:2.0:1789760789.885186:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733093995497 time 733108508087 state 0x0 ref 1 00000100:00100000:2.0:1789760789.885200:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733108768503 time 733108508087 state 0x1 ref 1 00000100:00100000:2.0:1789760789.885231:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733108768503 time 733130849701 state 0x0 ref 1 00000100:00100000:2.0:1789760789.885243:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554848256:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760789.885286:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66588672, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fc20 00010000:00010000:2.0:1789760789.885295:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66bd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.885311:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760789.885315:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.885321:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66588672->66592767] (req 66588672->66592767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.885335:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66588672->66592767] (req 66588672->66592767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.892835:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983566 is committed 00002000:00010000:2.0:1789760789.900813:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66588672 -> 66592768 00002000:00010000:2.0:1789760789.900825:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130080 -> 130088 00010000:00010000:2.0:1789760789.900838:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66588672->66592767] (req 66588672->66592767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.900852:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66bd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66588672->66592767] (req 66588672->66592767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.900861:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66588672->66592767] (req 66588672->66592767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.900868:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760789.900874:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66588672->66592767] (req 66588672->66592767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.900882:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66588672->66592767] (req 66588672->66592767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.900911:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66bd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66588672->66592767] (req 66588672->66592767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.900918:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66bd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66588672->66592767] (req 66588672->66592767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.901099:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554848256:12345-192.168.203.22@tcp:4:dd.0 Request processed in 15851us (16108us total) trans 4294983566 rc 0/0 00000100:00100000:2.0:1789760789.901110:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16268 00000100:00100000:2.0:1789760789.901120:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733108768503 time 733130849701 state 0x2 ref 0 00000100:00100000:3.0:1789760789.905939:0:3647:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb0985a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3647:1876699573921024:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760789.906189:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789760789.906212:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760789.906218:0:3649:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb046fb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3649:1876699573921152:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789760789.906248:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760789.906282:0:8439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573921024 00000100:00100000:3.0:1789760789.906289:0:8439:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573921024 00000100:00100000:0.0:1789760789.906532:0:11526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573921152 00000100:00100000:0.0:1789760789.906536:0:11526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573921152 00000100:00100000:0.0:1789760789.906599:0:11526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 368 00000100:00080000:3.0:1789760789.906605:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760789.906609:0:11526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1876699573921152:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1789760789.906642:0:11526:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484916928 pending 0 free 3856953344 avail 3625230336 00000100:00100000:0.0:1789760789.906916:0:11526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1876699573921152:12345-0@lo:13:osp-pre-0-1.0 Request processed in 307us (665us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760789.906924:0:11526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 368 00000100:00100000:1.0:1789760789.907028:0:8439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 369 00000100:00100000:1.0:1789760789.907036:0:8439:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3647:x1876699573921024:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760789.907048:0:8439:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484916928 pending 0 free 3856953344 avail 3625230336 00000100:00100000:1.0:1789760789.907087:0:8439:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3647:x1876699573921024:12345-0@lo:13:osp-pre-0-0.0 Request processed in 52us (875us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760789.907094:0:8439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 369 00000100:00100000:1.0:1789760789.907138:0:3647:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb0985a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3647:1876699573921024:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789760789.907415:0:3649:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb046fb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3649:1876699573921152:0@lo:13:osp-pre-0-1.0 00000100:00080000:0.0:1789760789.907519:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760789.910597:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760789.910687:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554848384 00000100:00100000:2.0:1789760789.910698:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554848384 00000100:00100000:2.0:1789760789.910753:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733108768503 time 733130849701 state 0x0 ref 1 00000100:00100000:2.0:1789760789.910768:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733131110117 time 733130849701 state 0x1 ref 1 00000100:00100000:2.0:1789760789.910801:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733131110117 time 733156418644 state 0x0 ref 1 00000100:00100000:2.0:1789760789.910816:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554848384:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760789.910868:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66592768, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fc28 00010000:00010000:2.0:1789760789.910878:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66c4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.910894:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760789.910900:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760789.910906:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66592768->66596863] (req 66592768->66596863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.910921:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66592768->66596863] (req 66592768->66596863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.918430:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983567 is committed 00002000:00010000:2.0:1789760789.918590:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66592768 -> 66596864 00002000:00010000:2.0:1789760789.918596:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130088 -> 130096 00010000:00010000:2.0:1789760789.918612:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66592768->66596863] (req 66592768->66596863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.918635:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66592768->66596863] (req 66592768->66596863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.918644:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66592768->66596863] (req 66592768->66596863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.918651:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760789.918656:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66592768->66596863] (req 66592768->66596863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.918665:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66592768->66596863] (req 66592768->66596863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.918686:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66c4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66592768->66596863] (req 66592768->66596863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760789.918698:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c66c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66592768->66596863] (req 66592768->66596863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760789.918872:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554848384:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8054us (8279us total) trans 4294983567 rc 0/0 00000100:00100000:2.0:1789760789.918885:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16269 00000100:00100000:2.0:1789760789.918894:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733131110117 time 733156418644 state 0x2 ref 0 00000100:00100000:0.0:1789760789.926101:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760789.926250:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554848512 00000100:00100000:1.0:1789760789.926261:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554848512 00000100:00100000:1.0:1789760789.926317:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733131110117 time 733156418644 state 0x0 ref 1 00000100:00100000:1.0:1789760789.926334:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733156679060 time 733156418644 state 0x1 ref 1 00000100:00100000:1.0:1789760789.926367:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733156679060 time 733171985954 state 0x0 ref 1 00000100:00100000:1.0:1789760789.926421:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554848512:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760789.926476:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66596864, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fc30 00010000:00010000:1.0:1789760789.926487:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c66cb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.926504:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760789.926508:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.926514:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c66cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66596864->66600959] (req 66596864->66600959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.926529:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c66cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66596864->66600959] (req 66596864->66600959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.934859:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983568 is committed 00002000:00010000:1.0:1789760789.935144:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66596864 -> 66600960 00002000:00010000:1.0:1789760789.935150:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130096 -> 130104 00010000:00010000:1.0:1789760789.935164:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c66cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66596864->66600959] (req 66596864->66600959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.935177:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c66cb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66596864->66600959] (req 66596864->66600959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.935186:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c66cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66596864->66600959] (req 66596864->66600959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.935192:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760789.935198:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c66cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66596864->66600959] (req 66596864->66600959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.935211:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c66cb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66596864->66600959] (req 66596864->66600959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.935230:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c66cb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66596864->66600959] (req 66596864->66600959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.935238:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c66cb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66596864->66600959] (req 66596864->66600959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.936262:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554848512:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9876us (10164us total) trans 4294983568 rc 0/0 00000100:00100000:1.0:1789760789.936274:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16270 00000100:00100000:1.0:1789760789.936283:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733156679060 time 733171985954 state 0x2 ref 0 00000100:00100000:1.0:1789760789.943920:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760789.944688:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554848640 00000100:00100000:3.0:1789760789.944700:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554848640 00000100:00100000:3.0:1789760789.944767:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733156679060 time 733171985954 state 0x0 ref 1 00000100:00100000:3.0:1789760789.944782:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733172246370 time 733171985954 state 0x1 ref 1 00000100:00100000:3.0:1789760789.944812:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733172246370 time 733190430049 state 0x0 ref 1 00000100:00100000:3.0:1789760789.944824:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554848640:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760789.944869:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66600960, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fc38 00010000:00010000:3.0:1789760789.944878:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66d2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.944894:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760789.944899:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.944906:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66600960->66605055] (req 66600960->66605055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.944919:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66600960->66605055] (req 66600960->66605055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760789.952206:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983569 is committed 00002000:00010000:3.0:1789760789.952328:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66600960 -> 66605056 00002000:00010000:3.0:1789760789.952334:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130104 -> 130112 00010000:00010000:3.0:1789760789.952347:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66600960->66605055] (req 66600960->66605055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.952359:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66600960->66605055] (req 66600960->66605055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.952367:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66600960->66605055] (req 66600960->66605055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.952374:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760789.952828:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66600960->66605055] (req 66600960->66605055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.952838:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66600960->66605055] (req 66600960->66605055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.952860:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66d2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66600960->66605055] (req 66600960->66605055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.952868:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66d2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66600960->66605055] (req 66600960->66605055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760789.952968:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554848640:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8141us (9053us total) trans 4294983569 rc 0/0 00000100:00100000:3.0:1789760789.952979:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16271 00000100:00100000:3.0:1789760789.952989:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733172246370 time 733190430049 state 0x2 ref 0 00000100:00100000:1.0:1789760789.961776:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760789.961926:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554848768 00000100:00100000:3.0:1789760789.961936:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554848768 00000100:00100000:3.0:1789760789.961989:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733172246370 time 733190430049 state 0x0 ref 1 00000100:00100000:3.0:1789760789.962010:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733190690465 time 733190430049 state 0x1 ref 1 00000100:00100000:3.0:1789760789.962046:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733190690465 time 733207664823 state 0x0 ref 1 00000100:00100000:3.0:1789760789.962058:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb020b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554848768:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760789.962105:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66605056, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fc40 00010000:00010000:3.0:1789760789.962114:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c66d9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.962129:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760789.962134:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760789.962141:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c66d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66605056->66609151] (req 66605056->66609151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760789.962154:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c66d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66605056->66609151] (req 66605056->66609151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760789.970311:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983570 is committed 00002000:00010000:0.0:1789760789.970851:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66605056 -> 66609152 00002000:00010000:0.0:1789760789.970857:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130112 -> 130120 00010000:00010000:0.0:1789760789.970871:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c66d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66605056->66609151] (req 66605056->66609151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.970883:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c66d9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66605056->66609151] (req 66605056->66609151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.970891:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c66d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66605056->66609151] (req 66605056->66609151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.970898:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760789.970904:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c66d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66605056->66609151] (req 66605056->66609151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.970912:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c66d9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66605056->66609151] (req 66605056->66609151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.970930:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c66d9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66605056->66609151] (req 66605056->66609151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760789.970944:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c66d9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66605056->66609151] (req 66605056->66609151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760789.971270:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb020b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554848768:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9208us (9502us total) trans 4294983570 rc 0/0 00000100:00100000:0.0:1789760789.971281:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16272 00000100:00100000:0.0:1789760789.971291:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733190690465 time 733207664823 state 0x2 ref 0 00000100:00100000:0.0:1789760789.976842:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760789.976965:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554848896 00000100:00100000:1.0:1789760789.976976:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554848896 00000100:00100000:1.0:1789760789.977045:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733190690465 time 733207664823 state 0x0 ref 1 00000100:00100000:1.0:1789760789.977061:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733207925239 time 733207664823 state 0x1 ref 1 00000100:00100000:1.0:1789760789.977089:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733207925239 time 733222707396 state 0x0 ref 1 00000100:00100000:1.0:1789760789.977101:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554848896:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760789.977317:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66609152, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fc48 00010000:00010000:1.0:1789760789.977326:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c66e0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.977342:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760789.977347:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760789.977354:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c66e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66609152->66613247] (req 66609152->66613247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.977369:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c66e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66609152->66613247] (req 66609152->66613247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760789.988084:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983571 is committed 00002000:00010000:1.0:1789760789.988345:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66609152 -> 66613248 00002000:00010000:1.0:1789760789.988351:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130120 -> 130128 00010000:00010000:1.0:1789760789.988363:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c66e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66609152->66613247] (req 66609152->66613247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.988464:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c66e0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66609152->66613247] (req 66609152->66613247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.988477:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c66e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66609152->66613247] (req 66609152->66613247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.988485:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760789.988491:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c66e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66609152->66613247] (req 66609152->66613247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.988507:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c66e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66609152->66613247] (req 66609152->66613247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.988528:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c66e0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66609152->66613247] (req 66609152->66613247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760789.988537:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c66e0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66609152->66613247] (req 66609152->66613247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760789.988641:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554848896:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11537us (11805us total) trans 4294983571 rc 0/0 00000100:00100000:1.0:1789760789.988651:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16273 00000100:00100000:1.0:1789760789.988660:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733207925239 time 733222707396 state 0x2 ref 0 00000100:00100000:1.0:1789760789.993590:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760790.000153:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554849024 00000100:00100000:2.0:1789760790.000170:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554849024 00000100:00100000:2.0:1789760790.000245:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733207925239 time 733222707396 state 0x0 ref 1 00000100:00100000:2.0:1789760790.000261:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733222967812 time 733222707396 state 0x1 ref 1 00000100:00100000:2.0:1789760790.000305:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733222967812 time 733245922909 state 0x0 ref 1 00000100:00100000:2.0:1789760790.000320:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb020b0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554849024:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760790.000365:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66613248, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fc50 00010000:00010000:2.0:1789760790.000374:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c66e7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.000422:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760790.000427:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.000433:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c66e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66613248->66617343] (req 66613248->66617343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.000449:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c66e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66613248->66617343] (req 66613248->66617343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.008918:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983572 is committed 00002000:00010000:2.0:1789760790.009105:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66613248 -> 66617344 00002000:00010000:2.0:1789760790.009111:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130128 -> 130136 00010000:00010000:2.0:1789760790.009123:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c66e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66613248->66617343] (req 66613248->66617343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.009136:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c66e7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66613248->66617343] (req 66613248->66617343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.009146:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c66e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66613248->66617343] (req 66613248->66617343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.009153:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760790.009158:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c66e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66613248->66617343] (req 66613248->66617343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.009166:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c66e7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66613248->66617343] (req 66613248->66617343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.009197:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c66e7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66613248->66617343] (req 66613248->66617343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.009204:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c66e7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66613248->66617343] (req 66613248->66617343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.009298:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb020b0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554849024:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8977us (15713us total) trans 4294983572 rc 0/0 00000100:00100000:2.0:1789760790.009309:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16274 00000100:00100000:2.0:1789760790.009319:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733222967812 time 733245922909 state 0x2 ref 0 00000100:00100000:2.0:1789760790.017548:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760790.017699:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554849152 00000100:00100000:3.0:1789760790.017709:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554849152 00000100:00100000:3.0:1789760790.017764:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733222967812 time 733245922909 state 0x0 ref 1 00000100:00100000:3.0:1789760790.017780:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733246183325 time 733245922909 state 0x1 ref 1 00000100:00100000:3.0:1789760790.017814:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733246183325 time 733263431939 state 0x0 ref 1 00000100:00100000:3.0:1789760790.017826:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09858380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554849152:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760790.017875:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66617344, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fc58 00010000:00010000:3.0:1789760790.017885:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66ee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.017903:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760790.017908:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.017915:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66617344->66621439] (req 66617344->66621439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.017930:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66617344->66621439] (req 66617344->66621439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.025648:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983573 is committed 00002000:00010000:3.0:1789760790.026502:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66617344 -> 66621440 00002000:00010000:3.0:1789760790.026510:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130136 -> 130144 00010000:00010000:3.0:1789760790.026524:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66617344->66621439] (req 66617344->66621439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.026537:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66617344->66621439] (req 66617344->66621439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.026552:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66617344->66621439] (req 66617344->66621439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.026561:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760790.026566:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66617344->66621439] (req 66617344->66621439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.026576:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66617344->66621439] (req 66617344->66621439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.026596:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66ee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66617344->66621439] (req 66617344->66621439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.026604:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c66ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66617344->66621439] (req 66617344->66621439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.026761:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09858380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554849152:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8932us (9217us total) trans 4294983573 rc 0/0 00000100:00100000:3.0:1789760790.026772:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16275 00000100:00100000:3.0:1789760790.026781:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733246183325 time 733263431939 state 0x2 ref 0 00000100:00100000:0.0:1789760790.038219:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760790.038466:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554849280 00000100:00100000:3.0:1789760790.038478:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554849280 00000100:00100000:2.0:1789760790.038597:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733246183325 time 733263431939 state 0x0 ref 1 00000100:00100000:2.0:1789760790.038609:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733263692355 time 733263431939 state 0x1 ref 1 00000100:00100000:2.0:1789760790.038648:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733263692355 time 733284263070 state 0x0 ref 1 00000100:00100000:2.0:1789760790.038661:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554849280:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760790.038709:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66621440, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fc60 00010000:00010000:2.0:1789760790.038719:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c66f5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.038735:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760790.038740:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.039548:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c66f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66621440->66625535] (req 66621440->66625535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.039570:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c66f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66621440->66625535] (req 66621440->66625535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.047204:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983574 is committed 00002000:00010000:1.0:1789760790.047350:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66621440 -> 66625536 00002000:00010000:1.0:1789760790.047355:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130144 -> 130152 00010000:00010000:1.0:1789760790.047366:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c66f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66621440->66625535] (req 66621440->66625535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.047474:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c66f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66621440->66625535] (req 66621440->66625535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.047485:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c66f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66621440->66625535] (req 66621440->66625535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.047494:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.047499:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c66f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66621440->66625535] (req 66621440->66625535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.047508:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c66f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66621440->66625535] (req 66621440->66625535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.047528:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c66f5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66621440->66625535] (req 66621440->66625535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.047535:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c66f5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66621440->66625535] (req 66621440->66625535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.047679:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554849280:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9010us (9462us total) trans 4294983574 rc 0/0 00000100:00100000:1.0:1789760790.047690:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16276 00000100:00100000:1.0:1789760790.047700:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733263692355 time 733284263070 state 0x2 ref 0 00000100:00100000:1.0:1789760790.052474:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760790.052629:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554849408 00000100:00100000:2.0:1789760790.052639:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554849408 00000100:00100000:2.0:1789760790.052692:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733263692355 time 733284263070 state 0x0 ref 1 00000100:00100000:2.0:1789760790.052708:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733284523486 time 733284263070 state 0x1 ref 1 00000100:00100000:2.0:1789760790.052738:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733284523486 time 733298356811 state 0x0 ref 1 00000100:00100000:2.0:1789760790.052761:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554849408:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760790.052808:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66625536, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fc68 00010000:00010000:2.0:1789760790.052816:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c66fc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.052833:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760790.052837:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.052844:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c66fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66625536->66629631] (req 66625536->66629631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.052857:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c66fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66625536->66629631] (req 66625536->66629631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.062528:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983575 is committed 00002000:00010000:1.0:1789760790.062697:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66625536 -> 66629632 00002000:00010000:1.0:1789760790.062702:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130152 -> 130160 00010000:00010000:1.0:1789760790.062715:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c66fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66625536->66629631] (req 66625536->66629631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.062729:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c66fc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66625536->66629631] (req 66625536->66629631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.062739:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c66fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66625536->66629631] (req 66625536->66629631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.062795:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.062801:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c66fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66625536->66629631] (req 66625536->66629631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.062812:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c66fc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66625536->66629631] (req 66625536->66629631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.062833:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c66fc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66625536->66629631] (req 66625536->66629631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.062841:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c66fc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66625536->66629631] (req 66625536->66629631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.063049:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554849408:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10280us (10574us total) trans 4294983575 rc 0/0 00000100:00100000:1.0:1789760790.063070:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16277 00000100:00100000:1.0:1789760790.063080:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733284523486 time 733298356811 state 0x2 ref 0 00000100:00100000:2.0:1789760790.066216:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760790.066303:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554849536 00000100:00100000:1.0:1789760790.066313:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554849536 00000100:00100000:1.0:1789760790.066366:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733284523486 time 733298356811 state 0x0 ref 1 00000100:00100000:1.0:1789760790.066417:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733298617227 time 733298356811 state 0x1 ref 1 00000100:00100000:1.0:1789760790.066441:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733298617227 time 733312059831 state 0x0 ref 1 00000100:00100000:1.0:1789760790.066453:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0985a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554849536:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760790.066499:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66629632, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fc70 00010000:00010000:1.0:1789760790.066509:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6703 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.066524:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760790.066529:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.066536:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6703 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66629632->66633727] (req 66629632->66633727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.066550:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6703 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66629632->66633727] (req 66629632->66633727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.071860:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983576 is committed 00002000:00010000:1.0:1789760790.072020:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66629632 -> 66633728 00002000:00010000:1.0:1789760790.072026:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130160 -> 130168 00010000:00010000:1.0:1789760790.072040:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6703 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66629632->66633727] (req 66629632->66633727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.072116:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6703 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66629632->66633727] (req 66629632->66633727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.072126:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6703 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66629632->66633727] (req 66629632->66633727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.072135:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.072140:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6703 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66629632->66633727] (req 66629632->66633727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.072146:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6703 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66629632->66633727] (req 66629632->66633727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.072160:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6703 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66629632->66633727] (req 66629632->66633727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.072165:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6703 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66629632->66633727] (req 66629632->66633727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.072262:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0985a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554849536:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5806us (6050us total) trans 4294983576 rc 0/0 00000100:00100000:1.0:1789760790.072273:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16278 00000100:00100000:1.0:1789760790.072283:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733298617227 time 733312059831 state 0x2 ref 0 00000100:00100000:0.0:1789760790.076271:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760790.076428:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554849664 00000100:00100000:3.0:1789760790.076439:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554849664 00000100:00100000:3.0:1789760790.076492:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733298617227 time 733312059831 state 0x0 ref 1 00000100:00100000:3.0:1789760790.076506:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733312320247 time 733312059831 state 0x1 ref 1 00000100:00100000:3.0:1789760790.076537:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733312320247 time 733322156001 state 0x0 ref 1 00000100:00100000:3.0:1789760790.076549:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554849664:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760790.076592:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66633728, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fc78 00010000:00010000:3.0:1789760790.076602:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c670a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.076619:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760790.076633:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.076640:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c670a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66633728->66637823] (req 66633728->66637823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.076654:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c670a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66633728->66637823] (req 66633728->66637823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.082974:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983577 is committed 00002000:00010000:0.0:1789760790.083102:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66633728 -> 66637824 00002000:00010000:0.0:1789760790.083107:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130168 -> 130176 00010000:00010000:0.0:1789760790.083123:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c670a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66633728->66637823] (req 66633728->66637823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.083135:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c670a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66633728->66637823] (req 66633728->66637823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.083143:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c670a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66633728->66637823] (req 66633728->66637823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.083151:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760790.083156:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c670a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66633728->66637823] (req 66633728->66637823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.083165:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c670a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66633728->66637823] (req 66633728->66637823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.083183:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c670a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66633728->66637823] (req 66633728->66637823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.083191:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c670a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66633728->66637823] (req 66633728->66637823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.083327:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554849664:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6774us (7065us total) trans 4294983577 rc 0/0 00000100:00100000:0.0:1789760790.083338:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16279 00000100:00100000:0.0:1789760790.083347:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733312320247 time 733322156001 state 0x2 ref 0 00000100:00100000:3.0:1789760790.087094:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760790.087193:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554849792 00000100:00100000:0.0:1789760790.087204:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554849792 00000100:00100000:0.0:1789760790.087258:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733312320247 time 733322156001 state 0x0 ref 1 00000100:00100000:0.0:1789760790.087274:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733322416417 time 733322156001 state 0x1 ref 1 00000100:00100000:0.0:1789760790.087309:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733322416417 time 733332927196 state 0x0 ref 1 00000100:00100000:0.0:1789760790.087320:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554849792:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760790.087366:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66637824, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fc80 00010000:00010000:0.0:1789760790.087415:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6711 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.087432:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760790.087436:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.087443:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6711 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66637824->66641919] (req 66637824->66641919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.087456:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6711 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66637824->66641919] (req 66637824->66641919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.093167:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983578 is committed 00002000:00010000:0.0:1789760790.093279:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66637824 -> 66641920 00002000:00010000:0.0:1789760790.093284:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130176 -> 130184 00010000:00010000:0.0:1789760790.093297:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6711 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66637824->66641919] (req 66637824->66641919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.093310:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6711 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66637824->66641919] (req 66637824->66641919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.093319:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6711 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66637824->66641919] (req 66637824->66641919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.093327:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760790.093331:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6711 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66637824->66641919] (req 66637824->66641919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.093340:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6711 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66637824->66641919] (req 66637824->66641919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.093359:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6711 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66637824->66641919] (req 66637824->66641919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.093366:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6711 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66637824->66641919] (req 66637824->66641919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.093475:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554849792:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6152us (6385us total) trans 4294983578 rc 0/0 00000100:00100000:0.0:1789760790.093486:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16280 00000100:00100000:0.0:1789760790.093495:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733322416417 time 733332927196 state 0x2 ref 0 00010000:00010000:3.0:1789760790.101545:0:8453:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789760790.109263:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760790.110916:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554849920 00000100:00100000:1.0:1789760790.110933:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554849920 00000100:00100000:1.0:1789760790.111001:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733322416417 time 733332927196 state 0x0 ref 1 00000100:00100000:1.0:1789760790.111018:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733333187612 time 733332927196 state 0x1 ref 1 00000100:00100000:1.0:1789760790.111061:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733333187612 time 733356678685 state 0x0 ref 1 00000100:00100000:1.0:1789760790.111080:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09858700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554849920:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760790.111133:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66641920, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fc88 00010000:00010000:1.0:1789760790.111142:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6718 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.111159:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760790.111163:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.111169:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6718 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66641920->66646015] (req 66641920->66646015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.111184:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6718 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66641920->66646015] (req 66641920->66646015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.117142:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983579 is committed 00002000:00010000:1.0:1789760790.120115:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66641920 -> 66646016 00002000:00010000:1.0:1789760790.120125:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130184 -> 130192 00010000:00010000:1.0:1789760790.120138:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6718 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66641920->66646015] (req 66641920->66646015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.120151:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6718 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66641920->66646015] (req 66641920->66646015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.120159:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6718 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66641920->66646015] (req 66641920->66646015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.120166:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.120172:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6718 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66641920->66646015] (req 66641920->66646015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.120181:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6718 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66641920->66646015] (req 66641920->66646015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.120202:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6718 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66641920->66646015] (req 66641920->66646015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.120209:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6718 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66641920->66646015] (req 66641920->66646015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.120319:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09858700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554849920:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9237us (11060us total) trans 4294983579 rc 0/0 00000100:00100000:1.0:1789760790.120330:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16281 00000100:00100000:1.0:1789760790.120338:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733333187612 time 733356678685 state 0x2 ref 0 00000100:00100000:2.0:1789760790.127757:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760790.127876:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554850048 00000100:00100000:1.0:1789760790.127886:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554850048 00000100:00100000:1.0:1789760790.127939:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733333187612 time 733356678685 state 0x0 ref 1 00000100:00100000:1.0:1789760790.127955:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733356939101 time 733356678685 state 0x1 ref 1 00000100:00100000:1.0:1789760790.127988:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733356939101 time 733373606707 state 0x0 ref 1 00000100:00100000:1.0:1789760790.128000:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0985bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554850048:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760790.128044:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66646016, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fc90 00010000:00010000:1.0:1789760790.128053:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c671f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.128070:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760790.128074:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.128080:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c671f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66646016->66650111] (req 66646016->66650111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.128094:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c671f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66646016->66650111] (req 66646016->66650111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.133256:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983580 is committed 00002000:00010000:1.0:1789760790.133717:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66646016 -> 66650112 00002000:00010000:1.0:1789760790.133724:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130192 -> 130200 00010000:00010000:1.0:1789760790.133738:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c671f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66646016->66650111] (req 66646016->66650111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.133753:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c671f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66646016->66650111] (req 66646016->66650111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.133763:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c671f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66646016->66650111] (req 66646016->66650111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.133772:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.133777:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c671f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66646016->66650111] (req 66646016->66650111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.133786:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c671f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66646016->66650111] (req 66646016->66650111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.133807:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c671f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66646016->66650111] (req 66646016->66650111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.134180:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c671f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66646016->66650111] (req 66646016->66650111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.134280:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0985bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554850048:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6277us (6529us total) trans 4294983580 rc 0/0 00000100:00100000:1.0:1789760790.134292:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16282 00000100:00100000:1.0:1789760790.134302:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733356939101 time 733373606707 state 0x2 ref 0 00000100:00100000:3.0:1789760790.137368:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760790.138974:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554850176 00000100:00100000:0.0:1789760790.138988:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554850176 00000100:00100000:0.0:1789760790.139048:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733356939101 time 733373606707 state 0x0 ref 1 00000100:00100000:0.0:1789760790.139064:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733373867123 time 733373606707 state 0x1 ref 1 00000100:00100000:0.0:1789760790.139099:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733373867123 time 733384717317 state 0x0 ref 1 00000100:00100000:0.0:1789760790.139112:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554850176:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760790.139157:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66650112, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fc98 00010000:00010000:0.0:1789760790.139167:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6726 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.139183:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760790.139188:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.139195:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6726 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66650112->66654207] (req 66650112->66654207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.139209:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6726 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66650112->66654207] (req 66650112->66654207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.149580:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983581 is committed 00002000:00010000:0.0:1789760790.149785:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66650112 -> 66654208 00002000:00010000:0.0:1789760790.149791:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130200 -> 130208 00010000:00010000:0.0:1789760790.149803:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6726 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66650112->66654207] (req 66650112->66654207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.149828:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6726 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66650112->66654207] (req 66650112->66654207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.149838:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6726 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66650112->66654207] (req 66650112->66654207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.149917:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760790.149921:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6726 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66650112->66654207] (req 66650112->66654207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.149931:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6726 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66650112->66654207] (req 66650112->66654207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.149951:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6726 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66650112->66654207] (req 66650112->66654207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.149964:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6726 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66650112->66654207] (req 66650112->66654207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.150119:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554850176:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11005us (12755us total) trans 4294983581 rc 0/0 00000100:00100000:0.0:1789760790.150129:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16283 00000100:00100000:0.0:1789760790.150138:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733373867123 time 733384717317 state 0x2 ref 0 00000100:00100000:2.0:1789760790.153308:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760790.153415:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554850304 00000100:00100000:0.0:1789760790.153425:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554850304 00000100:00100000:0.0:1789760790.153479:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733373867123 time 733384717317 state 0x0 ref 1 00000100:00100000:0.0:1789760790.153496:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733384977733 time 733384717317 state 0x1 ref 1 00000100:00100000:0.0:1789760790.153519:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733384977733 time 733399137238 state 0x0 ref 1 00000100:00100000:0.0:1789760790.153532:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0985b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554850304:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760790.153579:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66654208, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fca0 00010000:00010000:0.0:1789760790.153588:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c672d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.153604:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760790.153608:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.153615:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c672d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66654208->66658303] (req 66654208->66658303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.153645:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c672d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66654208->66658303] (req 66654208->66658303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.167652:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983582 is committed 00002000:00010000:0.0:1789760790.167970:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66654208 -> 66658304 00002000:00010000:0.0:1789760790.167975:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130208 -> 130216 00010000:00010000:0.0:1789760790.167988:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c672d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66654208->66658303] (req 66654208->66658303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.168012:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c672d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66654208->66658303] (req 66654208->66658303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.168023:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c672d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66654208->66658303] (req 66654208->66658303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.168031:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760790.168037:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c672d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66654208->66658303] (req 66654208->66658303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.168047:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c672d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66654208->66658303] (req 66654208->66658303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.168067:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c672d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66654208->66658303] (req 66654208->66658303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.168075:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c672d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66654208->66658303] (req 66654208->66658303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.168173:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0985b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554850304:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14637us (14870us total) trans 4294983582 rc 0/0 00000100:00100000:0.0:1789760790.168184:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16284 00000100:00100000:0.0:1789760790.168195:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733384977733 time 733399137238 state 0x2 ref 0 00000100:00100000:2.0:1789760790.171958:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760790.172055:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554850432 00000100:00100000:1.0:1789760790.172066:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554850432 00000100:00100000:1.0:1789760790.172125:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733384977733 time 733399137238 state 0x0 ref 1 00000100:00100000:1.0:1789760790.172141:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733399397654 time 733399137238 state 0x1 ref 1 00000100:00100000:1.0:1789760790.172167:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733399397654 time 733417785635 state 0x0 ref 1 00000100:00100000:1.0:1789760790.172179:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09859880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554850432:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760790.172229:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66658304, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fca8 00010000:00010000:1.0:1789760790.172238:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6734 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.172253:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760790.172258:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.172264:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6734 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66658304->66662399] (req 66658304->66662399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.172277:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6734 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66658304->66662399] (req 66658304->66662399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.182195:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983583 is committed 00002000:00010000:3.0:1789760790.182325:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66658304 -> 66662400 00002000:00010000:3.0:1789760790.182330:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130216 -> 130224 00010000:00010000:3.0:1789760790.182343:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6734 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66658304->66662399] (req 66658304->66662399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.182357:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6734 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66658304->66662399] (req 66658304->66662399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.182367:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6734 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66658304->66662399] (req 66658304->66662399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.182413:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760790.182419:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6734 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66658304->66662399] (req 66658304->66662399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.182430:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6734 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66658304->66662399] (req 66658304->66662399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.182450:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6734 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66658304->66662399] (req 66658304->66662399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.182459:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6734 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66658304->66662399] (req 66658304->66662399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.182586:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09859880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554850432:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10403us (10633us total) trans 4294983583 rc 0/0 00000100:00100000:3.0:1789760790.182597:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16285 00000100:00100000:3.0:1789760790.182608:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733399397654 time 733417785635 state 0x2 ref 0 00000100:00100000:0.0:1789760790.192894:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760790.193412:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554850560 00000100:00100000:2.0:1789760790.193423:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554850560 00000100:00100000:2.0:1789760790.193482:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733399397654 time 733417785635 state 0x0 ref 1 00000100:00100000:2.0:1789760790.193500:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733418046051 time 733417785635 state 0x1 ref 1 00000100:00100000:2.0:1789760790.193534:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733418046051 time 733439152239 state 0x0 ref 1 00000100:00100000:2.0:1789760790.193547:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554850560:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760790.193593:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66662400, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fcb0 00010000:00010000:2.0:1789760790.193601:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c673b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.193617:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760790.193631:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.193638:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c673b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66662400->66666495] (req 66662400->66666495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.193659:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c673b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66662400->66666495] (req 66662400->66666495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.206704:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983584 is committed 00002000:00010000:2.0:1789760790.206834:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66662400 -> 66666496 00002000:00010000:2.0:1789760790.206839:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130224 -> 130232 00010000:00010000:2.0:1789760790.206852:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c673b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66662400->66666495] (req 66662400->66666495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.206865:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c673b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66662400->66666495] (req 66662400->66666495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.206874:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c673b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66662400->66666495] (req 66662400->66666495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.206881:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760790.206886:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c673b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66662400->66666495] (req 66662400->66666495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.206895:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c673b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66662400->66666495] (req 66662400->66666495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.206915:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c673b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66662400->66666495] (req 66662400->66666495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.206922:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c673b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66662400->66666495] (req 66662400->66666495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.207113:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554850560:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13563us (14223us total) trans 4294983584 rc 0/0 00000100:00100000:2.0:1789760790.207125:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16286 00000100:00100000:2.0:1789760790.207134:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733418046051 time 733439152239 state 0x2 ref 0 00000100:00100000:2.0:1789760790.214486:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760790.215513:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554850688 00000100:00100000:0.0:1789760790.215524:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554850688 00000100:00100000:0.0:1789760790.215595:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733418046051 time 733439152239 state 0x0 ref 1 00000100:00100000:0.0:1789760790.215610:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733439412655 time 733439152239 state 0x1 ref 1 00000100:00100000:0.0:1789760790.215647:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733439412655 time 733461265513 state 0x0 ref 1 00000100:00100000:0.0:1789760790.215661:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0985aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554850688:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760790.215707:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66666496, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fcb8 00010000:00010000:0.0:1789760790.215716:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6742 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.215731:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760790.215736:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.215742:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6742 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66666496->66670591] (req 66666496->66670591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.215762:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6742 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66666496->66670591] (req 66666496->66670591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.225994:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983585 is committed 00002000:00010000:0.0:1789760790.226124:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66666496 -> 66670592 00002000:00010000:0.0:1789760790.226129:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130232 -> 130240 00010000:00010000:0.0:1789760790.226142:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6742 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66666496->66670591] (req 66666496->66670591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.226165:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6742 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66666496->66670591] (req 66666496->66670591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.226173:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6742 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66666496->66670591] (req 66666496->66670591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.226180:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760790.226185:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6742 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66666496->66670591] (req 66666496->66670591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.226195:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6742 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66666496->66670591] (req 66666496->66670591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.226539:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6742 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66666496->66670591] (req 66666496->66670591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.226549:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6742 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66666496->66670591] (req 66666496->66670591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.226661:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0985aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554850688:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10997us (12178us total) trans 4294983585 rc 0/0 00000100:00100000:0.0:1789760790.226672:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16287 00000100:00100000:0.0:1789760790.226682:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733439412655 time 733461265513 state 0x2 ref 0 00000100:00100000:0.0:1789760790.235998:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760790.236101:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554850816 00000100:00100000:1.0:1789760790.236112:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554850816 00000100:00100000:1.0:1789760790.236162:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733439412655 time 733461265513 state 0x0 ref 1 00000100:00100000:1.0:1789760790.236177:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733461525929 time 733461265513 state 0x1 ref 1 00000100:00100000:1.0:1789760790.236208:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733461525929 time 733481827050 state 0x0 ref 1 00000100:00100000:1.0:1789760790.236220:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554850816:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760790.236265:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66670592, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fcc0 00010000:00010000:1.0:1789760790.236274:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6749 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.236288:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760790.236303:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.236310:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6749 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66670592->66674687] (req 66670592->66674687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.236323:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6749 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66670592->66674687] (req 66670592->66674687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.245537:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983586 is committed 00002000:00010000:1.0:1789760790.245684:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66670592 -> 66674688 00002000:00010000:1.0:1789760790.245690:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130240 -> 130248 00010000:00010000:1.0:1789760790.245704:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6749 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66670592->66674687] (req 66670592->66674687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.245718:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6749 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66670592->66674687] (req 66670592->66674687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.245726:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6749 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66670592->66674687] (req 66670592->66674687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.245733:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.245737:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6749 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66670592->66674687] (req 66670592->66674687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.245746:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6749 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66670592->66674687] (req 66670592->66674687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.245765:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6749 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66670592->66674687] (req 66670592->66674687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.245773:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6749 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66670592->66674687] (req 66670592->66674687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.245875:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554850816:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9651us (9882us total) trans 4294983586 rc 0/0 00000100:00100000:1.0:1789760790.245888:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16288 00000100:00100000:1.0:1789760790.245899:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733461525929 time 733481827050 state 0x2 ref 0 00000100:00100000:3.0:1789760790.264876:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760790.265130:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554850944 00000100:00100000:1.0:1789760790.265140:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554850944 00000100:00100000:1.0:1789760790.265208:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733461525929 time 733481827050 state 0x0 ref 1 00000100:00100000:1.0:1789760790.265223:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733482087466 time 733481827050 state 0x1 ref 1 00000100:00100000:1.0:1789760790.265254:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733482087466 time 733510872486 state 0x0 ref 1 00000100:00100000:1.0:1789760790.265266:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554850944:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760790.265314:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66674688, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fcc8 00010000:00010000:1.0:1789760790.265324:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6750 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.265340:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760790.265344:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.265350:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6750 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66674688->66678783] (req 66674688->66678783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.265365:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6750 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66674688->66678783] (req 66674688->66678783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.273373:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983587 is committed 00002000:00010000:1.0:1789760790.273574:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66674688 -> 66678784 00002000:00010000:1.0:1789760790.273580:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130248 -> 130256 00010000:00010000:1.0:1789760790.273593:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6750 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66674688->66678783] (req 66674688->66678783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.273605:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6750 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66674688->66678783] (req 66674688->66678783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.273614:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6750 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66674688->66678783] (req 66674688->66678783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.273630:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.273635:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6750 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66674688->66678783] (req 66674688->66678783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.273643:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6750 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66674688->66678783] (req 66674688->66678783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.273663:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6750 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66674688->66678783] (req 66674688->66678783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.273671:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6750 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66674688->66678783] (req 66674688->66678783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.273773:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554850944:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8504us (8902us total) trans 4294983587 rc 0/0 00000100:00100000:1.0:1789760790.273785:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16289 00000100:00100000:1.0:1789760790.273795:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733482087466 time 733510872486 state 0x2 ref 0 00000100:00100000:2.0:1789760790.280677:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760790.280771:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554851072 00000100:00100000:1.0:1789760790.280782:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554851072 00000100:00100000:1.0:1789760790.280836:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733482087466 time 733510872486 state 0x0 ref 1 00000100:00100000:1.0:1789760790.280851:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733511132902 time 733510872486 state 0x1 ref 1 00000100:00100000:1.0:1789760790.280887:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733511132902 time 733526505121 state 0x0 ref 1 00000100:00100000:1.0:1789760790.280899:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09858a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554851072:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760790.280949:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66678784, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fcd0 00010000:00010000:1.0:1789760790.280959:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6757 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.280973:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760790.280977:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.280983:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6757 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66678784->66682879] (req 66678784->66682879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.280995:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6757 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66678784->66682879] (req 66678784->66682879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.287115:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983588 is committed 00002000:00010000:1.0:1789760790.287351:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66678784 -> 66682880 00002000:00010000:1.0:1789760790.287356:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130256 -> 130264 00010000:00010000:1.0:1789760790.287369:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6757 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66678784->66682879] (req 66678784->66682879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.287410:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6757 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66678784->66682879] (req 66678784->66682879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.287416:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6757 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66678784->66682879] (req 66678784->66682879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.287421:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.287424:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6757 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66678784->66682879] (req 66678784->66682879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.287429:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6757 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66678784->66682879] (req 66678784->66682879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.287444:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6757 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66678784->66682879] (req 66678784->66682879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.287448:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6757 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66678784->66682879] (req 66678784->66682879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.287558:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09858a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554851072:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6657us (6888us total) trans 4294983588 rc 0/0 00000100:00100000:1.0:1789760790.287566:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16290 00000100:00100000:1.0:1789760790.287573:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733511132902 time 733526505121 state 0x2 ref 0 00000100:00100000:0.0:1789760790.293321:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760790.293428:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554851200 00000100:00100000:2.0:1789760790.293439:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554851200 00000100:00100000:2.0:1789760790.293884:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733511132902 time 733526505121 state 0x0 ref 1 00000100:00100000:2.0:1789760790.293900:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733526765537 time 733526505121 state 0x1 ref 1 00000100:00100000:2.0:1789760790.293936:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733526765537 time 733539554053 state 0x0 ref 1 00000100:00100000:2.0:1789760790.293951:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554851200:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760790.294003:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66682880, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fcd8 00010000:00010000:2.0:1789760790.294027:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c675e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.294044:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760790.294048:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.294055:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c675e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66682880->66686975] (req 66682880->66686975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.294074:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c675e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66682880->66686975] (req 66682880->66686975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.299948:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983589 is committed 00002000:00010000:2.0:1789760790.300090:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66682880 -> 66686976 00002000:00010000:2.0:1789760790.300095:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130264 -> 130272 00010000:00010000:2.0:1789760790.300110:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c675e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66682880->66686975] (req 66682880->66686975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.300123:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c675e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66682880->66686975] (req 66682880->66686975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.300132:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c675e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66682880->66686975] (req 66682880->66686975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.300140:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760790.300145:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c675e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66682880->66686975] (req 66682880->66686975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.300154:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c675e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66682880->66686975] (req 66682880->66686975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.300173:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c675e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66682880->66686975] (req 66682880->66686975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.300181:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c675e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66682880->66686975] (req 66682880->66686975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.300282:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554851200:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6329us (6965us total) trans 4294983589 rc 0/0 00000100:00100000:2.0:1789760790.300292:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16291 00000100:00100000:2.0:1789760790.300301:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733526765537 time 733539554053 state 0x2 ref 0 00000100:00100000:3.0:1789760790.304083:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760790.304244:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554851328 00000100:00100000:0.0:1789760790.304253:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554851328 00000100:00100000:0.0:1789760790.304308:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733526765537 time 733539554053 state 0x0 ref 1 00000100:00100000:0.0:1789760790.304322:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733539814469 time 733539554053 state 0x1 ref 1 00000100:00100000:0.0:1789760790.304366:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733539814469 time 733549984449 state 0x0 ref 1 00000100:00100000:0.0:1789760790.304402:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554851328:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760790.304451:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66686976, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fce0 00010000:00010000:0.0:1789760790.304461:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6765 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.304476:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760790.304481:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.304488:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6765 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66686976->66691071] (req 66686976->66691071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.304508:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6765 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66686976->66691071] (req 66686976->66691071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.317274:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983590 is committed 00002000:00010000:0.0:1789760790.317421:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66686976 -> 66691072 00002000:00010000:0.0:1789760790.317427:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130272 -> 130280 00010000:00010000:0.0:1789760790.317440:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6765 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66686976->66691071] (req 66686976->66691071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.317453:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6765 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66686976->66691071] (req 66686976->66691071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.317464:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6765 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66686976->66691071] (req 66686976->66691071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.317472:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760790.317476:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6765 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66686976->66691071] (req 66686976->66691071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.317491:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6765 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66686976->66691071] (req 66686976->66691071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.317510:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6765 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66686976->66691071] (req 66686976->66691071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.317517:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6765 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66686976->66691071] (req 66686976->66691071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.317610:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554851328:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13229us (13533us total) trans 4294983590 rc 0/0 00000100:00100000:0.0:1789760790.317632:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16292 00000100:00100000:0.0:1789760790.317641:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733539814469 time 733549984449 state 0x2 ref 0 00000100:00100000:1.0:1789760790.330432:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760790.330697:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554851456 00000100:00100000:3.0:1789760790.330707:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554851456 00000100:00100000:3.0:1789760790.330760:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733539814469 time 733549984449 state 0x0 ref 1 00000100:00100000:3.0:1789760790.330777:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733550244865 time 733549984449 state 0x1 ref 1 00000100:00100000:3.0:1789760790.330866:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733550244865 time 733576483969 state 0x0 ref 1 00000100:00100000:3.0:1789760790.330883:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a387480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554851456:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760790.330941:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66691072, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fce8 00010000:00010000:3.0:1789760790.330950:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c676c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.330966:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760790.330970:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.330976:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c676c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66691072->66695167] (req 66691072->66695167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.330989:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c676c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66691072->66695167] (req 66691072->66695167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.339479:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983591 is committed 00002000:00010000:3.0:1789760790.339601:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66691072 -> 66695168 00002000:00010000:3.0:1789760790.339606:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130280 -> 130288 00010000:00010000:3.0:1789760790.339628:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c676c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66691072->66695167] (req 66691072->66695167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.339646:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c676c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66691072->66695167] (req 66691072->66695167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.339655:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c676c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66691072->66695167] (req 66691072->66695167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.339662:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760790.339667:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c676c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66691072->66695167] (req 66691072->66695167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.339678:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c676c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66691072->66695167] (req 66691072->66695167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.339699:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c676c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66691072->66695167] (req 66691072->66695167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.339706:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c676c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66691072->66695167] (req 66691072->66695167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.339858:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a387480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554851456:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8966us (9422us total) trans 4294983591 rc 0/0 00000100:00100000:3.0:1789760790.339869:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16293 00000100:00100000:3.0:1789760790.339878:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733550244865 time 733576483969 state 0x2 ref 0 00000100:00100000:2.0:1789760790.348225:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760790.348431:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554851584 00000100:00100000:1.0:1789760790.348444:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554851584 00000100:00100000:1.0:1789760790.348502:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733550244865 time 733576483969 state 0x0 ref 1 00000100:00100000:1.0:1789760790.348517:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733576744385 time 733576483969 state 0x1 ref 1 00000100:00100000:1.0:1789760790.348554:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733576744385 time 733594172085 state 0x0 ref 1 00000100:00100000:1.0:1789760790.348567:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09859c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554851584:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760790.348611:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66695168, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fcf0 00010000:00010000:1.0:1789760790.348766:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6773 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.348783:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760790.348788:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.348795:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6773 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66695168->66699263] (req 66695168->66699263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.348808:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6773 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66695168->66699263] (req 66695168->66699263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.359249:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983592 is committed 00002000:00010000:1.0:1789760790.359427:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66695168 -> 66699264 00002000:00010000:1.0:1789760790.359433:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130288 -> 130296 00010000:00010000:1.0:1789760790.359448:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6773 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66695168->66699263] (req 66695168->66699263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.359461:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6773 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66695168->66699263] (req 66695168->66699263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.359471:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6773 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66695168->66699263] (req 66695168->66699263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.359478:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655de00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.359484:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6773 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66695168->66699263] (req 66695168->66699263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.359493:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6773 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66695168->66699263] (req 66695168->66699263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.359512:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6773 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66695168->66699263] (req 66695168->66699263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.359520:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6773 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66695168->66699263] (req 66695168->66699263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.359635:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09859c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554851584:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11056us (11405us total) trans 4294983592 rc 0/0 00000100:00100000:1.0:1789760790.359645:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16294 00000100:00100000:1.0:1789760790.359654:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733576744385 time 733594172085 state 0x2 ref 0 00000100:00100000:1.0:1789760790.382100:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760790.382254:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554851712 00000100:00100000:2.0:1789760790.382264:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554851712 00000100:00100000:2.0:1789760790.382317:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733576744385 time 733594172085 state 0x0 ref 1 00000100:00100000:2.0:1789760790.382331:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733594432501 time 733594172085 state 0x1 ref 1 00000100:00100000:2.0:1789760790.382362:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733594432501 time 733627980999 state 0x0 ref 1 00000100:00100000:2.0:1789760790.382375:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a387b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554851712:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760790.382460:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66699264, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fcf8 00010000:00010000:2.0:1789760790.382469:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c677a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.382484:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760790.382488:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.382495:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c677a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66699264->66703359] (req 66699264->66703359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.382514:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c677a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66699264->66703359] (req 66699264->66703359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.390353:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983593 is committed 00002000:00010000:2.0:1789760790.396867:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66699264 -> 66703360 00002000:00010000:2.0:1789760790.396877:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130296 -> 130304 00010000:00010000:2.0:1789760790.396892:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c677a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66699264->66703359] (req 66699264->66703359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.396905:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c677a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66699264->66703359] (req 66699264->66703359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.396915:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c677a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66699264->66703359] (req 66699264->66703359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.396922:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760790.396928:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c677a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66699264->66703359] (req 66699264->66703359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.396936:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c677a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66699264->66703359] (req 66699264->66703359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.396956:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c677a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66699264->66703359] (req 66699264->66703359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.396964:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c677a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66699264->66703359] (req 66699264->66703359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.397078:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a387b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554851712:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14700us (14982us total) trans 4294983593 rc 0/0 00000100:00100000:2.0:1789760790.397090:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16295 00000100:00100000:2.0:1789760790.397100:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733594432501 time 733627980999 state 0x2 ref 0 00000100:00100000:2.0:1789760790.411673:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760790.411773:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554851840 00000100:00100000:0.0:1789760790.411782:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554851840 00000100:00100000:0.0:1789760790.411837:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733594432501 time 733627980999 state 0x0 ref 1 00000100:00100000:0.0:1789760790.411933:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733628241415 time 733627980999 state 0x1 ref 1 00000100:00100000:0.0:1789760790.411966:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733628241415 time 733657585451 state 0x0 ref 1 00000100:00100000:0.0:1789760790.411980:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09859180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554851840:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760790.412025:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66703360, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fd00 00010000:00010000:0.0:1789760790.412035:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6781 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.412050:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760790.412054:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.412061:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6781 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66703360->66707455] (req 66703360->66707455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.412081:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6781 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66703360->66707455] (req 66703360->66707455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.419128:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983594 is committed 00002000:00010000:1.0:1789760790.425611:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66703360 -> 66707456 00002000:00010000:1.0:1789760790.425631:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130304 -> 130312 00010000:00010000:1.0:1789760790.425644:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6781 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66703360->66707455] (req 66703360->66707455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.425658:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6781 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66703360->66707455] (req 66703360->66707455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.425667:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6781 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66703360->66707455] (req 66703360->66707455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.425674:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.425678:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6781 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66703360->66707455] (req 66703360->66707455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.425688:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6781 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66703360->66707455] (req 66703360->66707455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.425706:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6781 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66703360->66707455] (req 66703360->66707455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.425713:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6781 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66703360->66707455] (req 66703360->66707455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.425813:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09859180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554851840:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13831us (14143us total) trans 4294983594 rc 0/0 00000100:00100000:1.0:1789760790.425824:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16296 00000100:00100000:1.0:1789760790.425834:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733628241415 time 733657585451 state 0x2 ref 0 00000100:00100000:0.0:1789760790.433336:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760790.433967:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554851968 00000100:00100000:3.0:1789760790.433977:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554851968 00000100:00100000:3.0:1789760790.434171:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733628241415 time 733657585451 state 0x0 ref 1 00000100:00100000:3.0:1789760790.434187:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733657845867 time 733657585451 state 0x1 ref 1 00000100:00100000:3.0:1789760790.434225:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733657845867 time 733679842763 state 0x0 ref 1 00000100:00100000:3.0:1789760790.434239:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554851968:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760790.434287:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66707456, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fd08 00010000:00010000:3.0:1789760790.434296:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6788 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.434313:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760790.434317:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.434324:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6788 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66707456->66711551] (req 66707456->66711551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.434338:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6788 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66707456->66711551] (req 66707456->66711551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.445078:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983595 is committed 00002000:00010000:3.0:1789760790.445182:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66707456 -> 66711552 00002000:00010000:3.0:1789760790.445187:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130312 -> 130320 00010000:00010000:3.0:1789760790.445201:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6788 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66707456->66711551] (req 66707456->66711551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.445213:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6788 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66707456->66711551] (req 66707456->66711551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.445226:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6788 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66707456->66711551] (req 66707456->66711551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.445233:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760790.445239:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6788 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66707456->66711551] (req 66707456->66711551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.445248:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6788 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66707456->66711551] (req 66707456->66711551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.445267:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6788 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66707456->66711551] (req 66707456->66711551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.445275:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6788 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66707456->66711551] (req 66707456->66711551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.445366:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554851968:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11124us (12034us total) trans 4294983595 rc 0/0 00000100:00100000:3.0:1789760790.445543:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16297 00000100:00100000:3.0:1789760790.445553:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733657845867 time 733679842763 state 0x2 ref 0 00000100:00100000:3.0:1789760790.452280:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760790.452473:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554852096 00000100:00100000:1.0:1789760790.452482:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554852096 00000100:00100000:1.0:1789760790.452536:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733657845867 time 733679842763 state 0x0 ref 1 00000100:00100000:1.0:1789760790.452552:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733680103179 time 733679842763 state 0x1 ref 1 00000100:00100000:1.0:1789760790.452588:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733680103179 time 733698206005 state 0x0 ref 1 00000100:00100000:1.0:1789760790.452601:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554852096:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760790.452662:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66711552, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fd10 00010000:00010000:1.0:1789760790.452672:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c678f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.452688:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760790.452693:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.452700:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c678f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66711552->66715647] (req 66711552->66715647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.452714:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c678f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66711552->66715647] (req 66711552->66715647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.461135:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983596 is committed 00002000:00010000:1.0:1789760790.461278:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66711552 -> 66715648 00002000:00010000:1.0:1789760790.461285:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130320 -> 130328 00010000:00010000:1.0:1789760790.461297:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c678f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66711552->66715647] (req 66711552->66715647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.461309:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c678f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66711552->66715647] (req 66711552->66715647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.461323:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c678f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66711552->66715647] (req 66711552->66715647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.461330:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655de00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.461334:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c678f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66711552->66715647] (req 66711552->66715647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.461342:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c678f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66711552->66715647] (req 66711552->66715647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.461359:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c678f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66711552->66715647] (req 66711552->66715647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.461367:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c678f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66711552->66715647] (req 66711552->66715647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.461478:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554852096:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8874us (9204us total) trans 4294983596 rc 0/0 00000100:00100000:1.0:1789760790.461488:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16298 00000100:00100000:1.0:1789760790.461497:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733680103179 time 733698206005 state 0x2 ref 0 00000100:00100000:2.0:1789760790.467781:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760790.467932:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554852224 00000100:00100000:0.0:1789760790.467942:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554852224 00000100:00100000:0.0:1789760790.468014:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733680103179 time 733698206005 state 0x0 ref 1 00000100:00100000:0.0:1789760790.468032:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733698466421 time 733698206005 state 0x1 ref 1 00000100:00100000:0.0:1789760790.468070:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733698466421 time 733713688406 state 0x0 ref 1 00000100:00100000:0.0:1789760790.468084:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0985b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554852224:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760790.468435:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66715648, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fd18 00010000:00010000:0.0:1789760790.468447:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6796 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.468463:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789760790.468468:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.468475:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6796 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66715648->66719743] (req 66715648->66719743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.468488:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6796 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66715648->66719743] (req 66715648->66719743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.476703:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983597 is committed 00002000:00010000:1.0:1789760790.478311:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66715648 -> 66719744 00002000:00010000:1.0:1789760790.478319:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130328 -> 130336 00010000:00010000:1.0:1789760790.478333:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6796 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66715648->66719743] (req 66715648->66719743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.478352:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6796 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66715648->66719743] (req 66715648->66719743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.478361:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6796 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66715648->66719743] (req 66715648->66719743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.478368:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.478372:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6796 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66715648->66719743] (req 66715648->66719743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.478419:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6796 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66715648->66719743] (req 66715648->66719743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.478439:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6796 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66715648->66719743] (req 66715648->66719743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.478447:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6796 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66715648->66719743] (req 66715648->66719743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.478551:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0985b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554852224:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10463us (10894us total) trans 4294983597 rc 0/0 00000100:00100000:1.0:1789760790.478671:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16299 00000100:00100000:1.0:1789760790.478681:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733698466421 time 733713688406 state 0x2 ref 0 00000100:00100000:0.0:1789760790.486462:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760790.487583:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554852352 00000100:00100000:2.0:1789760790.487595:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554852352 00000100:00100000:2.0:1789760790.487668:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733698466421 time 733713688406 state 0x0 ref 1 00000100:00100000:2.0:1789760790.487695:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733713948822 time 733713688406 state 0x1 ref 1 00000100:00100000:2.0:1789760790.487734:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733713948822 time 733733352049 state 0x0 ref 1 00000100:00100000:2.0:1789760790.487746:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554852352:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760790.487790:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66719744, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fd20 00010000:00010000:2.0:1789760790.487799:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c679d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.487815:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760790.487820:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.487826:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c679d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66719744->66723839] (req 66719744->66723839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.487840:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c679d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66719744->66723839] (req 66719744->66723839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.499735:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983598 is committed 00002000:00010000:2.0:1789760790.499840:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66719744 -> 66723840 00002000:00010000:2.0:1789760790.499845:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130336 -> 130344 00010000:00010000:2.0:1789760790.499859:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c679d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66719744->66723839] (req 66719744->66723839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.499871:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c679d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66719744->66723839] (req 66719744->66723839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.499880:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c679d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66719744->66723839] (req 66719744->66723839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.499888:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760790.499893:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c679d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66719744->66723839] (req 66719744->66723839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.499901:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c679d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66719744->66723839] (req 66719744->66723839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.499921:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c679d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66719744->66723839] (req 66719744->66723839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.499929:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c679d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66719744->66723839] (req 66719744->66723839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.500039:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554852352:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12290us (13580us total) trans 4294983598 rc 0/0 00000100:00100000:2.0:1789760790.500050:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16300 00000100:00100000:2.0:1789760790.500060:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733713948822 time 733733352049 state 0x2 ref 0 00000100:00100000:3.0:1789760790.514032:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760790.514190:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554852480 00000100:00100000:1.0:1789760790.514200:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554852480 00000100:00100000:1.0:1789760790.514251:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733713948822 time 733733352049 state 0x0 ref 1 00000100:00100000:1.0:1789760790.514266:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733733612465 time 733733352049 state 0x1 ref 1 00000100:00100000:1.0:1789760790.514300:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733733612465 time 733759919336 state 0x0 ref 1 00000100:00100000:1.0:1789760790.514312:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554852480:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760790.514356:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66723840, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fd28 00010000:00010000:1.0:1789760790.514364:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67a4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.514425:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760790.514431:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.514437:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66723840->66727935] (req 66723840->66727935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.514451:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66723840->66727935] (req 66723840->66727935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.520978:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983599 is committed 00002000:00010000:1.0:1789760790.521107:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66723840 -> 66727936 00002000:00010000:1.0:1789760790.521113:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130344 -> 130352 00010000:00010000:1.0:1789760790.521127:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66723840->66727935] (req 66723840->66727935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.521140:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67a4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66723840->66727935] (req 66723840->66727935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.521155:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66723840->66727935] (req 66723840->66727935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.521163:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.521168:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66723840->66727935] (req 66723840->66727935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.521177:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66723840->66727935] (req 66723840->66727935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.521198:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67a4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66723840->66727935] (req 66723840->66727935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.521206:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67a4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66723840->66727935] (req 66723840->66727935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.521300:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554852480:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6985us (7274us total) trans 4294983599 rc 0/0 00000100:00100000:1.0:1789760790.521311:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16301 00000100:00100000:1.0:1789760790.521320:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733733612465 time 733759919336 state 0x2 ref 0 00000100:00100000:1.0:1789760790.526303:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760790.526423:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554852608 00000100:00100000:2.0:1789760790.526432:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554852608 00000100:00100000:2.0:1789760790.526486:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733733612465 time 733759919336 state 0x0 ref 1 00000100:00100000:2.0:1789760790.526502:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733760179752 time 733759919336 state 0x1 ref 1 00000100:00100000:2.0:1789760790.526529:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733760179752 time 733772147239 state 0x0 ref 1 00000100:00100000:2.0:1789760790.526541:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a384a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554852608:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760790.526591:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66727936, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fd30 00010000:00010000:2.0:1789760790.526600:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.526614:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760790.526627:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.526634:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66727936->66732031] (req 66727936->66732031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.526647:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66727936->66732031] (req 66727936->66732031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.539495:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983600 is committed 00002000:00010000:2.0:1789760790.540597:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66727936 -> 66732032 00002000:00010000:2.0:1789760790.540604:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130352 -> 130360 00010000:00010000:2.0:1789760790.540628:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66727936->66732031] (req 66727936->66732031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.540642:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66727936->66732031] (req 66727936->66732031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.540651:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66727936->66732031] (req 66727936->66732031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.540659:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760790.540663:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66727936->66732031] (req 66727936->66732031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.540672:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66727936->66732031] (req 66727936->66732031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.540693:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66727936->66732031] (req 66727936->66732031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.540701:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66727936->66732031] (req 66727936->66732031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.540840:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a384a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554852608:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14296us (14542us total) trans 4294983600 rc 0/0 00000100:00100000:2.0:1789760790.540851:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16302 00000100:00100000:2.0:1789760790.540860:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733760179752 time 733772147239 state 0x2 ref 0 00000100:00100000:0.0:1789760790.546583:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760790.546642:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554852736 00000100:00100000:2.0:1789760790.546652:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554852736 00000100:00100000:2.0:1789760790.546707:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733760179752 time 733772147239 state 0x0 ref 1 00000100:00100000:2.0:1789760790.546723:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733772407655 time 733772147239 state 0x1 ref 1 00000100:00100000:2.0:1789760790.546754:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733772407655 time 733792372924 state 0x0 ref 1 00000100:00100000:2.0:1789760790.546765:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554852736:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760790.546821:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66732032, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fd38 00010000:00010000:2.0:1789760790.546831:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c67b2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.546850:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760790.546855:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.546862:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c67b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66732032->66736127] (req 66732032->66736127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.546878:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c67b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66732032->66736127] (req 66732032->66736127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.552140:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983601 is committed 00002000:00010000:2.0:1789760790.552257:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66732032 -> 66736128 00002000:00010000:2.0:1789760790.552263:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130360 -> 130368 00010000:00010000:2.0:1789760790.552276:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c67b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66732032->66736127] (req 66732032->66736127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.552290:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c67b2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66732032->66736127] (req 66732032->66736127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.552298:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c67b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66732032->66736127] (req 66732032->66736127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.552305:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760790.552311:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c67b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66732032->66736127] (req 66732032->66736127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.552321:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c67b2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66732032->66736127] (req 66732032->66736127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.552340:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c67b2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66732032->66736127] (req 66732032->66736127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.552348:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c67b2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66732032->66736127] (req 66732032->66736127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.552493:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554852736:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5725us (5916us total) trans 4294983601 rc 0/0 00000100:00100000:2.0:1789760790.552504:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16303 00000100:00100000:2.0:1789760790.552513:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733772407655 time 733792372924 state 0x2 ref 0 00000100:00100000:3.0:1789760790.557577:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760790.557780:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554852864 00000100:00100000:1.0:1789760790.557800:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554852864 00000100:00100000:1.0:1789760790.557857:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733772407655 time 733792372924 state 0x0 ref 1 00000100:00100000:1.0:1789760790.557875:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733792633340 time 733792372924 state 0x1 ref 1 00000100:00100000:1.0:1789760790.557905:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733792633340 time 733803524067 state 0x0 ref 1 00000100:00100000:1.0:1789760790.557917:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554852864:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760790.557966:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66736128, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fd40 00010000:00010000:1.0:1789760790.557976:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67b9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.557992:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760790.558006:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.558013:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66736128->66740223] (req 66736128->66740223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.558027:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66736128->66740223] (req 66736128->66740223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.562703:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983602 is committed 00002000:00010000:1.0:1789760790.563175:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66736128 -> 66740224 00002000:00010000:1.0:1789760790.563181:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130368 -> 130376 00010000:00010000:1.0:1789760790.563249:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66736128->66740223] (req 66736128->66740223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.563264:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66736128->66740223] (req 66736128->66740223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.563274:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66736128->66740223] (req 66736128->66740223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.563283:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.563288:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66736128->66740223] (req 66736128->66740223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.563297:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67b9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66736128->66740223] (req 66736128->66740223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.563315:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67b9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66736128->66740223] (req 66736128->66740223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.563322:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67b9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66736128->66740223] (req 66736128->66740223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.563554:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554852864:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5634us (5982us total) trans 4294983602 rc 0/0 00000100:00100000:1.0:1789760790.563565:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16304 00000100:00100000:1.0:1789760790.563574:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733792633340 time 733803524067 state 0x2 ref 0 00000100:00100000:3.0:1789760790.568468:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760790.568682:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554852992 00000100:00100000:1.0:1789760790.568692:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554852992 00000100:00100000:1.0:1789760790.569421:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733792633340 time 733803524067 state 0x0 ref 1 00000100:00100000:1.0:1789760790.569440:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733803784483 time 733803524067 state 0x1 ref 1 00000100:00100000:1.0:1789760790.569475:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733803784483 time 733815093457 state 0x0 ref 1 00000100:00100000:1.0:1789760790.569493:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554852992:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760790.569538:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66740224, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fd48 00010000:00010000:1.0:1789760790.569547:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67c0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.569563:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760790.569568:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.569575:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66740224->66744319] (req 66740224->66744319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.569587:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66740224->66744319] (req 66740224->66744319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.575646:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983603 is committed 00002000:00010000:1.0:1789760790.575915:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66740224 -> 66744320 00002000:00010000:1.0:1789760790.575921:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130376 -> 130384 00010000:00010000:1.0:1789760790.575934:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66740224->66744319] (req 66740224->66744319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.575948:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67c0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66740224->66744319] (req 66740224->66744319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.575958:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66740224->66744319] (req 66740224->66744319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.575967:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.575973:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66740224->66744319] (req 66740224->66744319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.575983:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67c0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66740224->66744319] (req 66740224->66744319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.576063:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67c0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66740224->66744319] (req 66740224->66744319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.576074:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67c0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66740224->66744319] (req 66740224->66744319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.576199:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554852992:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6704us (7737us total) trans 4294983603 rc 0/0 00000100:00100000:1.0:1789760790.576210:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16305 00000100:00100000:1.0:1789760790.576219:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733803784483 time 733815093457 state 0x2 ref 0 00000100:00100000:2.0:1789760790.596296:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760790.596432:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554853120 00000100:00100000:1.0:1789760790.596444:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554853120 00000100:00100000:1.0:1789760790.596497:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733803784483 time 733815093457 state 0x0 ref 1 00000100:00100000:1.0:1789760790.596513:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733815353873 time 733815093457 state 0x1 ref 1 00000100:00100000:1.0:1789760790.596548:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733815353873 time 733842163977 state 0x0 ref 1 00000100:00100000:1.0:1789760790.596567:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09859500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554853120:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760790.596644:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66744320, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fd50 00010000:00010000:1.0:1789760790.596658:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67c7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.596680:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760790.596688:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.596697:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66744320->66748415] (req 66744320->66748415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.596713:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66744320->66748415] (req 66744320->66748415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.605705:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983604 is committed 00002000:00010000:1.0:1789760790.605934:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66744320 -> 66748416 00002000:00010000:1.0:1789760790.605939:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130384 -> 130392 00010000:00010000:1.0:1789760790.605952:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66744320->66748415] (req 66744320->66748415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.605965:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66744320->66748415] (req 66744320->66748415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.605974:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66744320->66748415] (req 66744320->66748415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.605981:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.605986:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66744320->66748415] (req 66744320->66748415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.606004:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66744320->66748415] (req 66744320->66748415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.606024:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67c7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66744320->66748415] (req 66744320->66748415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.606031:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67c7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66744320->66748415] (req 66744320->66748415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.606126:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09859500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554853120:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9559us (9833us total) trans 4294983604 rc 0/0 00000100:00100000:1.0:1789760790.606136:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16306 00000100:00100000:1.0:1789760790.606145:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733815353873 time 733842163977 state 0x2 ref 0 00000100:00100000:3.0:1789760790.612553:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760790.612829:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554853248 00000100:00100000:2.0:1789760790.612839:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554853248 00000100:00100000:2.0:1789760790.612888:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733815353873 time 733842163977 state 0x0 ref 1 00000100:00100000:2.0:1789760790.612903:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733842424393 time 733842163977 state 0x1 ref 1 00000100:00100000:2.0:1789760790.612932:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733842424393 time 733858550809 state 0x0 ref 1 00000100:00100000:2.0:1789760790.612944:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554853248:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760790.612987:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66748416, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fd58 00010000:00010000:2.0:1789760790.614981:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.615061:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760790.615066:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.615072:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66748416->66752511] (req 66748416->66752511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.615087:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66748416->66752511] (req 66748416->66752511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.621701:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983605 is committed 00002000:00010000:1.0:1789760790.621885:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66748416 -> 66752512 00002000:00010000:1.0:1789760790.621890:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130392 -> 130400 00010000:00010000:1.0:1789760790.621906:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66748416->66752511] (req 66748416->66752511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.621919:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66748416->66752511] (req 66748416->66752511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.621928:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66748416->66752511] (req 66748416->66752511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.621935:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.621939:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66748416->66752511] (req 66748416->66752511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.621953:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66748416->66752511] (req 66748416->66752511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.621972:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66748416->66752511] (req 66748416->66752511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.621979:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66748416->66752511] (req 66748416->66752511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.622128:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554853248:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9182us (9579us total) trans 4294983605 rc 0/0 00000100:00100000:1.0:1789760790.622139:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16307 00000100:00100000:1.0:1789760790.622149:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733842424393 time 733858550809 state 0x2 ref 0 00000100:00100000:3.0:1789760790.626133:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760790.626869:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554853376 00000100:00100000:2.0:1789760790.626879:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554853376 00000100:00100000:2.0:1789760790.626930:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733842424393 time 733858550809 state 0x0 ref 1 00000100:00100000:2.0:1789760790.626946:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733858811225 time 733858550809 state 0x1 ref 1 00000100:00100000:2.0:1789760790.627012:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733858811225 time 733872630030 state 0x0 ref 1 00000100:00100000:2.0:1789760790.627025:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554853376:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760790.627072:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66752512, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fd60 00010000:00010000:2.0:1789760790.627080:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c67d5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.627097:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760790.627102:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.627108:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c67d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66752512->66756607] (req 66752512->66756607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.627122:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c67d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66752512->66756607] (req 66752512->66756607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.632567:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983606 is committed 00002000:00010000:2.0:1789760790.632809:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66752512 -> 66756608 00002000:00010000:2.0:1789760790.632814:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130400 -> 130408 00010000:00010000:2.0:1789760790.632826:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c67d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66752512->66756607] (req 66752512->66756607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.632838:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c67d5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66752512->66756607] (req 66752512->66756607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.632853:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c67d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66752512->66756607] (req 66752512->66756607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.632860:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760790.632865:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c67d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66752512->66756607] (req 66752512->66756607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.632874:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c67d5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66752512->66756607] (req 66752512->66756607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.632892:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c67d5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66752512->66756607] (req 66752512->66756607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.632900:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c67d5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66752512->66756607] (req 66752512->66756607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.633089:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554853376:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6059us (6959us total) trans 4294983606 rc 0/0 00000100:00100000:2.0:1789760790.633100:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16308 00000100:00100000:2.0:1789760790.633109:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733858811225 time 733872630030 state 0x2 ref 0 00000100:00100000:2.0:1789760790.637086:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760790.637151:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554853504 00000100:00100000:1.0:1789760790.637160:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554853504 00000100:00100000:1.0:1789760790.637209:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733858811225 time 733872630030 state 0x0 ref 1 00000100:00100000:1.0:1789760790.637220:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733872890446 time 733872630030 state 0x1 ref 1 00000100:00100000:1.0:1789760790.637245:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733872890446 time 733882863355 state 0x0 ref 1 00000100:00100000:1.0:1789760790.637257:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09858e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554853504:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760790.637304:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66756608, mtime: 0x6aad9515, atime: 0x0, ctime: 0x6aad9515, blocks: 0x1fd68 00010000:00010000:1.0:1789760790.637314:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67dc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.637329:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760790.637334:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.637340:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66756608->66760703] (req 66756608->66760703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.637354:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66756608->66760703] (req 66756608->66760703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.644601:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983607 is committed 00002000:00010000:1.0:1789760790.644976:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66756608 -> 66760704 00002000:00010000:1.0:1789760790.644980:0:8442:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760789 -> 1789760790 00002000:00010000:1.0:1789760790.644983:0:8442:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760789 -> 1789760790 00002000:00010000:1.0:1789760790.644985:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130408 -> 130416 00010000:00010000:1.0:1789760790.645101:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66756608->66760703] (req 66756608->66760703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.645114:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67dc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66756608->66760703] (req 66756608->66760703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.645123:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66756608->66760703] (req 66756608->66760703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.645130:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.645134:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66756608->66760703] (req 66756608->66760703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.645143:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66756608->66760703] (req 66756608->66760703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.645163:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67dc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66756608->66760703] (req 66756608->66760703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.645170:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67dc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66756608->66760703] (req 66756608->66760703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.645257:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09858e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554853504:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7998us (8176us total) trans 4294983607 rc 0/0 00000100:00100000:1.0:1789760790.645267:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16309 00000100:00100000:1.0:1789760790.645276:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733872890446 time 733882863355 state 0x2 ref 0 00000100:00100000:3.0:1789760790.652191:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760790.652308:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554853632 00000100:00100000:1.0:1789760790.652319:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554853632 00000100:00100000:1.0:1789760790.652373:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733872890446 time 733882863355 state 0x0 ref 1 00000100:00100000:1.0:1789760790.652432:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733883123771 time 733882863355 state 0x1 ref 1 00000100:00100000:1.0:1789760790.652462:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733883123771 time 733898080235 state 0x0 ref 1 00000100:00100000:1.0:1789760790.652475:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb065a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554853632:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760790.652521:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66760704, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fd70 00010000:00010000:1.0:1789760790.652530:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67e3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.652545:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760790.652550:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.652556:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66760704->66764799] (req 66760704->66764799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.652569:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66760704->66764799] (req 66760704->66764799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.657426:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983608 is committed 00002000:00010000:1.0:1789760790.657943:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66760704 -> 66764800 00002000:00010000:1.0:1789760790.657949:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130416 -> 130424 00010000:00010000:1.0:1789760790.657962:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66760704->66764799] (req 66760704->66764799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.657974:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67e3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66760704->66764799] (req 66760704->66764799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.657983:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66760704->66764799] (req 66760704->66764799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.657990:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.657995:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66760704->66764799] (req 66760704->66764799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.658005:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67e3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66760704->66764799] (req 66760704->66764799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.658025:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67e3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66760704->66764799] (req 66760704->66764799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.658039:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c67e3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66760704->66764799] (req 66760704->66764799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.659367:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554853632:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6883us (7172us total) trans 4294983608 rc 0/0 00000100:00100000:1.0:1789760790.659401:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16310 00000100:00100000:1.0:1789760790.659418:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733883123771 time 733898080235 state 0x2 ref 0 00000100:00100000:3.0:1789760790.665441:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760790.665582:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554853760 00000100:00100000:2.0:1789760790.665593:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554853760 00000100:00100000:2.0:1789760790.665655:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733883123771 time 733898080235 state 0x0 ref 1 00000100:00100000:2.0:1789760790.665673:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733898340651 time 733898080235 state 0x1 ref 1 00000100:00100000:2.0:1789760790.665731:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733898340651 time 733911349248 state 0x0 ref 1 00000100:00100000:2.0:1789760790.665744:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe1bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554853760:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760790.665791:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66764800, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fd78 00010000:00010000:2.0:1789760790.665800:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.665815:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760790.665820:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.665827:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66764800->66768895] (req 66764800->66768895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.665838:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66764800->66768895] (req 66764800->66768895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.671472:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983609 is committed 00002000:00010000:2.0:1789760790.671689:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66764800 -> 66768896 00002000:00010000:2.0:1789760790.671694:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130424 -> 130432 00010000:00010000:2.0:1789760790.671706:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66764800->66768895] (req 66764800->66768895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.671719:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66764800->66768895] (req 66764800->66768895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.671729:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66764800->66768895] (req 66764800->66768895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.671737:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760790.671742:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66764800->66768895] (req 66764800->66768895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.671750:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66764800->66768895] (req 66764800->66768895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.671770:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66764800->66768895] (req 66764800->66768895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.671777:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c67ea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66764800->66768895] (req 66764800->66768895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.671876:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe1bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554853760:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6130us (6440us total) trans 4294983609 rc 0/0 00000100:00100000:2.0:1789760790.671888:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16311 00000100:00100000:2.0:1789760790.671898:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733898340651 time 733911349248 state 0x2 ref 0 00000100:00100000:2.0:1789760790.679165:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760790.679593:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554853888 00000100:00100000:1.0:1789760790.679604:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554853888 00000100:00100000:1.0:1789760790.679672:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733898340651 time 733911349248 state 0x0 ref 1 00000100:00100000:1.0:1789760790.679687:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733911609664 time 733911349248 state 0x1 ref 1 00000100:00100000:1.0:1789760790.679718:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733911609664 time 733925336754 state 0x0 ref 1 00000100:00100000:1.0:1789760790.679731:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09858380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554853888:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760790.679775:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66768896, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fd80 00010000:00010000:1.0:1789760790.679784:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67f1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.679799:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760790.679804:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.679811:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66768896->66772991] (req 66768896->66772991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.679829:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66768896->66772991] (req 66768896->66772991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.685595:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983610 is committed 00002000:00010000:2.0:1789760790.685817:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66768896 -> 66772992 00002000:00010000:2.0:1789760790.685822:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130432 -> 130440 00010000:00010000:2.0:1789760790.685836:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66768896->66772991] (req 66768896->66772991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.685848:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67f1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66768896->66772991] (req 66768896->66772991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.685859:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66768896->66772991] (req 66768896->66772991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.685866:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760790.685870:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66768896->66772991] (req 66768896->66772991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.685879:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66768896->66772991] (req 66768896->66772991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.685896:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67f1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66768896->66772991] (req 66768896->66772991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.685903:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c67f1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66768896->66772991] (req 66768896->66772991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.686083:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09858380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554853888:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6350us (6924us total) trans 4294983610 rc 0/0 00000100:00100000:2.0:1789760790.686092:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16312 00000100:00100000:2.0:1789760790.686101:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733911609664 time 733925336754 state 0x2 ref 0 00000100:00100000:1.0:1789760790.692062:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760790.692175:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554854016 00000100:00100000:3.0:1789760790.692185:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554854016 00000100:00100000:3.0:1789760790.692239:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733911609664 time 733925336754 state 0x0 ref 1 00000100:00100000:3.0:1789760790.692254:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733925597170 time 733925336754 state 0x1 ref 1 00000100:00100000:3.0:1789760790.692333:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733925597170 time 733937950158 state 0x0 ref 1 00000100:00100000:3.0:1789760790.692347:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554854016:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760790.692444:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66772992, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fd88 00010000:00010000:3.0:1789760790.692452:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c67f8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.692468:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789760790.692472:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.692479:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c67f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66772992->66777087] (req 66772992->66777087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.692491:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c67f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66772992->66777087] (req 66772992->66777087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.697715:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983611 is committed 00002000:00010000:3.0:1789760790.697879:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66772992 -> 66777088 00002000:00010000:3.0:1789760790.697884:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130440 -> 130448 00010000:00010000:3.0:1789760790.697897:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c67f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66772992->66777087] (req 66772992->66777087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.697910:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c67f8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66772992->66777087] (req 66772992->66777087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.697918:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c67f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66772992->66777087] (req 66772992->66777087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.697926:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760790.699837:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c67f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66772992->66777087] (req 66772992->66777087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.699860:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c67f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66772992->66777087] (req 66772992->66777087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.699881:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c67f8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66772992->66777087] (req 66772992->66777087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.699887:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c67f8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66772992->66777087] (req 66772992->66777087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760790.699982:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554854016:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7634us (8049us total) trans 4294983611 rc 0/0 00000100:00100000:3.0:1789760790.699995:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16313 00000100:00100000:3.0:1789760790.700004:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733925597170 time 733937950158 state 0x2 ref 0 00000100:00100000:3.0:1789760790.705437:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760790.705526:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554854144 00000100:00100000:2.0:1789760790.705535:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554854144 00000100:00100000:2.0:1789760790.705590:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733925597170 time 733937950158 state 0x0 ref 1 00000100:00100000:2.0:1789760790.705605:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733938210574 time 733937950158 state 0x1 ref 1 00000100:00100000:2.0:1789760790.705650:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733938210574 time 733951268068 state 0x0 ref 1 00000100:00100000:2.0:1789760790.705662:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb065a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554854144:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760790.705708:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66777088, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fd90 00010000:00010000:2.0:1789760790.705718:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c67ff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.705734:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760790.705739:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.705746:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c67ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66777088->66781183] (req 66777088->66781183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.705758:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c67ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66777088->66781183] (req 66777088->66781183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760790.715206:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983612 is committed 00002000:00010000:2.0:1789760790.716271:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66777088 -> 66781184 00002000:00010000:2.0:1789760790.716278:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130448 -> 130456 00010000:00010000:2.0:1789760790.716291:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c67ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66777088->66781183] (req 66777088->66781183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.716304:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c67ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66777088->66781183] (req 66777088->66781183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.716312:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c67ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66777088->66781183] (req 66777088->66781183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.716324:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760790.716330:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c67ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66777088->66781183] (req 66777088->66781183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.716338:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c67ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66777088->66781183] (req 66777088->66781183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.716358:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c67ff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66777088->66781183] (req 66777088->66781183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.716427:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c67ff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66777088->66781183] (req 66777088->66781183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.717552:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554854144:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10956us (11188us total) trans 4294983612 rc 0/0 00000100:00100000:2.0:1789760790.717564:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16314 00000100:00100000:2.0:1789760790.717574:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733938210574 time 733951268068 state 0x2 ref 0 00000100:00100000:2.0:1789760790.725847:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760790.725969:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554854272 00000100:00100000:1.0:1789760790.725979:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554854272 00000100:00100000:1.0:1789760790.726036:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733938210574 time 733951268068 state 0x0 ref 1 00000100:00100000:1.0:1789760790.726051:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733951528484 time 733951268068 state 0x1 ref 1 00000100:00100000:1.0:1789760790.726087:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733951528484 time 733971705273 state 0x0 ref 1 00000100:00100000:1.0:1789760790.726100:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0985a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554854272:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760790.726148:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66781184, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fd98 00010000:00010000:1.0:1789760790.726156:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6806 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.726172:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:1.0:1789760790.726176:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.726183:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6806 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66781184->66785279] (req 66781184->66785279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.726196:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6806 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66781184->66785279] (req 66781184->66785279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.733180:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983613 is committed 00002000:00010000:1.0:1789760790.733463:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66781184 -> 66785280 00002000:00010000:1.0:1789760790.733468:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130456 -> 130464 00010000:00010000:1.0:1789760790.733481:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6806 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66781184->66785279] (req 66781184->66785279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.733495:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6806 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66781184->66785279] (req 66781184->66785279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.733504:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6806 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66781184->66785279] (req 66781184->66785279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.733511:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.733515:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6806 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66781184->66785279] (req 66781184->66785279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.733523:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6806 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66781184->66785279] (req 66781184->66785279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.733542:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6806 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66781184->66785279] (req 66781184->66785279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.733549:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6806 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66781184->66785279] (req 66781184->66785279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.733648:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0985a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554854272:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7545us (7804us total) trans 4294983613 rc 0/0 00000100:00100000:1.0:1789760790.733658:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16315 00000100:00100000:1.0:1789760790.733666:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733951528484 time 733971705273 state 0x2 ref 0 00000100:00100000:1.0:1789760790.743256:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760790.743637:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554854400 00000100:00100000:2.0:1789760790.743646:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554854400 00000100:00100000:2.0:1789760790.743706:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733951528484 time 733971705273 state 0x0 ref 1 00000100:00100000:2.0:1789760790.743870:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733971965689 time 733971705273 state 0x1 ref 1 00000100:00100000:2.0:1789760790.743905:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733971965689 time 733989522796 state 0x0 ref 1 00000100:00100000:2.0:1789760790.743920:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a386680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554854400:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760790.743969:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66785280, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fda0 00010000:00010000:2.0:1789760790.743978:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c680d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.744003:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=733s v=(0 0) 00010000:00010000:2.0:1789760790.744008:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.744014:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c680d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66785280->66789375] (req 66785280->66789375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.744028:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c680d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66785280->66789375] (req 66785280->66789375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.751751:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983614 is committed 00002000:00010000:2.0:1789760790.751879:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66785280 -> 66789376 00002000:00010000:2.0:1789760790.751884:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130464 -> 130472 00010000:00010000:2.0:1789760790.751897:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c680d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66785280->66789375] (req 66785280->66789375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.751910:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c680d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66785280->66789375] (req 66785280->66789375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.751920:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c680d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66785280->66789375] (req 66785280->66789375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.751928:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760790.751934:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c680d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66785280->66789375] (req 66785280->66789375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.751944:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c680d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66785280->66789375] (req 66785280->66789375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.751963:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c680d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66785280->66789375] (req 66785280->66789375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.751972:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c680d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66785280->66789375] (req 66785280->66789375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.752163:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a386680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554854400:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8241us (8913us total) trans 4294983614 rc 0/0 00000100:00100000:2.0:1789760790.752175:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16316 00000100:00100000:2.0:1789760790.752184:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733971965689 time 733989522796 state 0x2 ref 0 00000100:00100000:3.0:1789760790.756282:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760790.756402:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554854528 00000100:00100000:1.0:1789760790.756413:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554854528 00000100:00100000:1.0:1789760790.756466:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733971965689 time 733989522796 state 0x0 ref 1 00000100:00100000:1.0:1789760790.756481:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733989783212 time 733989522796 state 0x1 ref 1 00000100:00100000:1.0:1789760790.756508:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733989783212 time 734002126959 state 0x0 ref 1 00000100:00100000:1.0:1789760790.756521:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb065a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554854528:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760790.756570:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66789376, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fda8 00010000:00010000:1.0:1789760790.756584:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6814 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.756601:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760790.756606:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.756612:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6814 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66789376->66793471] (req 66789376->66793471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.756636:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6814 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66789376->66793471] (req 66789376->66793471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.762929:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983615 is committed 00002000:00010000:1.0:1789760790.763047:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66789376 -> 66793472 00002000:00010000:1.0:1789760790.763052:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130472 -> 130480 00010000:00010000:1.0:1789760790.763065:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6814 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66789376->66793471] (req 66789376->66793471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.763077:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6814 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66789376->66793471] (req 66789376->66793471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.763086:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6814 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66789376->66793471] (req 66789376->66793471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.763093:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.763098:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6814 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66789376->66793471] (req 66789376->66793471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.763108:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6814 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66789376->66793471] (req 66789376->66793471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.763127:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6814 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66789376->66793471] (req 66789376->66793471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.763135:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6814 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66789376->66793471] (req 66789376->66793471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.763223:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554854528:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6699us (6944us total) trans 4294983615 rc 0/0 00000100:00100000:1.0:1789760790.763233:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16317 00000100:00100000:1.0:1789760790.763242:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733989783212 time 734002126959 state 0x2 ref 0 00000100:00100000:0.0:1789760790.766776:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760790.766927:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554854656 00000100:00100000:1.0:1789760790.766938:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554854656 00000100:00100000:1.0:1789760790.766990:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 733989783212 time 734002126959 state 0x0 ref 1 00000100:00100000:1.0:1789760790.767005:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734002387375 time 734002126959 state 0x1 ref 1 00000100:00100000:1.0:1789760790.767038:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734002387375 time 734012655638 state 0x0 ref 1 00000100:00100000:1.0:1789760790.767050:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554854656:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760790.767096:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66793472, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fdb0 00010000:00010000:1.0:1789760790.767106:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c681b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.767121:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760790.767126:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.767132:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c681b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66793472->66797567] (req 66793472->66797567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.767146:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c681b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66793472->66797567] (req 66793472->66797567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.773265:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983616 is committed 00002000:00010000:1.0:1789760790.773487:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66793472 -> 66797568 00002000:00010000:1.0:1789760790.773492:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130480 -> 130488 00010000:00010000:1.0:1789760790.773506:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c681b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66793472->66797567] (req 66793472->66797567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.773525:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c681b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66793472->66797567] (req 66793472->66797567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.773535:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c681b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66793472->66797567] (req 66793472->66797567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.773543:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.773548:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c681b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66793472->66797567] (req 66793472->66797567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.773556:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c681b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66793472->66797567] (req 66793472->66797567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.773576:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c681b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66793472->66797567] (req 66793472->66797567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.773594:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c681b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66793472->66797567] (req 66793472->66797567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.773692:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554854656:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6639us (6921us total) trans 4294983616 rc 0/0 00000100:00100000:1.0:1789760790.773702:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16318 00000100:00100000:1.0:1789760790.773711:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734002387375 time 734012655638 state 0x2 ref 0 00000100:00100000:3.0:1789760790.780733:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760790.780843:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554854784 00000100:00100000:2.0:1789760790.780854:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554854784 00000100:00100000:2.0:1789760790.780908:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734002387375 time 734012655638 state 0x0 ref 1 00000100:00100000:2.0:1789760790.780925:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734012916054 time 734012655638 state 0x1 ref 1 00000100:00100000:2.0:1789760790.780969:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734012916054 time 734026587116 state 0x0 ref 1 00000100:00100000:2.0:1789760790.780981:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb065a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554854784:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760790.781038:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66797568, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fdb8 00010000:00010000:2.0:1789760790.781048:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6822 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.781064:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760790.781069:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.781075:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6822 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66797568->66801663] (req 66797568->66801663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.781089:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6822 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66797568->66801663] (req 66797568->66801663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.786538:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983617 is committed 00002000:00010000:2.0:1789760790.787059:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66797568 -> 66801664 00002000:00010000:2.0:1789760790.787065:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130488 -> 130496 00010000:00010000:2.0:1789760790.787078:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6822 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66797568->66801663] (req 66797568->66801663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.787091:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6822 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66797568->66801663] (req 66797568->66801663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.787101:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6822 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66797568->66801663] (req 66797568->66801663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.787109:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760790.787113:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6822 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66797568->66801663] (req 66797568->66801663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.787122:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6822 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66797568->66801663] (req 66797568->66801663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.787143:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6822 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66797568->66801663] (req 66797568->66801663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.787151:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6822 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66797568->66801663] (req 66797568->66801663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.787526:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554854784:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6393us (6648us total) trans 4294983617 rc 0/0 00000100:00100000:2.0:1789760790.787540:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16319 00000100:00100000:2.0:1789760790.787552:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734012916054 time 734026587116 state 0x2 ref 0 00000100:00100000:3.0:1789760790.792841:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760790.793023:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554854912 00000100:00100000:2.0:1789760790.793034:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554854912 00000100:00100000:2.0:1789760790.793085:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734012916054 time 734026587116 state 0x0 ref 1 00000100:00100000:2.0:1789760790.793100:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734026847532 time 734026587116 state 0x1 ref 1 00000100:00100000:2.0:1789760790.793134:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734026847532 time 734038752421 state 0x0 ref 1 00000100:00100000:2.0:1789760790.793147:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb065a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554854912:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760790.793194:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66801664, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fdc0 00010000:00010000:2.0:1789760790.793204:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6829 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.793220:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760790.793226:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.793232:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6829 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66801664->66805759] (req 66801664->66805759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.793252:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6829 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66801664->66805759] (req 66801664->66805759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.797682:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983618 is committed 00002000:00010000:2.0:1789760790.797795:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66801664 -> 66805760 00002000:00010000:2.0:1789760790.797800:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130496 -> 130504 00010000:00010000:2.0:1789760790.797812:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6829 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66801664->66805759] (req 66801664->66805759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.797825:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6829 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66801664->66805759] (req 66801664->66805759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.797834:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6829 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66801664->66805759] (req 66801664->66805759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.797842:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760790.797846:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6829 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66801664->66805759] (req 66801664->66805759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.797854:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6829 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66801664->66805759] (req 66801664->66805759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.797872:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6829 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66801664->66805759] (req 66801664->66805759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.797885:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6829 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66801664->66805759] (req 66801664->66805759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.797980:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554854912:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4831us (5145us total) trans 4294983618 rc 0/0 00000100:00100000:2.0:1789760790.797999:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16320 00000100:00100000:2.0:1789760790.798009:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734026847532 time 734038752421 state 0x2 ref 0 00000100:00100000:0.0:1789760790.803108:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760790.803196:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554855040 00000100:00100000:1.0:1789760790.803206:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554855040 00000100:00100000:1.0:1789760790.803260:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734026847532 time 734038752421 state 0x0 ref 1 00000100:00100000:1.0:1789760790.803275:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734039012837 time 734038752421 state 0x1 ref 1 00000100:00100000:1.0:1789760790.803308:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734039012837 time 734048926714 state 0x0 ref 1 00000100:00100000:1.0:1789760790.803320:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554855040:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760790.803370:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66805760, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fdc8 00010000:00010000:1.0:1789760790.803421:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6830 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.803437:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760790.803441:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.803448:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6830 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66805760->66809855] (req 66805760->66809855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.803464:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6830 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66805760->66809855] (req 66805760->66809855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.808130:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983619 is committed 00002000:00010000:1.0:1789760790.808331:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66805760 -> 66809856 00002000:00010000:1.0:1789760790.808336:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130504 -> 130512 00010000:00010000:1.0:1789760790.808349:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6830 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66805760->66809855] (req 66805760->66809855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.808362:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6830 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66805760->66809855] (req 66805760->66809855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.808547:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6830 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66805760->66809855] (req 66805760->66809855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.808556:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.808562:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6830 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66805760->66809855] (req 66805760->66809855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.808571:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6830 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66805760->66809855] (req 66805760->66809855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.808591:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6830 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66805760->66809855] (req 66805760->66809855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.808599:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6830 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66805760->66809855] (req 66805760->66809855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.808698:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554855040:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5374us (5597us total) trans 4294983619 rc 0/0 00000100:00100000:1.0:1789760790.808709:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16321 00000100:00100000:1.0:1789760790.808718:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734039012837 time 734048926714 state 0x2 ref 0 00000100:00100000:3.0:1789760790.813317:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760790.813409:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554855168 00000100:00100000:1.0:1789760790.813419:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554855168 00000100:00100000:1.0:1789760790.813470:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734039012837 time 734048926714 state 0x0 ref 1 00000100:00100000:1.0:1789760790.813485:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734049187130 time 734048926714 state 0x1 ref 1 00000100:00100000:1.0:1789760790.813569:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734049187130 time 734059186410 state 0x0 ref 1 00000100:00100000:1.0:1789760790.813583:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb065a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554855168:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760790.813651:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66809856, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fdd0 00010000:00010000:1.0:1789760790.813661:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6837 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.813684:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760790.813688:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.813700:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6837 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66809856->66813951] (req 66809856->66813951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.813714:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6837 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66809856->66813951] (req 66809856->66813951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.818833:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983620 is committed 00002000:00010000:1.0:1789760790.818970:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66809856 -> 66813952 00002000:00010000:1.0:1789760790.818975:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130512 -> 130520 00010000:00010000:1.0:1789760790.818989:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6837 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66809856->66813951] (req 66809856->66813951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.819000:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6837 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66809856->66813951] (req 66809856->66813951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.819008:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6837 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66809856->66813951] (req 66809856->66813951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.819020:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.819025:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6837 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66809856->66813951] (req 66809856->66813951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.819033:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6837 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66809856->66813951] (req 66809856->66813951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.819051:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6837 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66809856->66813951] (req 66809856->66813951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.819059:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6837 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66809856->66813951] (req 66809856->66813951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.819149:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554855168:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5564us (5835us total) trans 4294983620 rc 0/0 00000100:00100000:1.0:1789760790.819158:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16322 00000100:00100000:1.0:1789760790.819169:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734049187130 time 734059186410 state 0x2 ref 0 00000100:00100000:3.0:1789760790.823176:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760790.823287:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554855296 00000100:00100000:2.0:1789760790.823296:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554855296 00000100:00100000:2.0:1789760790.823345:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734049187130 time 734059186410 state 0x0 ref 1 00000100:00100000:2.0:1789760790.823361:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734059446826 time 734059186410 state 0x1 ref 1 00000100:00100000:2.0:1789760790.823511:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734059446826 time 734069128186 state 0x0 ref 1 00000100:00100000:2.0:1789760790.823525:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb065a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554855296:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760790.823580:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66813952, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fdd8 00010000:00010000:2.0:1789760790.823718:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c683e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.823734:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760790.823739:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.823745:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c683e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66813952->66818047] (req 66813952->66818047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.823759:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c683e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66813952->66818047] (req 66813952->66818047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.831071:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983621 is committed 00002000:00010000:2.0:1789760790.831201:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66813952 -> 66818048 00002000:00010000:2.0:1789760790.831207:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130520 -> 130528 00010000:00010000:2.0:1789760790.831220:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c683e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66813952->66818047] (req 66813952->66818047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.831233:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c683e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66813952->66818047] (req 66813952->66818047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.831241:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c683e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66813952->66818047] (req 66813952->66818047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.831248:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760790.831253:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c683e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66813952->66818047] (req 66813952->66818047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.831263:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c683e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66813952->66818047] (req 66813952->66818047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.831282:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c683e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66813952->66818047] (req 66813952->66818047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.831289:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c683e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66813952->66818047] (req 66813952->66818047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.831437:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554855296:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7909us (8321us total) trans 4294983621 rc 0/0 00000100:00100000:2.0:1789760790.831447:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16323 00000100:00100000:2.0:1789760790.831456:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734059446826 time 734069128186 state 0x2 ref 0 00000100:00100000:0.0:1789760790.835658:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760790.836146:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554855424 00000100:00100000:2.0:1789760790.836157:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554855424 00000100:00100000:2.0:1789760790.836210:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734059446826 time 734069128186 state 0x0 ref 1 00000100:00100000:2.0:1789760790.836228:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734069388602 time 734069128186 state 0x1 ref 1 00000100:00100000:2.0:1789760790.836260:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734069388602 time 734081878492 state 0x0 ref 1 00000100:00100000:2.0:1789760790.836273:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554855424:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760790.836317:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66818048, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fde0 00010000:00010000:2.0:1789760790.836326:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6845 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.836341:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760790.836345:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.836350:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6845 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66818048->66822143] (req 66818048->66822143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.836364:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6845 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66818048->66822143] (req 66818048->66822143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.840344:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983622 is committed 00002000:00010000:0.0:1789760790.840511:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66818048 -> 66822144 00002000:00010000:0.0:1789760790.840517:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130528 -> 130536 00010000:00010000:0.0:1789760790.840532:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6845 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66818048->66822143] (req 66818048->66822143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.840545:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6845 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66818048->66822143] (req 66818048->66822143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.840554:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6845 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66818048->66822143] (req 66818048->66822143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.840562:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760790.840568:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6845 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66818048->66822143] (req 66818048->66822143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.840578:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6845 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66818048->66822143] (req 66818048->66822143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.840597:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6845 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66818048->66822143] (req 66818048->66822143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.840688:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6845 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66818048->66822143] (req 66818048->66822143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.840786:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554855424:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4510us (5131us total) trans 4294983622 rc 0/0 00000100:00100000:0.0:1789760790.840798:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16324 00000100:00100000:0.0:1789760790.840809:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734069388602 time 734081878492 state 0x2 ref 0 00000100:00100000:3.0:1789760790.843909:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760790.844007:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554855552 00000100:00100000:1.0:1789760790.844019:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554855552 00000100:00100000:1.0:1789760790.844335:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734069388602 time 734081878492 state 0x0 ref 1 00000100:00100000:1.0:1789760790.844350:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734082138908 time 734081878492 state 0x1 ref 1 00000100:00100000:2.0:1789760790.844592:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734082138908 time 734090206895 state 0x0 ref 1 00000100:00100000:2.0:1789760790.844608:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb065a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554855552:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760790.844672:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66822144, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fde8 00010000:00010000:2.0:1789760790.844682:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c684c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.844701:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760790.844705:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.844711:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c684c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66822144->66826239] (req 66822144->66826239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.844725:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c684c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66822144->66826239] (req 66822144->66826239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.850932:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983623 is committed 00002000:00010000:2.0:1789760790.851060:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66822144 -> 66826240 00002000:00010000:2.0:1789760790.851065:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130536 -> 130544 00010000:00010000:2.0:1789760790.851078:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c684c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66822144->66826239] (req 66822144->66826239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.851092:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c684c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66822144->66826239] (req 66822144->66826239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.851101:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c684c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66822144->66826239] (req 66822144->66826239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.851109:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760790.851114:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c684c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66822144->66826239] (req 66822144->66826239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.851122:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c684c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66822144->66826239] (req 66822144->66826239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.851141:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c684c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66822144->66826239] (req 66822144->66826239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.851149:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c684c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66822144->66826239] (req 66822144->66826239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760790.851296:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554855552:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6686us (7391us total) trans 4294983623 rc 0/0 00000100:00100000:2.0:1789760790.851308:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16325 00000100:00100000:2.0:1789760790.851318:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734082138908 time 734090206895 state 0x2 ref 0 00000100:00100000:3.0:1789760790.858839:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760790.858987:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554855680 00000100:00100000:1.0:1789760790.858999:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554855680 00000100:00100000:1.0:1789760790.859051:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734082138908 time 734090206895 state 0x0 ref 1 00000100:00100000:1.0:1789760790.859065:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734090467311 time 734090206895 state 0x1 ref 1 00000100:00100000:1.0:1789760790.859098:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734090467311 time 734104716302 state 0x0 ref 1 00000100:00100000:1.0:1789760790.859109:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb065a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554855680:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760790.859156:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66826240, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fdf0 00010000:00010000:1.0:1789760790.859166:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6853 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.859182:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760790.859186:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.859192:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6853 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66826240->66830335] (req 66826240->66830335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.859205:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6853 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66826240->66830335] (req 66826240->66830335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.865441:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983624 is committed 00002000:00010000:1.0:1789760790.865679:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66826240 -> 66830336 00002000:00010000:1.0:1789760790.865685:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130544 -> 130552 00010000:00010000:1.0:1789760790.865698:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6853 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66826240->66830335] (req 66826240->66830335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.865711:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6853 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66826240->66830335] (req 66826240->66830335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.865725:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6853 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66826240->66830335] (req 66826240->66830335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.865732:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.865737:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6853 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66826240->66830335] (req 66826240->66830335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.865751:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6853 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66826240->66830335] (req 66826240->66830335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.865769:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6853 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66826240->66830335] (req 66826240->66830335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.865777:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6853 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66826240->66830335] (req 66826240->66830335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.865866:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554855680:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6753us (7032us total) trans 4294983624 rc 0/0 00000100:00100000:1.0:1789760790.865876:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16326 00000100:00100000:1.0:1789760790.865885:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734090467311 time 734104716302 state 0x2 ref 0 00000100:00100000:0.0:1789760790.873611:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760790.874570:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554855808 00000100:00100000:1.0:1789760790.874581:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554855808 00000100:00100000:1.0:1789760790.875024:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734090467311 time 734104716302 state 0x0 ref 1 00000100:00100000:1.0:1789760790.875042:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734104976718 time 734104716302 state 0x1 ref 1 00000100:00100000:1.0:1789760790.875081:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734104976718 time 734120699179 state 0x0 ref 1 00000100:00100000:1.0:1789760790.875096:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554855808:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760790.875144:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66830336, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fdf8 00010000:00010000:1.0:1789760790.875153:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c685a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.875167:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760790.875172:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.875178:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c685a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66830336->66834431] (req 66830336->66834431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.875192:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c685a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66830336->66834431] (req 66830336->66834431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.884023:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983625 is committed 00002000:00010000:1.0:1789760790.884143:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66830336 -> 66834432 00002000:00010000:1.0:1789760790.884148:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130552 -> 130560 00010000:00010000:1.0:1789760790.884161:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c685a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66830336->66834431] (req 66830336->66834431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.884173:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c685a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66830336->66834431] (req 66830336->66834431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.884181:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c685a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66830336->66834431] (req 66830336->66834431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.884194:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.884199:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c685a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66830336->66834431] (req 66830336->66834431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.884208:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c685a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66830336->66834431] (req 66830336->66834431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.884226:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c685a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66830336->66834431] (req 66830336->66834431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.884234:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c685a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66830336->66834431] (req 66830336->66834431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.884323:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554855808:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9227us (10719us total) trans 4294983625 rc 0/0 00000100:00100000:1.0:1789760790.884333:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16327 00000100:00100000:1.0:1789760790.884341:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734104976718 time 734120699179 state 0x2 ref 0 00000100:00100000:3.0:1789760790.887719:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760790.888483:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554855936 00000100:00100000:0.0:1789760790.888493:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554855936 00000100:00100000:0.0:1789760790.888548:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734104976718 time 734120699179 state 0x0 ref 1 00000100:00100000:0.0:1789760790.888564:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734120959595 time 734120699179 state 0x1 ref 1 00000100:00100000:0.0:1789760790.888583:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734120959595 time 734134202252 state 0x0 ref 1 00000100:00100000:0.0:1789760790.888595:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb065a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554855936:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760790.888650:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66834432, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fe00 00010000:00010000:0.0:1789760790.888659:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6861 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.888673:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760790.888678:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.888684:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6861 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66834432->66838527] (req 66834432->66838527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.888697:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6861 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66834432->66838527] (req 66834432->66838527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.893214:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983626 is committed 00002000:00010000:0.0:1789760790.893314:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66834432 -> 66838528 00002000:00010000:0.0:1789760790.893319:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130560 -> 130568 00010000:00010000:0.0:1789760790.893333:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6861 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66834432->66838527] (req 66834432->66838527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.893345:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6861 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66834432->66838527] (req 66834432->66838527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.893352:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6861 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66834432->66838527] (req 66834432->66838527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.893359:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760790.893364:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6861 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66834432->66838527] (req 66834432->66838527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.893456:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6861 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66834432->66838527] (req 66834432->66838527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.893475:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6861 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66834432->66838527] (req 66834432->66838527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.893483:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6861 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66834432->66838527] (req 66834432->66838527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.893624:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554855936:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5018us (5901us total) trans 4294983626 rc 0/0 00000100:00100000:0.0:1789760790.893634:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16328 00000100:00100000:0.0:1789760790.893643:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734120959595 time 734134202252 state 0x2 ref 0 00000100:00100000:1.0:1789760790.897713:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760790.897808:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554856064 00000100:00100000:2.0:1789760790.897817:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554856064 00000100:00100000:2.0:1789760790.897870:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734120959595 time 734134202252 state 0x0 ref 1 00000100:00100000:2.0:1789760790.897885:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734134462668 time 734134202252 state 0x1 ref 1 00000100:00100000:2.0:1789760790.897949:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734134462668 time 734143568135 state 0x0 ref 1 00000100:00100000:2.0:1789760790.897965:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554856064:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760790.898020:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66838528, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fe08 00010000:00010000:2.0:1789760790.898029:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6868 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.898046:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760790.898051:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.898057:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6868 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66838528->66842623] (req 66838528->66842623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.898071:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6868 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66838528->66842623] (req 66838528->66842623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.905268:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983627 is committed 00002000:00010000:1.0:1789760790.905519:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66838528 -> 66842624 00002000:00010000:1.0:1789760790.905524:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130568 -> 130576 00010000:00010000:1.0:1789760790.905538:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6868 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66838528->66842623] (req 66838528->66842623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.905551:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6868 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66838528->66842623] (req 66838528->66842623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.905561:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6868 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66838528->66842623] (req 66838528->66842623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.905569:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.905573:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6868 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66838528->66842623] (req 66838528->66842623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.905583:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6868 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66838528->66842623] (req 66838528->66842623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.905601:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6868 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66838528->66842623] (req 66838528->66842623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.905609:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6868 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66838528->66842623] (req 66838528->66842623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.905790:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554856064:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7821us (8081us total) trans 4294983627 rc 0/0 00000100:00100000:1.0:1789760790.905800:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16329 00000100:00100000:1.0:1789760790.905808:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734134462668 time 734143568135 state 0x2 ref 0 00000100:00100000:2.0:1789760790.909363:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760790.909942:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554856192 00000100:00100000:1.0:1789760790.909952:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554856192 00000100:00100000:1.0:1789760790.910019:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734134462668 time 734143568135 state 0x0 ref 1 00000100:00100000:1.0:1789760790.910033:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734143828551 time 734143568135 state 0x1 ref 1 00000100:00100000:1.0:1789760790.910115:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734143828551 time 734155732155 state 0x0 ref 1 00000100:00100000:1.0:1789760790.910128:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09858700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554856192:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760790.910175:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66842624, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fe10 00010000:00010000:1.0:1789760790.910185:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c686f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.910201:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760790.910206:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.910213:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c686f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66842624->66846719] (req 66842624->66846719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.910228:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c686f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66842624->66846719] (req 66842624->66846719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.914671:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983628 is committed 00002000:00010000:1.0:1789760790.914779:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66842624 -> 66846720 00002000:00010000:1.0:1789760790.914785:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130576 -> 130584 00010000:00010000:1.0:1789760790.914799:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c686f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66842624->66846719] (req 66842624->66846719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.914812:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c686f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66842624->66846719] (req 66842624->66846719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.914821:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c686f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66842624->66846719] (req 66842624->66846719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.914833:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.914838:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c686f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66842624->66846719] (req 66842624->66846719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.914847:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c686f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66842624->66846719] (req 66842624->66846719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.914865:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c686f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66842624->66846719] (req 66842624->66846719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.914873:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c686f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66842624->66846719] (req 66842624->66846719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.915116:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09858700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554856192:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4984us (5756us total) trans 4294983628 rc 0/0 00000100:00100000:1.0:1789760790.915125:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16330 00000100:00100000:1.0:1789760790.915134:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734143828551 time 734155732155 state 0x2 ref 0 00000100:00100000:3.0:1789760790.920353:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760790.920436:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554856320 00000100:00100000:1.0:1789760790.920446:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554856320 00000100:00100000:1.0:1789760790.920497:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734143828551 time 734155732155 state 0x0 ref 1 00000100:00100000:1.0:1789760790.920512:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734155992571 time 734155732155 state 0x1 ref 1 00000100:00100000:1.0:1789760790.920544:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734155992571 time 734166162713 state 0x0 ref 1 00000100:00100000:1.0:1789760790.920559:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb065a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554856320:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760790.920602:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66846720, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fe18 00010000:00010000:1.0:1789760790.920610:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6876 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.920635:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760790.920639:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.920645:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6876 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66846720->66850815] (req 66846720->66850815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.920657:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6876 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66846720->66850815] (req 66846720->66850815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760790.929571:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983629 is committed 00000100:00100000:2.0:1789760790.929838:0:3649:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb0985bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3649:1876699573921280:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1789760790.929904:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760790.929971:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573921280 00000100:00100000:2.0:1789760790.929978:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573921280 00000100:00100000:2.0:1789760790.930029:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 393 00000100:00100000:2.0:1789760790.930038:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0985b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3649:x1876699573921280:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1789760790.930068:0:6525:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446383616 avail 1300144128 00000100:00100000:2.0:1789760790.930108:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0985b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3649:x1876699573921280:12345-0@lo:41:osp-pre-1-0.0 Request processed in 71us (204us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760790.930119:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 393 00000100:00100000:3.0:1789760790.930147:0:3649:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb0985bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3649:1876699573921280:0@lo:41:osp-pre-1-0.0 00002000:00010000:1.0:1789760790.932919:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66846720 -> 66850816 00002000:00010000:1.0:1789760790.932928:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130584 -> 130592 00010000:00010000:1.0:1789760790.932939:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6876 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66846720->66850815] (req 66846720->66850815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.932952:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6876 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66846720->66850815] (req 66846720->66850815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.932961:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6876 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66846720->66850815] (req 66846720->66850815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.932968:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.932973:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6876 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66846720->66850815] (req 66846720->66850815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.932981:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6876 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66846720->66850815] (req 66846720->66850815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.933056:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6876 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66846720->66850815] (req 66846720->66850815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.933065:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6876 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66846720->66850815] (req 66846720->66850815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.933218:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb065a0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554856320:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12657us (12869us total) trans 4294983629 rc 0/0 00000100:00100000:1.0:1789760790.933228:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16331 00000100:00100000:1.0:1789760790.933237:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734155992571 time 734166162713 state 0x2 ref 0 00000100:00100000:2.0:1789760790.939188:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760790.939320:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554856448 00000100:00100000:3.0:1789760790.939330:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554856448 00000100:00100000:3.0:1789760790.939414:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734155992571 time 734166162713 state 0x0 ref 1 00000100:00100000:3.0:1789760790.939429:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734166423129 time 734166162713 state 0x1 ref 1 00000100:00100000:3.0:1789760790.939460:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734166423129 time 734185078263 state 0x0 ref 1 00000100:00100000:3.0:1789760790.939471:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09859880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554856448:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760790.939526:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66850816, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fe20 00010000:00010000:3.0:1789760790.939535:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c687d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.939551:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760790.939556:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760790.939562:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c687d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66850816->66854911] (req 66850816->66854911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760790.939576:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c687d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66850816->66854911] (req 66850816->66854911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760790.946205:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983630 is committed 00002000:00010000:0.0:1789760790.946310:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66850816 -> 66854912 00002000:00010000:0.0:1789760790.946315:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130592 -> 130600 00010000:00010000:0.0:1789760790.946328:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c687d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66850816->66854911] (req 66850816->66854911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.946341:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c687d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66850816->66854911] (req 66850816->66854911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.946349:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c687d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66850816->66854911] (req 66850816->66854911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.946361:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760790.946366:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c687d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66850816->66854911] (req 66850816->66854911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.946374:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c687d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66850816->66854911] (req 66850816->66854911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.946460:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c687d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66850816->66854911] (req 66850816->66854911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.946468:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c687d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66850816->66854911] (req 66850816->66854911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.946887:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09859880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554856448:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7228us (7518us total) trans 4294983630 rc 0/0 00000100:00100000:0.0:1789760790.946899:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16332 00000100:00100000:0.0:1789760790.946911:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734166423129 time 734185078263 state 0x2 ref 0 00000100:00100000:2.0:1789760790.950813:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760790.950909:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554856576 00000100:00100000:1.0:1789760790.950920:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554856576 00000100:00100000:1.0:1789760790.950984:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734166423129 time 734185078263 state 0x0 ref 1 00000100:00100000:1.0:1789760790.950999:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734185338679 time 734185078263 state 0x1 ref 1 00000100:00100000:1.0:1789760790.951030:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734185338679 time 734196648750 state 0x0 ref 1 00000100:00100000:1.0:1789760790.951044:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0985aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554856576:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760790.951091:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66854912, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fe28 00010000:00010000:1.0:1789760790.951100:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6884 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.951116:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760790.951120:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760790.951126:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6884 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66854912->66859007] (req 66854912->66859007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.951140:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6884 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66854912->66859007] (req 66854912->66859007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.956879:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983631 is committed 00002000:00010000:1.0:1789760790.957035:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66854912 -> 66859008 00002000:00010000:1.0:1789760790.957041:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130600 -> 130608 00010000:00010000:1.0:1789760790.957054:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6884 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66854912->66859007] (req 66854912->66859007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.957067:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6884 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66854912->66859007] (req 66854912->66859007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.957076:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6884 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66854912->66859007] (req 66854912->66859007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.957084:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.957089:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6884 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66854912->66859007] (req 66854912->66859007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.957097:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6884 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66854912->66859007] (req 66854912->66859007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.957117:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6884 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66854912->66859007] (req 66854912->66859007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.957125:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6884 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66854912->66859007] (req 66854912->66859007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.957265:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0985aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554856576:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6219us (6455us total) trans 4294983631 rc 0/0 00000100:00100000:1.0:1789760790.957276:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16333 00000100:00100000:1.0:1789760790.957286:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734185338679 time 734196648750 state 0x2 ref 0 00000100:00100000:3.0:1789760790.962658:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760790.962991:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554856704 00000100:00100000:2.0:1789760790.963006:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554856704 00000100:00100000:2.0:1789760790.963090:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734185338679 time 734196648750 state 0x0 ref 1 00000100:00100000:2.0:1789760790.963107:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734196909166 time 734196648750 state 0x1 ref 1 00000100:00100000:2.0:1789760790.963141:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734196909166 time 734208759275 state 0x0 ref 1 00000100:00100000:2.0:1789760790.963153:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554856704:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760790.963195:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66859008, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fe30 00010000:00010000:2.0:1789760790.963204:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c688b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.963219:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760790.963223:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760790.963228:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c688b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66859008->66863103] (req 66859008->66863103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760790.963242:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c688b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66859008->66863103] (req 66859008->66863103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.969422:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983632 is committed 00002000:00010000:1.0:1789760790.969543:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66859008 -> 66863104 00002000:00010000:1.0:1789760790.969549:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130608 -> 130616 00010000:00010000:1.0:1789760790.969561:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c688b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66859008->66863103] (req 66859008->66863103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.969574:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c688b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66859008->66863103] (req 66859008->66863103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.969670:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c688b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66859008->66863103] (req 66859008->66863103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.969747:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760790.969753:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c688b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66859008->66863103] (req 66859008->66863103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.969768:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c688b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66859008->66863103] (req 66859008->66863103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.969788:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c688b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66859008->66863103] (req 66859008->66863103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760790.969797:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c688b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66859008->66863103] (req 66859008->66863103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760790.969904:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554856704:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6748us (7251us total) trans 4294983632 rc 0/0 00000100:00100000:1.0:1789760790.969915:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16334 00000100:00100000:1.0:1789760790.969924:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734196909166 time 734208759275 state 0x2 ref 0 00000100:00100000:2.0:1789760790.975241:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760790.976201:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554856832 00000100:00100000:0.0:1789760790.976213:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554856832 00000100:00100000:0.0:1789760790.976267:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734196909166 time 734208759275 state 0x0 ref 1 00000100:00100000:0.0:1789760790.976281:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734209019691 time 734208759275 state 0x1 ref 1 00000100:00100000:0.0:1789760790.976303:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734209019691 time 734221922248 state 0x0 ref 1 00000100:00100000:0.0:1789760790.976319:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09858a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554856832:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760790.976366:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66863104, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fe38 00010000:00010000:0.0:1789760790.976548:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6892 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.976566:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760790.976571:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.976577:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6892 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66863104->66867199] (req 66863104->66867199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.976591:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6892 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66863104->66867199] (req 66863104->66867199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.985072:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983633 is committed 00002000:00010000:0.0:1789760790.985261:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66863104 -> 66867200 00002000:00010000:0.0:1789760790.985267:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130616 -> 130624 00010000:00010000:0.0:1789760790.985280:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6892 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66863104->66867199] (req 66863104->66867199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.985294:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6892 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66863104->66867199] (req 66863104->66867199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.985303:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6892 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66863104->66867199] (req 66863104->66867199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.985311:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760790.985316:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6892 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66863104->66867199] (req 66863104->66867199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.985325:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6892 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66863104->66867199] (req 66863104->66867199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.985344:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6892 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66863104->66867199] (req 66863104->66867199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.985352:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6892 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66863104->66867199] (req 66863104->66867199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.985695:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09858a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554856832:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9371us (10456us total) trans 4294983633 rc 0/0 00000100:00100000:0.0:1789760790.985707:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16335 00000100:00100000:0.0:1789760790.985717:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734209019691 time 734221922248 state 0x2 ref 0 00000100:00100000:2.0:1789760790.990624:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760790.990693:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554856960 00000100:00100000:0.0:1789760790.990702:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554856960 00000100:00100000:0.0:1789760790.990763:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734209019691 time 734221922248 state 0x0 ref 1 00000100:00100000:0.0:1789760790.990778:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734222182664 time 734221922248 state 0x1 ref 1 00000100:00100000:0.0:1789760790.990805:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734222182664 time 734236423942 state 0x0 ref 1 00000100:00100000:0.0:1789760790.990816:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09859c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554856960:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760790.990856:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66867200, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fe40 00010000:00010000:0.0:1789760790.990865:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6899 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.990878:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760790.990882:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760790.990888:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6899 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66867200->66871295] (req 66867200->66871295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.990900:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6899 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66867200->66871295] (req 66867200->66871295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760790.996271:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983634 is committed 00002000:00010000:0.0:1789760790.996508:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66867200 -> 66871296 00002000:00010000:0.0:1789760790.996513:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130624 -> 130632 00010000:00010000:0.0:1789760790.996525:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6899 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66867200->66871295] (req 66867200->66871295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.996537:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6899 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66867200->66871295] (req 66867200->66871295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.996546:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6899 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66867200->66871295] (req 66867200->66871295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.996553:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760790.996558:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6899 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66867200->66871295] (req 66867200->66871295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.996567:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6899 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66867200->66871295] (req 66867200->66871295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.996585:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6899 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66867200->66871295] (req 66867200->66871295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760790.996593:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6899 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66867200->66871295] (req 66867200->66871295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760790.996710:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09859c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554856960:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5891us (6098us total) trans 4294983634 rc 0/0 00000100:00100000:0.0:1789760790.996719:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16336 00000100:00100000:0.0:1789760790.996728:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734222182664 time 734236423942 state 0x2 ref 0 00000100:00100000:3.0:1789760790.999575:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760791.000070:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554857088 00000100:00100000:1.0:1789760791.000080:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554857088 00000100:00100000:1.0:1789760791.000131:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734222182664 time 734236423942 state 0x0 ref 1 00000100:00100000:1.0:1789760791.000146:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734236684358 time 734236423942 state 0x1 ref 1 00000100:00100000:1.0:1789760791.000205:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734236684358 time 734245822971 state 0x0 ref 1 00000100:00100000:1.0:1789760791.000217:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554857088:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760791.000261:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66871296, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fe48 00010000:00010000:1.0:1789760791.000270:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68a0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.000285:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760791.000289:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.000295:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66871296->66875391] (req 66871296->66875391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.000307:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66871296->66875391] (req 66871296->66875391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.009792:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983635 is committed 00002000:00010000:1.0:1789760791.009922:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66871296 -> 66875392 00002000:00010000:1.0:1789760791.009928:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130632 -> 130640 00010000:00010000:1.0:1789760791.009940:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68a0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66871296->66875391] (req 66871296->66875391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.009952:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68a0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66871296->66875391] (req 66871296->66875391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.009969:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66871296->66875391] (req 66871296->66875391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.009977:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760791.009982:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68a0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66871296->66875391] (req 66871296->66875391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.009998:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68a0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66871296->66875391] (req 66871296->66875391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.010022:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68a0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66871296->66875391] (req 66871296->66875391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.010030:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68a0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66871296->66875391] (req 66871296->66875391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.010161:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554857088:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9942us (10592us total) trans 4294983635 rc 0/0 00000100:00100000:1.0:1789760791.010171:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16337 00000100:00100000:1.0:1789760791.010180:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734236684358 time 734245822971 state 0x2 ref 0 00000100:00100000:2.0:1789760791.014121:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760791.017115:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554857216 00000100:00100000:1.0:1789760791.017128:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554857216 00000100:00100000:1.0:1789760791.017369:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734236684358 time 734245822971 state 0x0 ref 1 00000100:00100000:1.0:1789760791.017409:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734246083387 time 734245822971 state 0x1 ref 1 00000100:00100000:1.0:1789760791.017433:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734246083387 time 734263051980 state 0x0 ref 1 00000100:00100000:1.0:1789760791.017449:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09859180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554857216:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760791.017495:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66875392, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fe50 00010000:00010000:1.0:1789760791.017504:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c68a7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.017520:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760791.017525:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.017531:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c68a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66875392->66879487] (req 66875392->66879487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.017545:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c68a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66875392->66879487] (req 66875392->66879487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760791.018341:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760791.024581:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983636 is committed 00002000:00010000:2.0:1789760791.024906:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66875392 -> 66879488 00002000:00010000:2.0:1789760791.024912:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130640 -> 130648 00010000:00010000:2.0:1789760791.024925:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c68a7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66875392->66879487] (req 66875392->66879487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.024939:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c68a7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66875392->66879487] (req 66875392->66879487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.024950:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c68a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66875392->66879487] (req 66875392->66879487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.024957:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760791.024962:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c68a7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66875392->66879487] (req 66875392->66879487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.024970:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c68a7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66875392->66879487] (req 66875392->66879487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.024988:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c68a7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66875392->66879487] (req 66875392->66879487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.024996:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c68a7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66875392->66879487] (req 66875392->66879487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.025154:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09859180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554857216:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7697us (11035us total) trans 4294983636 rc 0/0 00000100:00100000:2.0:1789760791.025166:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16338 00000100:00100000:2.0:1789760791.025176:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734246083387 time 734263051980 state 0x2 ref 0 00000100:00100000:2.0:1789760791.029055:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760791.029262:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554857344 00000100:00100000:0.0:1789760791.029271:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554857344 00000100:00100000:0.0:1789760791.029321:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734246083387 time 734263051980 state 0x0 ref 1 00000100:00100000:0.0:1789760791.029336:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734263312396 time 734263051980 state 0x1 ref 1 00000100:00100000:0.0:1789760791.029448:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734263312396 time 734275065001 state 0x0 ref 1 00000100:00100000:0.0:1789760791.029461:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0985b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554857344:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760791.029508:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66879488, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fe58 00010000:00010000:0.0:1789760791.029517:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68ae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.029537:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760791.029542:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.029548:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66879488->66883583] (req 66879488->66883583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.029561:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66879488->66883583] (req 66879488->66883583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.035308:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983637 is committed 00002000:00010000:0.0:1789760791.035443:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66879488 -> 66883584 00002000:00010000:0.0:1789760791.035448:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130648 -> 130656 00010000:00010000:0.0:1789760791.035462:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68ae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66879488->66883583] (req 66879488->66883583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.035475:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68ae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66879488->66883583] (req 66879488->66883583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.035489:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66879488->66883583] (req 66879488->66883583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.035497:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760791.035501:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68ae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66879488->66883583] (req 66879488->66883583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.035509:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68ae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66879488->66883583] (req 66879488->66883583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.035533:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68ae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66879488->66883583] (req 66879488->66883583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.035540:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68ae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66879488->66883583] (req 66879488->66883583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.035672:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0985b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554857344:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6207us (6674us total) trans 4294983637 rc 0/0 00000100:00100000:0.0:1789760791.035682:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16339 00000100:00100000:0.0:1789760791.035691:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734263312396 time 734275065001 state 0x2 ref 0 00000100:00100000:2.0:1789760791.041598:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760791.041816:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554857472 00000100:00100000:3.0:1789760791.041826:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554857472 00000100:00100000:3.0:1789760791.041881:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734263312396 time 734275065001 state 0x0 ref 1 00000100:00100000:3.0:1789760791.041898:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734275325417 time 734275065001 state 0x1 ref 1 00000100:00100000:3.0:1789760791.041936:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734275325417 time 734287554022 state 0x0 ref 1 00000100:00100000:3.0:1789760791.041948:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09859500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554857472:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760791.042009:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66883584, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fe60 00010000:00010000:3.0:1789760791.042018:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c68b5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.042035:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760791.042040:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.042047:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c68b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66883584->66887679] (req 66883584->66887679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.042062:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c68b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66883584->66887679] (req 66883584->66887679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.047254:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983638 is committed 00002000:00010000:3.0:1789760791.047372:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66883584 -> 66887680 00002000:00010000:3.0:1789760791.047402:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130656 -> 130664 00010000:00010000:3.0:1789760791.047416:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c68b5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66883584->66887679] (req 66883584->66887679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.047429:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c68b5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66883584->66887679] (req 66883584->66887679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.047437:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c68b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66883584->66887679] (req 66883584->66887679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.047450:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760791.047455:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c68b5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66883584->66887679] (req 66883584->66887679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.047463:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c68b5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66883584->66887679] (req 66883584->66887679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.047480:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c68b5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66883584->66887679] (req 66883584->66887679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.047487:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c68b5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66883584->66887679] (req 66883584->66887679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.047575:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09859500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554857472:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5624us (5982us total) trans 4294983638 rc 0/0 00000100:00100000:3.0:1789760791.047586:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16340 00000100:00100000:3.0:1789760791.047594:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734275325417 time 734287554022 state 0x2 ref 0 00000100:00100000:1.0:1789760791.053258:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760791.053337:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554857600 00000100:00100000:3.0:1789760791.053346:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554857600 00000100:00100000:3.0:1789760791.053537:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734275325417 time 734287554022 state 0x0 ref 1 00000100:00100000:3.0:1789760791.053552:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734287814438 time 734287554022 state 0x1 ref 1 00000100:00100000:3.0:1789760791.053626:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734287814438 time 734299244461 state 0x0 ref 1 00000100:00100000:3.0:1789760791.053642:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554857600:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760791.053692:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66887680, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fe68 00010000:00010000:3.0:1789760791.053700:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68bc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.053715:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760791.053719:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.053725:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66887680->66891775] (req 66887680->66891775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.053739:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66887680->66891775] (req 66887680->66891775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.061132:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983639 is committed 00002000:00010000:3.0:1789760791.061290:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66887680 -> 66891776 00002000:00010000:3.0:1789760791.061296:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130664 -> 130672 00010000:00010000:3.0:1789760791.061309:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68bc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66887680->66891775] (req 66887680->66891775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.061323:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68bc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66887680->66891775] (req 66887680->66891775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.061332:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66887680->66891775] (req 66887680->66891775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.061339:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760791.061345:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68bc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66887680->66891775] (req 66887680->66891775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.061354:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68bc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66887680->66891775] (req 66887680->66891775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.061505:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68bc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66887680->66891775] (req 66887680->66891775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.061514:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68bc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66887680->66891775] (req 66887680->66891775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.061657:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554857600:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8014us (8404us total) trans 4294983639 rc 0/0 00000100:00100000:3.0:1789760791.061667:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16341 00000100:00100000:3.0:1789760791.061677:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734287814438 time 734299244461 state 0x2 ref 0 00000100:00100000:2.0:1789760791.068316:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760791.068463:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554857728 00000100:00100000:1.0:1789760791.068474:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554857728 00000100:00100000:1.0:1789760791.068526:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734287814438 time 734299244461 state 0x0 ref 1 00000100:00100000:1.0:1789760791.068542:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734299504877 time 734299244461 state 0x1 ref 1 00000100:00100000:1.0:1789760791.068570:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734299504877 time 734314189246 state 0x0 ref 1 00000100:00100000:1.0:1789760791.068582:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09858e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554857728:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760791.068638:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66891776, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fe70 00010000:00010000:1.0:1789760791.068647:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68c3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.068662:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760791.068667:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.068673:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66891776->66895871] (req 66891776->66895871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.068692:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66891776->66895871] (req 66891776->66895871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.074721:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983640 is committed 00002000:00010000:1.0:1789760791.074832:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66891776 -> 66895872 00002000:00010000:1.0:1789760791.074837:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130672 -> 130680 00010000:00010000:1.0:1789760791.074848:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68c3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66891776->66895871] (req 66891776->66895871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.074861:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68c3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66891776->66895871] (req 66891776->66895871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.074870:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66891776->66895871] (req 66891776->66895871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.074877:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760791.074881:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68c3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66891776->66895871] (req 66891776->66895871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.074890:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68c3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66891776->66895871] (req 66891776->66895871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.074909:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68c3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66891776->66895871] (req 66891776->66895871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.074918:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68c3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66891776->66895871] (req 66891776->66895871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.075014:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09858e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554857728:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6429us (6703us total) trans 4294983640 rc 0/0 00000100:00100000:1.0:1789760791.075025:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16342 00000100:00100000:1.0:1789760791.075034:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734299504877 time 734314189246 state 0x2 ref 0 00000100:00100000:2.0:1789760791.084482:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760791.084770:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554857856 00000100:00100000:0.0:1789760791.084779:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554857856 00000100:00100000:0.0:1789760791.084830:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734299504877 time 734314189246 state 0x0 ref 1 00000100:00100000:0.0:1789760791.084845:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734314449662 time 734314189246 state 0x1 ref 1 00000100:00100000:0.0:1789760791.084874:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734314449662 time 734330493003 state 0x0 ref 1 00000100:00100000:0.0:1789760791.084887:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0985b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554857856:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760791.084990:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66895872, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fe78 00010000:00010000:0.0:1789760791.085000:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68ca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.085126:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760791.085132:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.085139:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66895872->66899967] (req 66895872->66899967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.085154:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66895872->66899967] (req 66895872->66899967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.091722:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983641 is committed 00002000:00010000:0.0:1789760791.091941:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66895872 -> 66899968 00002000:00010000:0.0:1789760791.091946:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130680 -> 130688 00010000:00010000:0.0:1789760791.091958:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68ca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66895872->66899967] (req 66895872->66899967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.091970:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68ca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66895872->66899967] (req 66895872->66899967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.091979:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66895872->66899967] (req 66895872->66899967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.091987:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760791.092041:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68ca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66895872->66899967] (req 66895872->66899967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.092051:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68ca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66895872->66899967] (req 66895872->66899967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.092070:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68ca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66895872->66899967] (req 66895872->66899967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.092083:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68ca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66895872->66899967] (req 66895872->66899967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.092588:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0985b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554857856:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7698us (8111us total) trans 4294983641 rc 0/0 00000100:00100000:0.0:1789760791.092599:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16343 00000100:00100000:0.0:1789760791.092608:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734314449662 time 734330493003 state 0x2 ref 0 00000100:00100000:2.0:1789760791.097665:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760791.097737:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554857984 00000100:00100000:3.0:1789760791.097747:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554857984 00000100:00100000:3.0:1789760791.097796:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734314449662 time 734330493003 state 0x0 ref 1 00000100:00100000:3.0:1789760791.097811:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734330753419 time 734330493003 state 0x1 ref 1 00000100:00100000:3.0:1789760791.097846:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734330753419 time 734343464111 state 0x0 ref 1 00000100:00100000:3.0:1789760791.097858:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0985ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554857984:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760791.097902:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66899968, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fe80 00010000:00010000:3.0:1789760791.097912:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68d1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.097927:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760791.097944:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.097950:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66899968->66904063] (req 66899968->66904063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.097964:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66899968->66904063] (req 66899968->66904063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.102065:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983642 is committed 00002000:00010000:3.0:1789760791.102170:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66899968 -> 66904064 00002000:00010000:3.0:1789760791.102173:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130688 -> 130696 00010000:00010000:3.0:1789760791.102183:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68d1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66899968->66904063] (req 66899968->66904063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.102194:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68d1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66899968->66904063] (req 66899968->66904063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.102203:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66899968->66904063] (req 66899968->66904063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.102210:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760791.102214:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68d1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66899968->66904063] (req 66899968->66904063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.102222:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68d1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66899968->66904063] (req 66899968->66904063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.102242:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68d1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66899968->66904063] (req 66899968->66904063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.102251:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68d1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66899968->66904063] (req 66899968->66904063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.102422:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0985ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554857984:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4561us (4761us total) trans 4294983642 rc 0/0 00000100:00100000:3.0:1789760791.102432:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16344 00000100:00100000:3.0:1789760791.102441:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734330753419 time 734343464111 state 0x2 ref 0 00000100:00100000:2.0:1789760791.105968:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760791.107514:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554858112 00000100:00100000:3.0:1789760791.107527:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554858112 00000100:00100000:3.0:1789760791.107583:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734330753419 time 734343464111 state 0x0 ref 1 00000100:00100000:3.0:1789760791.107597:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734343724527 time 734343464111 state 0x1 ref 1 00000100:00100000:0.0:1789760791.107711:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734343724527 time 734353327377 state 0x0 ref 1 00000100:00100000:0.0:1789760791.107725:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09858000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554858112:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760791.107773:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66904064, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fe88 00010000:00010000:0.0:1789760791.107782:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68d8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.107797:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760791.107802:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.107808:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66904064->66908159] (req 66904064->66908159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.107822:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66904064->66908159] (req 66904064->66908159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.112443:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983643 is committed 00002000:00010000:0.0:1789760791.112531:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66904064 -> 66908160 00002000:00010000:0.0:1789760791.112536:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130696 -> 130704 00010000:00010000:0.0:1789760791.112549:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68d8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66904064->66908159] (req 66904064->66908159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.112561:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68d8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66904064->66908159] (req 66904064->66908159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.112570:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66904064->66908159] (req 66904064->66908159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.112577:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760791.112583:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68d8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66904064->66908159] (req 66904064->66908159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.112591:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68d8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66904064->66908159] (req 66904064->66908159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.112609:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68d8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66904064->66908159] (req 66904064->66908159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.112632:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68d8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66904064->66908159] (req 66904064->66908159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.112776:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09858000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554858112:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5050us (6812us total) trans 4294983643 rc 0/0 00000100:00100000:0.0:1789760791.112787:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16345 00000100:00100000:0.0:1789760791.112795:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734343724527 time 734353327377 state 0x2 ref 0 00000100:00100000:2.0:1789760791.118782:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760791.120674:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554858240 00000100:00100000:1.0:1789760791.120740:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554858240 00000100:00100000:1.0:1789760791.120801:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734343724527 time 734353327377 state 0x0 ref 1 00000100:00100000:1.0:1789760791.120817:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734353587793 time 734353327377 state 0x1 ref 1 00000100:00100000:1.0:1789760791.120908:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734353587793 time 734366525880 state 0x0 ref 1 00000100:00100000:1.0:1789760791.120922:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0985bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554858240:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760791.120968:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66908160, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fe90 00010000:00010000:1.0:1789760791.120978:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68df lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.121010:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760791.121016:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.121022:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66908160->66912255] (req 66908160->66912255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.121037:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66908160->66912255] (req 66908160->66912255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.121968:0:6509:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789760791.124652:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983644 is committed 00002000:00010000:1.0:1789760791.124826:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66908160 -> 66912256 00002000:00010000:1.0:1789760791.125017:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130704 -> 130712 00010000:00010000:1.0:1789760791.125031:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68df lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66908160->66912255] (req 66908160->66912255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.125045:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68df lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66908160->66912255] (req 66908160->66912255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.125054:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66908160->66912255] (req 66908160->66912255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.125062:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760791.125067:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68df lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66908160->66912255] (req 66908160->66912255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.125077:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68df lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66908160->66912255] (req 66908160->66912255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.125096:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68df lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66908160->66912255] (req 66908160->66912255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.125104:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68df lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66908160->66912255] (req 66908160->66912255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.125304:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0985bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554858240:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4380us (6525us total) trans 4294983644 rc 0/0 00000100:00100000:1.0:1789760791.125315:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16346 00000100:00100000:1.0:1789760791.125326:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734353587793 time 734366525880 state 0x2 ref 0 00000100:00100000:2.0:1789760791.129327:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760791.129573:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554858368 00000100:00100000:3.0:1789760791.129582:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554858368 00000100:00100000:3.0:1789760791.129643:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734353587793 time 734366525880 state 0x0 ref 1 00000100:00100000:3.0:1789760791.129658:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734366786296 time 734366525880 state 0x1 ref 1 00000100:00100000:3.0:1789760791.129689:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734366786296 time 734375307585 state 0x0 ref 1 00000100:00100000:3.0:1789760791.129701:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0985a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554858368:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760791.129756:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66912256, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fe98 00010000:00010000:3.0:1789760791.129764:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68e6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.129779:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760791.129783:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.129790:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66912256->66916351] (req 66912256->66916351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.129803:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66912256->66916351] (req 66912256->66916351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.134781:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983645 is committed 00002000:00010000:1.0:1789760791.134966:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66912256 -> 66916352 00002000:00010000:1.0:1789760791.134971:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130712 -> 130720 00010000:00010000:1.0:1789760791.134984:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68e6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66912256->66916351] (req 66912256->66916351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.135007:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68e6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66912256->66916351] (req 66912256->66916351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.135016:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66912256->66916351] (req 66912256->66916351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.135024:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760791.135028:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68e6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66912256->66916351] (req 66912256->66916351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.135038:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68e6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66912256->66916351] (req 66912256->66916351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.135058:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68e6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66912256->66916351] (req 66912256->66916351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.135066:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c68e6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66912256->66916351] (req 66912256->66916351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.135161:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0985a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554858368:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5457us (5838us total) trans 4294983645 rc 0/0 00000100:00100000:1.0:1789760791.135171:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16347 00000100:00100000:1.0:1789760791.135180:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734366786296 time 734375307585 state 0x2 ref 0 00000100:00100000:2.0:1789760791.139797:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760791.139878:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554858496 00000100:00100000:3.0:1789760791.139889:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554858496 00000100:00100000:3.0:1789760791.139942:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734366786296 time 734375307585 state 0x0 ref 1 00000100:00100000:3.0:1789760791.139957:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734375568001 time 734375307585 state 0x1 ref 1 00000100:00100000:3.0:1789760791.140028:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734375568001 time 734385645479 state 0x0 ref 1 00000100:00100000:3.0:1789760791.140041:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554858496:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760791.140090:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66916352, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fea0 00010000:00010000:3.0:1789760791.140100:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c68ed lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.140130:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760791.140142:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.140149:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c68ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66916352->66920447] (req 66916352->66920447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.140163:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c68ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66916352->66920447] (req 66916352->66920447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.145470:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983646 is committed 00002000:00010000:3.0:1789760791.146451:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66916352 -> 66920448 00002000:00010000:3.0:1789760791.146458:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130720 -> 130728 00010000:00010000:3.0:1789760791.146471:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c68ed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66916352->66920447] (req 66916352->66920447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.146484:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c68ed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66916352->66920447] (req 66916352->66920447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.146493:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c68ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66916352->66920447] (req 66916352->66920447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.146500:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760791.146505:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c68ed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66916352->66920447] (req 66916352->66920447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.146513:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c68ed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66916352->66920447] (req 66916352->66920447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.146537:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c68ed lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66916352->66920447] (req 66916352->66920447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.146544:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c68ed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66916352->66920447] (req 66916352->66920447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.146741:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554858496:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6696us (6948us total) trans 4294983646 rc 0/0 00000100:00100000:3.0:1789760791.146752:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16348 00000100:00100000:3.0:1789760791.146761:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734375568001 time 734385645479 state 0x2 ref 0 00000100:00100000:2.0:1789760791.150186:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760791.150249:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554858624 00000100:00100000:0.0:1789760791.150267:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554858624 00000100:00100000:0.0:1789760791.150317:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734375568001 time 734385645479 state 0x0 ref 1 00000100:00100000:0.0:1789760791.150332:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734385905895 time 734385645479 state 0x1 ref 1 00000100:00100000:0.0:1789760791.150365:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734385905895 time 734395983188 state 0x0 ref 1 00000100:00100000:0.0:1789760791.150658:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554858624:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760791.150702:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66920448, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fea8 00010000:00010000:0.0:1789760791.150711:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68f4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.150728:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760791.150732:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.150738:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66920448->66924543] (req 66920448->66924543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.150751:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66920448->66924543] (req 66920448->66924543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.154828:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983647 is committed 00002000:00010000:0.0:1789760791.154977:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66920448 -> 66924544 00002000:00010000:0.0:1789760791.154982:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130728 -> 130736 00010000:00010000:0.0:1789760791.155003:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68f4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66920448->66924543] (req 66920448->66924543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.155022:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68f4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66920448->66924543] (req 66920448->66924543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.155030:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66920448->66924543] (req 66920448->66924543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.155037:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760791.155042:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68f4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66920448->66924543] (req 66920448->66924543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.155051:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68f4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66920448->66924543] (req 66920448->66924543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.155071:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68f4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66920448->66924543] (req 66920448->66924543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.155079:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c68f4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66920448->66924543] (req 66920448->66924543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.155167:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554858624:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4510us (4986us total) trans 4294983647 rc 0/0 00000100:00100000:0.0:1789760791.155177:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16349 00000100:00100000:0.0:1789760791.155185:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734385905895 time 734395983188 state 0x2 ref 0 00000100:00100000:2.0:1789760791.164041:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760791.164212:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554858752 00000100:00100000:1.0:1789760791.164221:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554858752 00000100:00100000:1.0:1789760791.164275:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734385905895 time 734395983188 state 0x0 ref 1 00000100:00100000:1.0:1789760791.164290:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734396243604 time 734395983188 state 0x1 ref 1 00000100:00100000:1.0:1789760791.164319:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734396243604 time 734409938593 state 0x0 ref 1 00000100:00100000:1.0:1789760791.164330:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554858752:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760791.164374:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66924544, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1feb0 00010000:00010000:1.0:1789760791.165553:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68fb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.165571:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760791.165576:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.165582:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66924544->66928639] (req 66924544->66928639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.165597:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66924544->66928639] (req 66924544->66928639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.172438:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983648 is committed 00002000:00010000:1.0:1789760791.172852:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66924544 -> 66928640 00002000:00010000:1.0:1789760791.172858:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130736 -> 130744 00010000:00010000:1.0:1789760791.172882:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68fb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66924544->66928639] (req 66924544->66928639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.172899:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68fb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66924544->66928639] (req 66924544->66928639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.172910:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66924544->66928639] (req 66924544->66928639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.172918:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760791.172924:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68fb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66924544->66928639] (req 66924544->66928639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.172934:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68fb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66924544->66928639] (req 66924544->66928639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.172955:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68fb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66924544->66928639] (req 66924544->66928639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.172963:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c68fb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66924544->66928639] (req 66924544->66928639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.173103:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554858752:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8770us (9067us total) trans 4294983648 rc 0/0 00000100:00100000:1.0:1789760791.173114:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16350 00000100:00100000:1.0:1789760791.173123:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734396243604 time 734409938593 state 0x2 ref 0 00000100:00100000:2.0:1789760791.180963:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760791.181007:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554858880 00000100:00100000:1.0:1789760791.181018:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554858880 00000100:00100000:1.0:1789760791.181074:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734396243604 time 734409938593 state 0x0 ref 1 00000100:00100000:1.0:1789760791.181089:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734410199009 time 734409938593 state 0x1 ref 1 00000100:00100000:1.0:1789760791.181122:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734410199009 time 734426740228 state 0x0 ref 1 00000100:00100000:1.0:1789760791.181135:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554858880:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760791.181188:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66928640, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1feb8 00010000:00010000:1.0:1789760791.181198:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6902 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.181215:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760791.181220:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.181226:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6902 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66928640->66932735] (req 66928640->66932735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.181241:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6902 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66928640->66932735] (req 66928640->66932735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.188156:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983649 is committed 00002000:00010000:0.0:1789760791.190735:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66928640 -> 66932736 00002000:00010000:0.0:1789760791.190743:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130744 -> 130752 00010000:00010000:0.0:1789760791.190756:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6902 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66928640->66932735] (req 66928640->66932735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.190769:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6902 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66928640->66932735] (req 66928640->66932735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.190777:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6902 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66928640->66932735] (req 66928640->66932735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.190784:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760791.190789:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6902 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66928640->66932735] (req 66928640->66932735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.190798:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6902 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66928640->66932735] (req 66928640->66932735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.190818:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6902 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66928640->66932735] (req 66928640->66932735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.190825:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6902 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66928640->66932735] (req 66928640->66932735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.190919:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554858880:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9782us (9962us total) trans 4294983649 rc 0/0 00000100:00100000:0.0:1789760791.190929:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16351 00000100:00100000:0.0:1789760791.190938:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734410199009 time 734426740228 state 0x2 ref 0 00000100:00100000:2.0:1789760791.196815:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760791.196920:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554859008 00000100:00100000:3.0:1789760791.196930:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554859008 00000100:00100000:3.0:1789760791.196981:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734410199009 time 734426740228 state 0x0 ref 1 00000100:00100000:3.0:1789760791.197008:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734427000644 time 734426740228 state 0x1 ref 1 00000100:00100000:3.0:1789760791.197037:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734427000644 time 734442656182 state 0x0 ref 1 00000100:00100000:3.0:1789760791.197048:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554859008:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760791.197090:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66932736, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fec0 00010000:00010000:3.0:1789760791.197098:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6909 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.197118:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760791.197122:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.197127:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6909 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66932736->66936831] (req 66932736->66936831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.197141:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6909 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66932736->66936831] (req 66932736->66936831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.201496:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983650 is committed 00002000:00010000:3.0:1789760791.205209:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66932736 -> 66936832 00002000:00010000:3.0:1789760791.205219:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130752 -> 130760 00010000:00010000:3.0:1789760791.205233:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6909 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66932736->66936831] (req 66932736->66936831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.205246:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6909 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66932736->66936831] (req 66932736->66936831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.205266:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6909 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66932736->66936831] (req 66932736->66936831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.205274:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760791.205279:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6909 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66932736->66936831] (req 66932736->66936831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.205288:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6909 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66932736->66936831] (req 66932736->66936831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.205314:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6909 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66932736->66936831] (req 66932736->66936831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.205323:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6909 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66932736->66936831] (req 66932736->66936831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.205512:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554859008:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8460us (8701us total) trans 4294983650 rc 0/0 00000100:00100000:3.0:1789760791.205522:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16352 00000100:00100000:3.0:1789760791.205531:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734427000644 time 734442656182 state 0x2 ref 0 00000100:00100000:2.0:1789760791.212492:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760791.212572:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554859136 00000100:00100000:0.0:1789760791.212581:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554859136 00000100:00100000:0.0:1789760791.212642:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734427000644 time 734442656182 state 0x0 ref 1 00000100:00100000:0.0:1789760791.212657:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734442916598 time 734442656182 state 0x1 ref 1 00000100:00100000:0.0:1789760791.212689:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734442916598 time 734458307434 state 0x0 ref 1 00000100:00100000:0.0:1789760791.212702:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554859136:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760791.212747:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66936832, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fec8 00010000:00010000:0.0:1789760791.212757:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6910 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.212773:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760791.212778:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.212785:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6910 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66936832->66940927] (req 66936832->66940927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.212800:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6910 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66936832->66940927] (req 66936832->66940927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.216664:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983651 is committed 00002000:00010000:0.0:1789760791.216782:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66936832 -> 66940928 00002000:00010000:0.0:1789760791.216787:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130760 -> 130768 00010000:00010000:0.0:1789760791.216802:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6910 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66936832->66940927] (req 66936832->66940927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.216821:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6910 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66936832->66940927] (req 66936832->66940927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.216831:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6910 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66936832->66940927] (req 66936832->66940927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.216838:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760791.216843:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6910 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66936832->66940927] (req 66936832->66940927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.216852:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6910 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66936832->66940927] (req 66936832->66940927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.216873:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6910 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66936832->66940927] (req 66936832->66940927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.216881:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6910 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66936832->66940927] (req 66936832->66940927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.216979:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554859136:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4275us (4491us total) trans 4294983651 rc 0/0 00000100:00100000:0.0:1789760791.217630:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16353 00000100:00100000:0.0:1789760791.217641:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734442916598 time 734458307434 state 0x2 ref 0 00000100:00100000:2.0:1789760791.224776:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760791.224866:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554859264 00000100:00100000:1.0:1789760791.224876:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554859264 00000100:00100000:1.0:1789760791.224934:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734442916598 time 734458307434 state 0x0 ref 1 00000100:00100000:1.0:1789760791.224949:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734458567850 time 734458307434 state 0x1 ref 1 00000100:00100000:1.0:1789760791.224977:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734458567850 time 734470596319 state 0x0 ref 1 00000100:00100000:1.0:1789760791.224998:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554859264:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760791.225046:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66940928, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fed0 00010000:00010000:1.0:1789760791.225056:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6917 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.225071:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760791.225078:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.225084:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6917 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66940928->66945023] (req 66940928->66945023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.225097:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6917 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66940928->66945023] (req 66940928->66945023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.229760:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983652 is committed 00002000:00010000:1.0:1789760791.229879:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66940928 -> 66945024 00002000:00010000:1.0:1789760791.229885:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130768 -> 130776 00010000:00010000:1.0:1789760791.229897:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6917 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66940928->66945023] (req 66940928->66945023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.229910:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6917 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66940928->66945023] (req 66940928->66945023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.229925:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6917 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66940928->66945023] (req 66940928->66945023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.229933:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760791.229938:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6917 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66940928->66945023] (req 66940928->66945023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.229947:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6917 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66940928->66945023] (req 66940928->66945023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.229965:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6917 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66940928->66945023] (req 66940928->66945023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.229973:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6917 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66940928->66945023] (req 66940928->66945023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.230073:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554859264:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5080us (5302us total) trans 4294983652 rc 0/0 00000100:00100000:1.0:1789760791.230083:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16354 00000100:00100000:1.0:1789760791.230092:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734458567850 time 734470596319 state 0x2 ref 0 00000100:00100000:2.0:1789760791.234731:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760791.234807:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554859392 00000100:00100000:3.0:1789760791.234817:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554859392 00000100:00100000:3.0:1789760791.234869:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734458567850 time 734470596319 state 0x0 ref 1 00000100:00100000:3.0:1789760791.234883:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734470856735 time 734470596319 state 0x1 ref 1 00000100:00100000:3.0:1789760791.234915:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734470856735 time 734480533329 state 0x0 ref 1 00000100:00100000:3.0:1789760791.234927:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554859392:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760791.234974:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66945024, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fed8 00010000:00010000:3.0:1789760791.234983:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c691e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.235003:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760791.235007:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.235013:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c691e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66945024->66949119] (req 66945024->66949119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.235026:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c691e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66945024->66949119] (req 66945024->66949119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.239487:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983653 is committed 00002000:00010000:3.0:1789760791.239596:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66945024 -> 66949120 00002000:00010000:3.0:1789760791.239602:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130776 -> 130784 00010000:00010000:3.0:1789760791.239623:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c691e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66945024->66949119] (req 66945024->66949119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.239637:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c691e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66945024->66949119] (req 66945024->66949119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.239648:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c691e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66945024->66949119] (req 66945024->66949119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.239656:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760791.239661:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c691e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66945024->66949119] (req 66945024->66949119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.239670:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c691e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66945024->66949119] (req 66945024->66949119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.239695:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c691e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66945024->66949119] (req 66945024->66949119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.239704:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c691e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66945024->66949119] (req 66945024->66949119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.239795:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554859392:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4864us (5068us total) trans 4294983653 rc 0/0 00000100:00100000:3.0:1789760791.239806:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16355 00000100:00100000:3.0:1789760791.239815:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734470856735 time 734480533329 state 0x2 ref 0 00000100:00100000:2.0:1789760791.245650:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760791.245726:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554859520 00000100:00100000:3.0:1789760791.245736:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554859520 00000100:00100000:3.0:1789760791.245791:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734470856735 time 734480533329 state 0x0 ref 1 00000100:00100000:3.0:1789760791.245807:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734480793745 time 734480533329 state 0x1 ref 1 00000100:00100000:3.0:1789760791.245841:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734480793745 time 734491459835 state 0x0 ref 1 00000100:00100000:3.0:1789760791.245855:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554859520:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760791.245902:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66949120, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fee0 00010000:00010000:3.0:1789760791.245911:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6925 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.245926:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760791.245931:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.245937:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6925 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66949120->66953215] (req 66949120->66953215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.245950:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6925 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66949120->66953215] (req 66949120->66953215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.249596:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983654 is committed 00002000:00010000:3.0:1789760791.249718:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66949120 -> 66953216 00002000:00010000:3.0:1789760791.249725:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130784 -> 130792 00010000:00010000:3.0:1789760791.249739:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6925 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66949120->66953215] (req 66949120->66953215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.249754:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6925 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66949120->66953215] (req 66949120->66953215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.249765:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6925 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66949120->66953215] (req 66949120->66953215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.249773:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760791.249778:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6925 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66949120->66953215] (req 66949120->66953215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.249789:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6925 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66949120->66953215] (req 66949120->66953215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.249810:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6925 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66949120->66953215] (req 66949120->66953215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.249825:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c6925 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66949120->66953215] (req 66949120->66953215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.249913:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554859520:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4056us (4269us total) trans 4294983654 rc 0/0 00000100:00100000:3.0:1789760791.249924:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16356 00000100:00100000:3.0:1789760791.249933:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734480793745 time 734491459835 state 0x2 ref 0 00000100:00100000:2.0:1789760791.256490:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760791.256591:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554859648 00000100:00100000:0.0:1789760791.256604:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554859648 00000100:00100000:0.0:1789760791.256676:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734480793745 time 734491459835 state 0x0 ref 1 00000100:00100000:0.0:1789760791.256691:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734491720251 time 734491459835 state 0x1 ref 1 00000100:00100000:0.0:1789760791.256720:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734491720251 time 734502338082 state 0x0 ref 1 00000100:00100000:0.0:1789760791.256732:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb036c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554859648:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760791.256777:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66953216, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fee8 00010000:00010000:0.0:1789760791.256786:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c692c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.256803:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760791.256807:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.256815:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c692c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66953216->66957311] (req 66953216->66957311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.256830:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c692c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66953216->66957311] (req 66953216->66957311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.260691:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983655 is committed 00002000:00010000:0.0:1789760791.260808:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66953216 -> 66957312 00002000:00010000:0.0:1789760791.260813:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130792 -> 130800 00010000:00010000:0.0:1789760791.260827:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c692c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66953216->66957311] (req 66953216->66957311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.260840:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c692c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66953216->66957311] (req 66953216->66957311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.260854:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c692c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66953216->66957311] (req 66953216->66957311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.260862:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760791.260867:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c692c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66953216->66957311] (req 66953216->66957311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.260875:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c692c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66953216->66957311] (req 66953216->66957311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.260907:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c692c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66953216->66957311] (req 66953216->66957311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.260915:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c692c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66953216->66957311] (req 66953216->66957311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.261010:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb036c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554859648:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4275us (4527us total) trans 4294983655 rc 0/0 00000100:00100000:0.0:1789760791.261029:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16357 00000100:00100000:0.0:1789760791.261039:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734491720251 time 734502338082 state 0x2 ref 0 00000100:00100000:2.0:1789760791.265712:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760791.265791:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554859776 00000100:00100000:1.0:1789760791.265800:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554859776 00000100:00100000:1.0:1789760791.265854:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734491720251 time 734502338082 state 0x0 ref 1 00000100:00100000:1.0:1789760791.265869:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734502598498 time 734502338082 state 0x1 ref 1 00000100:00100000:1.0:1789760791.265900:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734502598498 time 734511518243 state 0x0 ref 1 00000100:00100000:1.0:1789760791.265912:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb020b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554859776:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760791.265957:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66957312, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fef0 00010000:00010000:1.0:1789760791.265967:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6933 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.265983:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760791.265993:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.266001:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6933 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66957312->66961407] (req 66957312->66961407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.266015:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6933 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66957312->66961407] (req 66957312->66961407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.273252:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983656 is committed 00002000:00010000:1.0:1789760791.273550:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66957312 -> 66961408 00002000:00010000:1.0:1789760791.273556:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130800 -> 130808 00010000:00010000:1.0:1789760791.273569:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6933 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66957312->66961407] (req 66957312->66961407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.273583:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6933 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66957312->66961407] (req 66957312->66961407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.273593:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6933 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66957312->66961407] (req 66957312->66961407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.273606:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760791.273610:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6933 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66957312->66961407] (req 66957312->66961407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.273630:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6933 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66957312->66961407] (req 66957312->66961407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.273649:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6933 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66957312->66961407] (req 66957312->66961407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.273658:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6933 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66957312->66961407] (req 66957312->66961407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.273755:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb020b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554859776:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7840us (8049us total) trans 4294983656 rc 0/0 00000100:00100000:1.0:1789760791.273767:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16358 00000100:00100000:1.0:1789760791.273777:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734502598498 time 734511518243 state 0x2 ref 0 00000100:00100000:2.0:1789760791.283459:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760791.283569:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554859904 00000100:00100000:3.0:1789760791.283587:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554859904 00000100:00100000:3.0:1789760791.283647:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734502598498 time 734511518243 state 0x0 ref 1 00000100:00100000:3.0:1789760791.283661:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734511778659 time 734511518243 state 0x1 ref 1 00000100:00100000:3.0:1789760791.283688:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734511778659 time 734529306886 state 0x0 ref 1 00000100:00100000:3.0:1789760791.283700:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb020b0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554859904:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760791.283745:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66961408, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1fef8 00010000:00010000:3.0:1789760791.283753:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c693a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.283769:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760791.283773:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.283779:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c693a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66961408->66965503] (req 66961408->66965503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.283799:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c693a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66961408->66965503] (req 66961408->66965503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.291848:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983657 is committed 00002000:00010000:2.0:1789760791.294497:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66961408 -> 66965504 00002000:00010000:2.0:1789760791.294507:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130808 -> 130816 00010000:00010000:2.0:1789760791.294520:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c693a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66961408->66965503] (req 66961408->66965503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.294532:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c693a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66961408->66965503] (req 66961408->66965503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.294541:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c693a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66961408->66965503] (req 66961408->66965503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.294548:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760791.294553:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c693a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66961408->66965503] (req 66961408->66965503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.294562:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c693a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66961408->66965503] (req 66961408->66965503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.296129:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c693a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66961408->66965503] (req 66961408->66965503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.296146:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c693a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66961408->66965503] (req 66961408->66965503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.296258:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb020b0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554859904:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12555us (12802us total) trans 4294983657 rc 0/0 00000100:00100000:2.0:1789760791.296269:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16359 00000100:00100000:2.0:1789760791.296279:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734511778659 time 734529306886 state 0x2 ref 0 00000100:00100000:2.0:1789760791.300288:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760791.300530:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554860032 00000100:00100000:3.0:1789760791.300540:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554860032 00000100:00100000:3.0:1789760791.300593:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734511778659 time 734529306886 state 0x0 ref 1 00000100:00100000:3.0:1789760791.300610:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734529567302 time 734529306886 state 0x1 ref 1 00000100:00100000:3.0:1789760791.300651:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734529567302 time 734546269250 state 0x0 ref 1 00000100:00100000:3.0:1789760791.300663:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554860032:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760791.300709:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66965504, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1ff00 00010000:00010000:3.0:1789760791.300720:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6941 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.300736:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760791.300741:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.300747:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6941 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66965504->66969599] (req 66965504->66969599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.300761:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6941 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66965504->66969599] (req 66965504->66969599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.305360:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983658 is committed 00002000:00010000:3.0:1789760791.305575:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66965504 -> 66969600 00002000:00010000:3.0:1789760791.305581:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130816 -> 130824 00010000:00010000:3.0:1789760791.305595:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6941 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66965504->66969599] (req 66965504->66969599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.305609:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6941 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66965504->66969599] (req 66965504->66969599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.305631:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6941 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66965504->66969599] (req 66965504->66969599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.305640:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760791.305645:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6941 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66965504->66969599] (req 66965504->66969599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.305655:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6941 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66965504->66969599] (req 66965504->66969599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.305676:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6941 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66965504->66969599] (req 66965504->66969599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.305685:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6941 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66965504->66969599] (req 66965504->66969599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.305779:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554860032:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5112us (5496us total) trans 4294983658 rc 0/0 00000100:00100000:3.0:1789760791.305789:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16360 00000100:00100000:3.0:1789760791.305799:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734529567302 time 734546269250 state 0x2 ref 0 00000100:00100000:1.0:1789760791.311667:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760791.311761:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554860160 00000100:00100000:0.0:1789760791.311772:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554860160 00000100:00100000:0.0:1789760791.311825:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734529567302 time 734546269250 state 0x0 ref 1 00000100:00100000:0.0:1789760791.311840:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734546529666 time 734546269250 state 0x1 ref 1 00000100:00100000:0.0:1789760791.311925:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734546529666 time 734557542330 state 0x0 ref 1 00000100:00100000:0.0:1789760791.311939:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a384e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554860160:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760791.311988:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66969600, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1ff08 00010000:00010000:0.0:1789760791.311999:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6948 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.312014:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760791.312019:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.312025:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6948 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66969600->66973695] (req 66969600->66973695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.312039:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6948 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66969600->66973695] (req 66969600->66973695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.318290:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983659 is committed 00002000:00010000:1.0:1789760791.320822:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66969600 -> 66973696 00002000:00010000:1.0:1789760791.320841:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130824 -> 130832 00010000:00010000:1.0:1789760791.320854:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6948 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66969600->66973695] (req 66969600->66973695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.320867:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6948 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66969600->66973695] (req 66969600->66973695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.320877:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6948 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66969600->66973695] (req 66969600->66973695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.320884:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760791.320889:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6948 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66969600->66973695] (req 66969600->66973695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.320898:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6948 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66969600->66973695] (req 66969600->66973695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.320919:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6948 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66969600->66973695] (req 66969600->66973695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.320927:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6948 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66969600->66973695] (req 66969600->66973695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.321031:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a384e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554860160:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9090us (9367us total) trans 4294983659 rc 0/0 00000100:00100000:1.0:1789760791.321041:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16361 00000100:00100000:1.0:1789760791.321050:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734546529666 time 734557542330 state 0x2 ref 0 00000100:00100000:2.0:1789760791.327810:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760791.328040:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554860288 00000100:00100000:1.0:1789760791.328051:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554860288 00000100:00100000:1.0:1789760791.328101:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734546529666 time 734557542330 state 0x0 ref 1 00000100:00100000:1.0:1789760791.328116:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734557802746 time 734557542330 state 0x1 ref 1 00000100:00100000:1.0:1789760791.328193:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734557802746 time 734573802475 state 0x0 ref 1 00000100:00100000:1.0:1789760791.328205:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554860288:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760791.328250:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66973696, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1ff10 00010000:00010000:1.0:1789760791.328259:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c694f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.328273:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760791.328278:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.328284:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c694f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66973696->66977791] (req 66973696->66977791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.328298:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c694f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66973696->66977791] (req 66973696->66977791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.332450:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983660 is committed 00002000:00010000:1.0:1789760791.332610:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66973696 -> 66977792 00002000:00010000:1.0:1789760791.332623:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130832 -> 130840 00010000:00010000:1.0:1789760791.332637:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c694f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66973696->66977791] (req 66973696->66977791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.332650:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c694f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66973696->66977791] (req 66973696->66977791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.332660:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c694f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66973696->66977791] (req 66973696->66977791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.332674:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760791.332680:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c694f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66973696->66977791] (req 66973696->66977791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.332689:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c694f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66973696->66977791] (req 66973696->66977791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.332708:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c694f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66973696->66977791] (req 66973696->66977791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.332789:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c694f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66973696->66977791] (req 66973696->66977791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.332884:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554860288:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4676us (5079us total) trans 4294983660 rc 0/0 00000100:00100000:1.0:1789760791.332894:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16362 00000100:00100000:1.0:1789760791.332905:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734557802746 time 734573802475 state 0x2 ref 0 00000100:00100000:2.0:1789760791.337437:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760791.337503:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554860416 00000100:00100000:0.0:1789760791.337515:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554860416 00000100:00100000:1.0:1789760791.337600:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734557802746 time 734573802475 state 0x0 ref 1 00000100:00100000:1.0:1789760791.337625:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734574062891 time 734573802475 state 0x1 ref 1 00000100:00100000:1.0:1789760791.337656:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734574062891 time 734583274566 state 0x0 ref 1 00000100:00100000:1.0:1789760791.337668:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554860416:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760791.337727:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66977792, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1ff18 00010000:00010000:1.0:1789760791.337738:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6956 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.337754:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760791.337759:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.337765:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6956 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66977792->66981887] (req 66977792->66981887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.337779:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6956 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66977792->66981887] (req 66977792->66981887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.342115:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983661 is committed 00002000:00010000:1.0:1789760791.342237:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66977792 -> 66981888 00002000:00010000:1.0:1789760791.342243:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130840 -> 130848 00010000:00010000:1.0:1789760791.342256:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6956 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66977792->66981887] (req 66977792->66981887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.342268:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6956 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66977792->66981887] (req 66977792->66981887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.342277:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6956 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66977792->66981887] (req 66977792->66981887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.342284:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760791.342289:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6956 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66977792->66981887] (req 66977792->66981887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.342303:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6956 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66977792->66981887] (req 66977792->66981887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.342322:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6956 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66977792->66981887] (req 66977792->66981887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.342329:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6956 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66977792->66981887] (req 66977792->66981887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.342465:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554860416:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4794us (5033us total) trans 4294983661 rc 0/0 00000100:00100000:1.0:1789760791.342476:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16363 00000100:00100000:1.0:1789760791.342485:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734574062891 time 734583274566 state 0x2 ref 0 00000100:00100000:2.0:1789760791.351789:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760791.351877:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554860544 00000100:00100000:3.0:1789760791.351887:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554860544 00000100:00100000:3.0:1789760791.351942:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734574062891 time 734583274566 state 0x0 ref 1 00000100:00100000:3.0:1789760791.351957:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734583534982 time 734583274566 state 0x1 ref 1 00000100:00100000:3.0:1789760791.352039:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734583534982 time 734597656302 state 0x0 ref 1 00000100:00100000:3.0:1789760791.352052:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554860544:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760791.352100:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66981888, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1ff20 00010000:00010000:3.0:1789760791.352110:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c695d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.352127:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760791.352132:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.352139:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c695d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66981888->66985983] (req 66981888->66985983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.352153:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c695d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66981888->66985983] (req 66981888->66985983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.356151:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983662 is committed 00002000:00010000:3.0:1789760791.356264:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66981888 -> 66985984 00002000:00010000:3.0:1789760791.356270:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130848 -> 130856 00010000:00010000:3.0:1789760791.356283:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c695d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66981888->66985983] (req 66981888->66985983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.356302:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c695d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66981888->66985983] (req 66981888->66985983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.356312:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c695d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66981888->66985983] (req 66981888->66985983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.356319:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760791.356324:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c695d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66981888->66985983] (req 66981888->66985983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.356335:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c695d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66981888->66985983] (req 66981888->66985983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.356355:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c695d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66981888->66985983] (req 66981888->66985983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.356364:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c695d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66981888->66985983] (req 66981888->66985983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.356592:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554860544:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4537us (4809us total) trans 4294983662 rc 0/0 00000100:00100000:3.0:1789760791.356603:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16364 00000100:00100000:3.0:1789760791.356622:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734583534982 time 734597656302 state 0x2 ref 0 00000100:00100000:3.0:1789760791.374085:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760791.374176:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554860672 00000100:00100000:2.0:1789760791.374185:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554860672 00000100:00100000:2.0:1789760791.374233:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734583534982 time 734597656302 state 0x0 ref 1 00000100:00100000:2.0:1789760791.374247:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734597916718 time 734597656302 state 0x1 ref 1 00000100:00100000:2.0:1789760791.374276:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734597916718 time 734619894756 state 0x0 ref 1 00000100:00100000:2.0:1789760791.374286:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554860672:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760791.374326:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66985984, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1ff28 00010000:00010000:2.0:1789760791.374334:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6964 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.374350:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760791.374354:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.374359:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6964 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66985984->66990079] (req 66985984->66990079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.374413:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6964 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66985984->66990079] (req 66985984->66990079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.380517:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983663 is committed 00002000:00010000:2.0:1789760791.380648:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66985984 -> 66990080 00002000:00010000:2.0:1789760791.380654:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130856 -> 130864 00010000:00010000:2.0:1789760791.380667:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6964 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66985984->66990079] (req 66985984->66990079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.380681:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6964 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66985984->66990079] (req 66985984->66990079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.380692:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6964 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66985984->66990079] (req 66985984->66990079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.380700:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760791.380705:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6964 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66985984->66990079] (req 66985984->66990079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.380719:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6964 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66985984->66990079] (req 66985984->66990079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.380739:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6964 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66985984->66990079] (req 66985984->66990079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.380746:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6964 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66985984->66990079] (req 66985984->66990079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.380845:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554860672:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6555us (6764us total) trans 4294983663 rc 0/0 00000100:00100000:2.0:1789760791.380856:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16365 00000100:00100000:2.0:1789760791.380865:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734597916718 time 734619894756 state 0x2 ref 0 00000100:00100000:2.0:1789760791.384181:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760791.384267:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554860800 00000100:00100000:1.0:1789760791.384277:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554860800 00000100:00100000:1.0:1789760791.384330:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734597916718 time 734619894756 state 0x0 ref 1 00000100:00100000:1.0:1789760791.384344:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734620155172 time 734619894756 state 0x1 ref 1 00000100:00100000:1.0:1789760791.384417:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734620155172 time 734629993118 state 0x0 ref 1 00000100:00100000:1.0:1789760791.384431:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554860800:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760791.384477:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66990080, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1ff30 00010000:00010000:1.0:1789760791.384487:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c696b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.384503:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760791.384507:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.384513:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c696b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66990080->66994175] (req 66990080->66994175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.384527:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c696b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66990080->66994175] (req 66990080->66994175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.391001:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983664 is committed 00002000:00010000:1.0:1789760791.391147:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66990080 -> 66994176 00002000:00010000:1.0:1789760791.391152:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130864 -> 130872 00010000:00010000:1.0:1789760791.391165:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c696b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66990080->66994175] (req 66990080->66994175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.391177:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c696b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66990080->66994175] (req 66990080->66994175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.391185:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c696b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66990080->66994175] (req 66990080->66994175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.391193:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760791.391198:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c696b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66990080->66994175] (req 66990080->66994175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.391206:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c696b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66990080->66994175] (req 66990080->66994175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.391228:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c696b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66990080->66994175] (req 66990080->66994175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.391235:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c696b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66990080->66994175] (req 66990080->66994175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.391328:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554860800:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6895us (7151us total) trans 4294983664 rc 0/0 00000100:00100000:1.0:1789760791.391339:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16366 00000100:00100000:1.0:1789760791.391348:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734620155172 time 734629993118 state 0x2 ref 0 00000100:00100000:1.0:1789760791.395813:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760791.395892:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554860928 00000100:00100000:3.0:1789760791.395900:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554860928 00000100:00100000:3.0:1789760791.395950:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734620155172 time 734629993118 state 0x0 ref 1 00000100:00100000:3.0:1789760791.395964:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734630253534 time 734629993118 state 0x1 ref 1 00000100:00100000:3.0:1789760791.396000:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734630253534 time 734641618986 state 0x0 ref 1 00000100:00100000:3.0:1789760791.396012:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a386d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554860928:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760791.396059:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66994176, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1ff38 00010000:00010000:3.0:1789760791.396067:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6972 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.396083:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760791.396087:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.396093:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6972 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66994176->66998271] (req 66994176->66998271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.396106:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6972 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66994176->66998271] (req 66994176->66998271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.409925:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983665 is committed 00002000:00010000:3.0:1789760791.411261:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66994176 -> 66998272 00002000:00010000:3.0:1789760791.411270:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130872 -> 130880 00010000:00010000:3.0:1789760791.411284:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6972 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66994176->66998271] (req 66994176->66998271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.411298:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6972 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66994176->66998271] (req 66994176->66998271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.411307:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6972 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66994176->66998271] (req 66994176->66998271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.411315:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760791.411321:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6972 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66994176->66998271] (req 66994176->66998271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.411330:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6972 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66994176->66998271] (req 66994176->66998271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.411349:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6972 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66994176->66998271] (req 66994176->66998271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.411357:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6972 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66994176->66998271] (req 66994176->66998271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.411661:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a386d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554860928:12345-192.168.203.22@tcp:4:dd.0 Request processed in 15645us (15852us total) trans 4294983665 rc 0/0 00000100:00100000:3.0:1789760791.411672:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16367 00000100:00100000:3.0:1789760791.411681:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734630253534 time 734641618986 state 0x2 ref 0 00000100:00100000:3.0:1789760791.417956:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760791.418033:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554861056 00000100:00100000:0.0:1789760791.418043:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554861056 00000100:00100000:0.0:1789760791.418092:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734630253534 time 734641618986 state 0x0 ref 1 00000100:00100000:0.0:1789760791.418106:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734641879402 time 734641618986 state 0x1 ref 1 00000100:00100000:0.0:1789760791.418139:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734641879402 time 734663757052 state 0x0 ref 1 00000100:00100000:0.0:1789760791.418155:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe1a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554861056:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760791.418200:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 66998272, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1ff40 00010000:00010000:0.0:1789760791.418209:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6979 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.418223:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760791.418227:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.418234:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6979 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66998272->67002367] (req 66998272->67002367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.418245:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6979 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66998272->67002367] (req 66998272->67002367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.424096:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983666 is committed 00002000:00010000:3.0:1789760791.424640:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 66998272 -> 67002368 00002000:00010000:3.0:1789760791.424645:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130880 -> 130888 00010000:00010000:3.0:1789760791.424658:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6979 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66998272->67002367] (req 66998272->67002367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.424671:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6979 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66998272->67002367] (req 66998272->67002367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.424681:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6979 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66998272->67002367] (req 66998272->67002367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.424690:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760791.424694:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6979 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66998272->67002367] (req 66998272->67002367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.424703:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6979 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66998272->67002367] (req 66998272->67002367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.424723:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6979 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66998272->67002367] (req 66998272->67002367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.424730:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6979 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [66998272->67002367] (req 66998272->67002367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.424859:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe1a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554861056:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6704us (6906us total) trans 4294983666 rc 0/0 00000100:00100000:3.0:1789760791.424869:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16368 00000100:00100000:3.0:1789760791.424878:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734641879402 time 734663757052 state 0x2 ref 0 00000100:00100000:2.0:1789760791.436118:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760791.436256:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554861184 00000100:00100000:1.0:1789760791.436266:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554861184 00000100:00100000:1.0:1789760791.436315:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734641879402 time 734663757052 state 0x0 ref 1 00000100:00100000:1.0:1789760791.436328:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734664017468 time 734663757052 state 0x1 ref 1 00000100:00100000:0.0:1789760791.437655:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734664017468 time 734683265430 state 0x0 ref 1 00000100:00100000:0.0:1789760791.437680:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554861184:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760791.437732:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67002368, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1ff48 00010000:00010000:0.0:1789760791.437742:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6980 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.437762:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760791.437767:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.437773:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6980 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67002368->67006463] (req 67002368->67006463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.437785:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6980 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67002368->67006463] (req 67002368->67006463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.443237:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983667 is committed 00002000:00010000:0.0:1789760791.443337:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67002368 -> 67006464 00002000:00010000:0.0:1789760791.443341:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130888 -> 130896 00010000:00010000:0.0:1789760791.443354:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6980 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67002368->67006463] (req 67002368->67006463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.443406:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6980 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67002368->67006463] (req 67002368->67006463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.443416:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6980 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67002368->67006463] (req 67002368->67006463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.443424:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760791.443428:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6980 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67002368->67006463] (req 67002368->67006463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.443438:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6980 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67002368->67006463] (req 67002368->67006463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.443462:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6980 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67002368->67006463] (req 67002368->67006463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.443469:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c6980 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67002368->67006463] (req 67002368->67006463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.443552:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554861184:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5876us (7440us total) trans 4294983667 rc 0/0 00000100:00100000:0.0:1789760791.443561:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16369 00000100:00100000:0.0:1789760791.443570:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734664017468 time 734683265430 state 0x2 ref 0 00000100:00100000:1.0:1789760791.456230:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760791.456587:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554861312 00000100:00100000:2.0:1789760791.456596:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554861312 00000100:00100000:2.0:1789760791.456655:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734664017468 time 734683265430 state 0x0 ref 1 00000100:00100000:2.0:1789760791.456669:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734683525846 time 734683265430 state 0x1 ref 1 00000100:00100000:2.0:1789760791.456697:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734683525846 time 734702315931 state 0x0 ref 1 00000100:00100000:2.0:1789760791.456712:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554861312:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760791.456754:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67006464, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1ff50 00010000:00010000:2.0:1789760791.456763:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6987 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.456778:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760791.456782:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.456789:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6987 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67006464->67010559] (req 67006464->67010559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.456803:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6987 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67006464->67010559] (req 67006464->67010559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.464129:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983668 is committed 00002000:00010000:2.0:1789760791.464247:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67006464 -> 67010560 00002000:00010000:2.0:1789760791.464252:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130896 -> 130904 00010000:00010000:2.0:1789760791.464264:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6987 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67006464->67010559] (req 67006464->67010559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.464276:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6987 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67006464->67010559] (req 67006464->67010559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.464286:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6987 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67006464->67010559] (req 67006464->67010559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.464294:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760791.464299:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6987 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67006464->67010559] (req 67006464->67010559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.464308:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6987 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67006464->67010559] (req 67006464->67010559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.464326:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6987 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67006464->67010559] (req 67006464->67010559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.464333:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6987 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67006464->67010559] (req 67006464->67010559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.464610:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554861312:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7892us (8384us total) trans 4294983668 rc 0/0 00000100:00100000:2.0:1789760791.464629:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16370 00000100:00100000:2.0:1789760791.464639:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734683525846 time 734702315931 state 0x2 ref 0 00000100:00100000:3.0:1789760791.472444:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760791.472572:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554861440 00000100:00100000:1.0:1789760791.472582:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554861440 00000100:00100000:1.0:1789760791.472645:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734683525846 time 734702315931 state 0x0 ref 1 00000100:00100000:1.0:1789760791.472660:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734702576347 time 734702315931 state 0x1 ref 1 00000100:00100000:1.0:1789760791.472690:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734702576347 time 734718308817 state 0x0 ref 1 00000100:00100000:1.0:1789760791.472704:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554861440:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760791.472750:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67010560, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1ff58 00010000:00010000:1.0:1789760791.472758:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c698e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.472774:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760791.472778:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.472784:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c698e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67010560->67014655] (req 67010560->67014655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.472797:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c698e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67010560->67014655] (req 67010560->67014655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.476977:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983669 is committed 00002000:00010000:1.0:1789760791.477140:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67010560 -> 67014656 00002000:00010000:1.0:1789760791.477145:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130904 -> 130912 00010000:00010000:1.0:1789760791.477158:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c698e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67010560->67014655] (req 67010560->67014655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.477172:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c698e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67010560->67014655] (req 67010560->67014655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.477182:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c698e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67010560->67014655] (req 67010560->67014655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.477190:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760791.477195:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c698e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67010560->67014655] (req 67010560->67014655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.477204:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c698e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67010560->67014655] (req 67010560->67014655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.477223:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c698e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67010560->67014655] (req 67010560->67014655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.477231:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c698e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67010560->67014655] (req 67010560->67014655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.477327:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554861440:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4620us (4887us total) trans 4294983669 rc 0/0 00000100:00100000:1.0:1789760791.477337:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16371 00000100:00100000:1.0:1789760791.477346:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734702576347 time 734718308817 state 0x2 ref 0 00000100:00100000:0.0:1789760791.492028:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760791.492189:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554861568 00000100:00100000:3.0:1789760791.492199:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554861568 00000100:00100000:3.0:1789760791.492252:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734702576347 time 734718308817 state 0x0 ref 1 00000100:00100000:3.0:1789760791.492268:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734718569233 time 734718308817 state 0x1 ref 1 00000100:00100000:3.0:1789760791.492303:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734718569233 time 734737921065 state 0x0 ref 1 00000100:00100000:3.0:1789760791.492315:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554861568:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760791.492359:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67014656, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1ff60 00010000:00010000:3.0:1789760791.492368:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6995 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.492429:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760791.492433:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.492439:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6995 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67014656->67018751] (req 67014656->67018751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.492453:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6995 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67014656->67018751] (req 67014656->67018751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.500096:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983670 is committed 00002000:00010000:0.0:1789760791.500217:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67014656 -> 67018752 00002000:00010000:0.0:1789760791.500223:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130912 -> 130920 00010000:00010000:0.0:1789760791.500235:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6995 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67014656->67018751] (req 67014656->67018751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.500248:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6995 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67014656->67018751] (req 67014656->67018751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.500257:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6995 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67014656->67018751] (req 67014656->67018751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.500266:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760791.500271:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6995 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67014656->67018751] (req 67014656->67018751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.500281:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6995 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67014656->67018751] (req 67014656->67018751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.500302:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6995 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67014656->67018751] (req 67014656->67018751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.500309:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c6995 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67014656->67018751] (req 67014656->67018751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.500464:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554861568:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8146us (8440us total) trans 4294983670 rc 0/0 00000100:00100000:0.0:1789760791.500475:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16372 00000100:00100000:0.0:1789760791.500485:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734718569233 time 734737921065 state 0x2 ref 0 00000100:00100000:2.0:1789760791.505601:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760791.505696:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554861696 00000100:00100000:0.0:1789760791.505706:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554861696 00000100:00100000:0.0:1789760791.505757:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734718569233 time 734737921065 state 0x0 ref 1 00000100:00100000:0.0:1789760791.505774:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734738181481 time 734737921065 state 0x1 ref 1 00000100:00100000:0.0:1789760791.505806:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734738181481 time 734751424719 state 0x0 ref 1 00000100:00100000:0.0:1789760791.505818:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554861696:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760791.505865:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67018752, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1ff68 00010000:00010000:0.0:1789760791.505874:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c699c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.505889:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760791.505894:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.505900:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c699c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67018752->67022847] (req 67018752->67022847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.505914:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c699c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67018752->67022847] (req 67018752->67022847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.510850:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983671 is committed 00002000:00010000:2.0:1789760791.511071:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67018752 -> 67022848 00002000:00010000:2.0:1789760791.511076:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130920 -> 130928 00010000:00010000:2.0:1789760791.511090:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c699c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67018752->67022847] (req 67018752->67022847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.511104:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c699c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67018752->67022847] (req 67018752->67022847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.511119:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c699c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67018752->67022847] (req 67018752->67022847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.511127:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760791.511132:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c699c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67018752->67022847] (req 67018752->67022847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.511142:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c699c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67018752->67022847] (req 67018752->67022847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.511162:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c699c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67018752->67022847] (req 67018752->67022847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.511170:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c699c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67018752->67022847] (req 67018752->67022847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.511293:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554861696:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5471us (5696us total) trans 4294983671 rc 0/0 00000100:00100000:2.0:1789760791.511303:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16373 00000100:00100000:2.0:1789760791.511313:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734738181481 time 734751424719 state 0x2 ref 0 00000100:00100000:3.0:1789760791.522468:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760791.522806:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554861824 00000100:00100000:2.0:1789760791.522816:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554861824 00000100:00100000:2.0:1789760791.522871:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734738181481 time 734751424719 state 0x0 ref 1 00000100:00100000:2.0:1789760791.522887:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734751685135 time 734751424719 state 0x1 ref 1 00000100:00100000:2.0:1789760791.522924:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734751685135 time 734768542079 state 0x0 ref 1 00000100:00100000:2.0:1789760791.522937:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554861824:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760791.522984:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67022848, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1ff70 00010000:00010000:2.0:1789760791.523003:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c69a3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.523019:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760791.523024:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.523030:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c69a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67022848->67026943] (req 67022848->67026943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.523045:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c69a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67022848->67026943] (req 67022848->67026943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.527251:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983672 is committed 00002000:00010000:2.0:1789760791.529903:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67022848 -> 67026944 00002000:00010000:2.0:1789760791.529911:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130928 -> 130936 00010000:00010000:2.0:1789760791.529924:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c69a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67022848->67026943] (req 67022848->67026943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.529936:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c69a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67022848->67026943] (req 67022848->67026943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.529944:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c69a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67022848->67026943] (req 67022848->67026943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.529957:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760791.529962:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c69a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67022848->67026943] (req 67022848->67026943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.529971:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c69a3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67022848->67026943] (req 67022848->67026943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.529992:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c69a3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67022848->67026943] (req 67022848->67026943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.529999:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c69a3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67022848->67026943] (req 67022848->67026943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.530108:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554861824:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7168us (7642us total) trans 4294983672 rc 0/0 00000100:00100000:2.0:1789760791.530120:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16374 00000100:00100000:2.0:1789760791.530130:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734751685135 time 734768542079 state 0x2 ref 0 00000100:00100000:0.0:1789760791.536945:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760791.537294:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554861952 00000100:00100000:1.0:1789760791.537305:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554861952 00000100:00100000:1.0:1789760791.537360:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734751685135 time 734768542079 state 0x0 ref 1 00000100:00100000:1.0:1789760791.537403:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734768802495 time 734768542079 state 0x1 ref 1 00000100:00100000:1.0:1789760791.537446:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734768802495 time 734783063794 state 0x0 ref 1 00000100:00100000:1.0:1789760791.537459:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554861952:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760791.537508:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67026944, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1ff78 00010000:00010000:1.0:1789760791.537518:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c69aa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.537534:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760791.537538:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.537544:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c69aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67026944->67031039] (req 67026944->67031039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.537558:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c69aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67026944->67031039] (req 67026944->67031039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.545634:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983673 is committed 00002000:00010000:1.0:1789760791.545757:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67026944 -> 67031040 00002000:00010000:1.0:1789760791.545761:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130936 -> 130944 00010000:00010000:1.0:1789760791.545773:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c69aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67026944->67031039] (req 67026944->67031039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.545781:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c69aa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67026944->67031039] (req 67026944->67031039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.545787:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c69aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67026944->67031039] (req 67026944->67031039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.545792:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760791.545795:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c69aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67026944->67031039] (req 67026944->67031039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.545801:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c69aa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67026944->67031039] (req 67026944->67031039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.545817:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c69aa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67026944->67031039] (req 67026944->67031039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.545822:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c69aa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67026944->67031039] (req 67026944->67031039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.545903:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554861952:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8437us (8958us total) trans 4294983673 rc 0/0 00000100:00100000:1.0:1789760791.545913:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16375 00000100:00100000:1.0:1789760791.545924:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734768802495 time 734783063794 state 0x2 ref 0 00000100:00100000:1.0:1789760791.550631:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760791.550719:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554862080 00000100:00100000:0.0:1789760791.550730:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554862080 00000100:00100000:0.0:1789760791.550792:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734768802495 time 734783063794 state 0x0 ref 1 00000100:00100000:0.0:1789760791.550808:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734783324210 time 734783063794 state 0x1 ref 1 00000100:00100000:0.0:1789760791.554829:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734783324210 time 734800434352 state 0x0 ref 1 00000100:00100000:0.0:1789760791.554861:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a386a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554862080:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760791.554916:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67031040, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1ff80 00010000:00010000:0.0:1789760791.554927:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69b1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.554950:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760791.554955:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.554962:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67031040->67035135] (req 67031040->67035135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.554976:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67031040->67035135] (req 67031040->67035135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.559420:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983674 is committed 00002000:00010000:0.0:1789760791.559535:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67031040 -> 67035136 00002000:00010000:0.0:1789760791.559540:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130944 -> 130952 00010000:00010000:0.0:1789760791.559553:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67031040->67035135] (req 67031040->67035135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.559567:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69b1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67031040->67035135] (req 67031040->67035135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.559577:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67031040->67035135] (req 67031040->67035135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.559585:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760791.559591:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67031040->67035135] (req 67031040->67035135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.559601:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69b1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67031040->67035135] (req 67031040->67035135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.559638:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69b1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67031040->67035135] (req 67031040->67035135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.559647:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69b1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67031040->67035135] (req 67031040->67035135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.559744:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a386a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554862080:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4890us (9126us total) trans 4294983674 rc 0/0 00000100:00100000:0.0:1789760791.559755:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16376 00000100:00100000:0.0:1789760791.559764:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734783324210 time 734800434352 state 0x2 ref 0 00000100:00100000:3.0:1789760791.565862:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760791.565948:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554862208 00000100:00100000:2.0:1789760791.565958:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554862208 00000100:00100000:2.0:1789760791.566008:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734783324210 time 734800434352 state 0x0 ref 1 00000100:00100000:2.0:1789760791.566023:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734800694768 time 734800434352 state 0x1 ref 1 00000100:00100000:2.0:1789760791.566054:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734800694768 time 734811672713 state 0x0 ref 1 00000100:00100000:2.0:1789760791.566066:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554862208:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760791.566109:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67035136, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1ff88 00010000:00010000:2.0:1789760791.566117:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c69b8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.566133:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760791.566137:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.566143:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c69b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67035136->67039231] (req 67035136->67039231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.566156:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c69b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67035136->67039231] (req 67035136->67039231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.577052:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983675 is committed 00002000:00010000:3.0:1789760791.581238:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67035136 -> 67039232 00002000:00010000:3.0:1789760791.581248:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130952 -> 130960 00010000:00010000:3.0:1789760791.581261:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c69b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67035136->67039231] (req 67035136->67039231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.581279:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c69b8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67035136->67039231] (req 67035136->67039231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.581288:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c69b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67035136->67039231] (req 67035136->67039231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.581295:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760791.581300:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c69b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67035136->67039231] (req 67035136->67039231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.581308:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c69b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67035136->67039231] (req 67035136->67039231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.581327:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c69b8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67035136->67039231] (req 67035136->67039231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.581334:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c69b8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67035136->67039231] (req 67035136->67039231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.582220:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554862208:12345-192.168.203.22@tcp:4:dd.0 Request processed in 16148us (16360us total) trans 4294983675 rc 0/0 00000100:00100000:3.0:1789760791.582233:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16377 00000100:00100000:3.0:1789760791.582242:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734800694768 time 734811672713 state 0x2 ref 0 00000100:00100000:2.0:1789760791.586186:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760791.586350:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554862336 00000100:00100000:1.0:1789760791.586359:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554862336 00000100:00100000:1.0:1789760791.586466:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734800694768 time 734811672713 state 0x0 ref 1 00000100:00100000:1.0:1789760791.586483:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734811933129 time 734811672713 state 0x1 ref 1 00000100:00100000:1.0:1789760791.586505:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734811933129 time 734832123482 state 0x0 ref 1 00000100:00100000:1.0:1789760791.586517:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554862336:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760791.586563:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67039232, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1ff90 00010000:00010000:1.0:1789760791.586572:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c69bf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.586593:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760791.586597:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.586604:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c69bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67039232->67043327] (req 67039232->67043327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.586632:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c69bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67039232->67043327] (req 67039232->67043327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.594438:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983676 is committed 00002000:00010000:1.0:1789760791.594655:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67039232 -> 67043328 00002000:00010000:1.0:1789760791.594661:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130960 -> 130968 00010000:00010000:1.0:1789760791.594674:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c69bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67039232->67043327] (req 67039232->67043327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.594686:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c69bf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67039232->67043327] (req 67039232->67043327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.594695:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c69bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67039232->67043327] (req 67039232->67043327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.594702:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760791.594707:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c69bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67039232->67043327] (req 67039232->67043327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.594716:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c69bf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67039232->67043327] (req 67039232->67043327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.594741:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c69bf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67039232->67043327] (req 67039232->67043327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.594750:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c69bf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67039232->67043327] (req 67039232->67043327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.594843:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554862336:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8323us (8662us total) trans 4294983676 rc 0/0 00000100:00100000:1.0:1789760791.594853:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16378 00000100:00100000:1.0:1789760791.594863:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734811933129 time 734832123482 state 0x2 ref 0 00000100:00100000:1.0:1789760791.602433:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760791.602553:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554862464 00000100:00100000:3.0:1789760791.602561:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554862464 00000100:00100000:3.0:1789760791.602625:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734811933129 time 734832123482 state 0x0 ref 1 00000100:00100000:3.0:1789760791.602641:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734832383898 time 734832123482 state 0x1 ref 1 00000100:00100000:3.0:1789760791.602665:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734832383898 time 734848283806 state 0x0 ref 1 00000100:00100000:3.0:1789760791.602677:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a386300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554862464:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760791.602726:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67043328, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1ff98 00010000:00010000:3.0:1789760791.602735:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c69c6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.602750:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760791.602755:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.602762:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c69c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67043328->67047423] (req 67043328->67047423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.602776:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c69c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67043328->67047423] (req 67043328->67047423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.610300:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983677 is committed 00002000:00010000:3.0:1789760791.610579:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67043328 -> 67047424 00002000:00010000:3.0:1789760791.610584:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130968 -> 130976 00010000:00010000:3.0:1789760791.610598:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c69c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67043328->67047423] (req 67043328->67047423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.610621:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c69c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67043328->67047423] (req 67043328->67047423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.610632:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c69c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67043328->67047423] (req 67043328->67047423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.610640:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760791.610644:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c69c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67043328->67047423] (req 67043328->67047423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.610654:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c69c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67043328->67047423] (req 67043328->67047423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.610684:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c69c6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67043328->67047423] (req 67043328->67047423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.610691:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c69c6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67043328->67047423] (req 67043328->67047423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.610789:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a386300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554862464:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8109us (8361us total) trans 4294983677 rc 0/0 00000100:00100000:3.0:1789760791.610800:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16379 00000100:00100000:3.0:1789760791.610810:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734832383898 time 734848283806 state 0x2 ref 0 00000100:00100000:1.0:1789760791.618978:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760791.619178:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554862592 00000100:00100000:0.0:1789760791.619192:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554862592 00000100:00100000:0.0:1789760791.619251:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734832383898 time 734848283806 state 0x0 ref 1 00000100:00100000:0.0:1789760791.619334:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734848544222 time 734848283806 state 0x1 ref 1 00000100:00100000:0.0:1789760791.619646:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734848544222 time 734865262878 state 0x0 ref 1 00000100:00100000:0.0:1789760791.619662:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554862592:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760791.619711:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67047424, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1ffa0 00010000:00010000:0.0:1789760791.619720:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69cd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.619737:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760791.619743:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.619749:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67047424->67051519] (req 67047424->67051519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.619763:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67047424->67051519] (req 67047424->67051519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.625848:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983678 is committed 00002000:00010000:0.0:1789760791.626700:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67047424 -> 67051520 00002000:00010000:0.0:1789760791.626707:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130976 -> 130984 00010000:00010000:0.0:1789760791.626721:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67047424->67051519] (req 67047424->67051519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.626735:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67047424->67051519] (req 67047424->67051519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.626744:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67047424->67051519] (req 67047424->67051519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.626752:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760791.626757:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67047424->67051519] (req 67047424->67051519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.626767:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69cd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67047424->67051519] (req 67047424->67051519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.626786:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69cd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67047424->67051519] (req 67047424->67051519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.626801:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69cd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67047424->67051519] (req 67047424->67051519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.626940:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554862592:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7276us (7966us total) trans 4294983678 rc 0/0 00000100:00100000:0.0:1789760791.626951:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16380 00000100:00100000:0.0:1789760791.626960:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734848544222 time 734865262878 state 0x2 ref 0 00000100:00100000:2.0:1789760791.633852:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760791.633947:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554862720 00000100:00100000:0.0:1789760791.633959:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554862720 00000100:00100000:0.0:1789760791.634024:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734848544222 time 734865262878 state 0x0 ref 1 00000100:00100000:0.0:1789760791.634040:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734865523294 time 734865262878 state 0x1 ref 1 00000100:00100000:0.0:1789760791.634072:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734865523294 time 734879690107 state 0x0 ref 1 00000100:00100000:0.0:1789760791.634085:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554862720:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760791.634134:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67051520, mtime: 0x6aad9516, atime: 0x0, ctime: 0x6aad9516, blocks: 0x1ffa8 00010000:00010000:0.0:1789760791.634143:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69d4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.634160:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789760791.634165:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.634171:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67051520->67055615] (req 67051520->67055615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.634190:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67051520->67055615] (req 67051520->67055615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.641045:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983679 is committed 00002000:00010000:0.0:1789760791.642124:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67051520 -> 67055616 00002000:00010000:0.0:1789760791.642134:0:8442:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760790 -> 1789760791 00002000:00010000:0.0:1789760791.642137:0:8442:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760790 -> 1789760791 00002000:00010000:0.0:1789760791.642139:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130984 -> 130992 00010000:00010000:0.0:1789760791.642156:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67051520->67055615] (req 67051520->67055615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.642170:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67051520->67055615] (req 67051520->67055615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.642184:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67051520->67055615] (req 67051520->67055615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.642192:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760791.642197:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67051520->67055615] (req 67051520->67055615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.642208:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69d4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67051520->67055615] (req 67051520->67055615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.642228:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69d4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67051520->67055615] (req 67051520->67055615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.642240:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c69d4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67051520->67055615] (req 67051520->67055615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.642803:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554862720:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8713us (8953us total) trans 4294983679 rc 0/0 00000100:00100000:0.0:1789760791.642814:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16381 00000100:00100000:0.0:1789760791.642823:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734865523294 time 734879690107 state 0x2 ref 0 00000100:00100000:0.0:1789760791.647134:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760791.647221:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554862848 00000100:00100000:1.0:1789760791.647230:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554862848 00000100:00100000:1.0:1789760791.647284:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734865523294 time 734879690107 state 0x0 ref 1 00000100:00100000:1.0:1789760791.647297:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734879950523 time 734879690107 state 0x1 ref 1 00000100:00100000:1.0:1789760791.647349:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734879950523 time 734892967650 state 0x0 ref 1 00000100:00100000:1.0:1789760791.647361:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554862848:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760791.647936:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67055616, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x1ffb0 00010000:00010000:1.0:1789760791.647947:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c69db lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.647965:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760791.647969:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.647976:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c69db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67055616->67059711] (req 67055616->67059711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.648096:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c69db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67055616->67059711] (req 67055616->67059711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.655957:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983680 is committed 00002000:00010000:1.0:1789760791.656188:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67055616 -> 67059712 00002000:00010000:1.0:1789760791.656194:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 130992 -> 131000 00010000:00010000:1.0:1789760791.656207:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c69db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67055616->67059711] (req 67055616->67059711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.656220:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c69db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67055616->67059711] (req 67055616->67059711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.656230:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c69db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67055616->67059711] (req 67055616->67059711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.656239:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760791.656244:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c69db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67055616->67059711] (req 67055616->67059711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.656266:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c69db lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67055616->67059711] (req 67055616->67059711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.656292:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c69db lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67055616->67059711] (req 67055616->67059711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.656301:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c69db lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67055616->67059711] (req 67055616->67059711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.656670:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554862848:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9305us (9538us total) trans 4294983680 rc 0/0 00000100:00100000:1.0:1789760791.656681:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16382 00000100:00100000:1.0:1789760791.656690:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734879950523 time 734892967650 state 0x2 ref 0 00000100:00100000:1.0:1789760791.660658:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760791.660737:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554862976 00000100:00100000:3.0:1789760791.660746:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554862976 00000100:00100000:3.0:1789760791.660796:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734879950523 time 734892967650 state 0x0 ref 1 00000100:00100000:3.0:1789760791.660817:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734893228066 time 734892967650 state 0x1 ref 1 00000100:00100000:3.0:1789760791.660838:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734893228066 time 734906457425 state 0x0 ref 1 00000100:00100000:3.0:1789760791.660851:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a386680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554862976:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760791.660896:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67059712, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x1ffb8 00010000:00010000:3.0:1789760791.660906:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c69e2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.660923:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760791.660927:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.660933:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c69e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67059712->67063807] (req 67059712->67063807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.660949:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c69e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67059712->67063807] (req 67059712->67063807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.669058:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983681 is committed 00002000:00010000:1.0:1789760791.669234:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67059712 -> 67063808 00002000:00010000:1.0:1789760791.669240:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131000 -> 131008 00010000:00010000:1.0:1789760791.669312:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c69e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67059712->67063807] (req 67059712->67063807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.669327:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c69e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67059712->67063807] (req 67059712->67063807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.669337:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c69e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67059712->67063807] (req 67059712->67063807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.669345:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760791.669351:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c69e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67059712->67063807] (req 67059712->67063807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.669361:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c69e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67059712->67063807] (req 67059712->67063807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.669896:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c69e2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67059712->67063807] (req 67059712->67063807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.669906:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c69e2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67059712->67063807] (req 67059712->67063807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.670202:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a386680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554862976:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9348us (9548us total) trans 4294983681 rc 0/0 00000100:00100000:1.0:1789760791.670213:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16383 00000100:00100000:1.0:1789760791.670222:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734893228066 time 734906457425 state 0x2 ref 0 00000100:00100000:0.0:1789760791.675278:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760791.675407:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554863104 00000100:00100000:2.0:1789760791.675417:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554863104 00000100:00100000:2.0:1789760791.675483:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734893228066 time 734906457425 state 0x0 ref 1 00000100:00100000:2.0:1789760791.675499:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734906717841 time 734906457425 state 0x1 ref 1 00000100:00100000:2.0:1789760791.675530:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734906717841 time 734921148749 state 0x0 ref 1 00000100:00100000:2.0:1789760791.675547:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554863104:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760791.675592:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67063808, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x1ffc0 00010000:00010000:2.0:1789760791.675602:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c69e9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.675631:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760791.675637:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.675643:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c69e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67063808->67067903] (req 67063808->67067903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.675657:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c69e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67063808->67067903] (req 67063808->67067903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.681214:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983682 is committed 00002000:00010000:2.0:1789760791.681331:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67063808 -> 67067904 00002000:00010000:2.0:1789760791.681337:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131008 -> 131016 00010000:00010000:2.0:1789760791.681350:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c69e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67063808->67067903] (req 67063808->67067903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.681363:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c69e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67063808->67067903] (req 67063808->67067903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.681417:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c69e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67063808->67067903] (req 67063808->67067903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.681425:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760791.681430:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c69e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67063808->67067903] (req 67063808->67067903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.681440:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c69e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67063808->67067903] (req 67063808->67067903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.681459:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c69e9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67063808->67067903] (req 67063808->67067903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.681467:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c69e9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67063808->67067903] (req 67063808->67067903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.681930:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554863104:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6019us (6295us total) trans 4294983682 rc 0/0 00000100:00100000:2.0:1789760791.681942:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16384 00000100:00100000:2.0:1789760791.681951:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734906717841 time 734921148749 state 0x2 ref 0 00000100:00100000:0.0:1789760791.685321:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760791.685593:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554863232 00000100:00100000:3.0:1789760791.685603:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554863232 00000100:00100000:3.0:1789760791.685664:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734906717841 time 734921148749 state 0x0 ref 1 00000100:00100000:3.0:1789760791.685679:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734921409165 time 734921148749 state 0x1 ref 1 00000100:00100000:3.0:1789760791.685750:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734921409165 time 734931368338 state 0x0 ref 1 00000100:00100000:3.0:1789760791.685767:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554863232:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760791.685811:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67067904, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x1ffc8 00010000:00010000:3.0:1789760791.685820:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c69f0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.685836:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760791.685840:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.685845:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c69f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67067904->67071999] (req 67067904->67071999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.685860:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c69f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67067904->67071999] (req 67067904->67071999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.693495:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983683 is committed 00002000:00010000:3.0:1789760791.693706:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67067904 -> 67072000 00002000:00010000:3.0:1789760791.693711:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131016 -> 131024 00010000:00010000:3.0:1789760791.693726:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c69f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67067904->67071999] (req 67067904->67071999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.693739:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c69f0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67067904->67071999] (req 67067904->67071999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.693748:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c69f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67067904->67071999] (req 67067904->67071999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.693755:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760791.693760:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c69f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67067904->67071999] (req 67067904->67071999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.693775:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c69f0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67067904->67071999] (req 67067904->67071999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.694064:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c69f0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67067904->67071999] (req 67067904->67071999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.694072:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c69f0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67067904->67071999] (req 67067904->67071999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.694219:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554863232:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8446us (8895us total) trans 4294983683 rc 0/0 00000100:00100000:3.0:1789760791.694231:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16385 00000100:00100000:3.0:1789760791.694240:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734921409165 time 734931368338 state 0x2 ref 0 00000100:00100000:1.0:1789760791.702330:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760791.702451:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554863360 00000100:00100000:2.0:1789760791.702462:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554863360 00000100:00100000:2.0:1789760791.702517:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734921409165 time 734931368338 state 0x0 ref 1 00000100:00100000:2.0:1789760791.702533:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734931628754 time 734931368338 state 0x1 ref 1 00000100:00100000:2.0:1789760791.702564:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734931628754 time 734948182568 state 0x0 ref 1 00000100:00100000:2.0:1789760791.702576:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554863360:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760791.702696:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67072000, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x1ffd0 00010000:00010000:2.0:1789760791.702707:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c69f7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.702723:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760791.702728:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.702734:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c69f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67072000->67076095] (req 67072000->67076095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.702749:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c69f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67072000->67076095] (req 67072000->67076095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.710908:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983684 is committed 00002000:00010000:2.0:1789760791.711025:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67072000 -> 67076096 00002000:00010000:2.0:1789760791.711030:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131024 -> 131032 00010000:00010000:2.0:1789760791.711043:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c69f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67072000->67076095] (req 67072000->67076095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.711057:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c69f7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67072000->67076095] (req 67072000->67076095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.711066:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c69f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67072000->67076095] (req 67072000->67076095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.711073:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760791.711078:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c69f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67072000->67076095] (req 67072000->67076095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.711087:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c69f7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67072000->67076095] (req 67072000->67076095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.711109:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c69f7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67072000->67076095] (req 67072000->67076095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.711117:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c69f7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67072000->67076095] (req 67072000->67076095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.711222:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554863360:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8642us (8902us total) trans 4294983684 rc 0/0 00000100:00100000:2.0:1789760791.711233:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16386 00000100:00100000:2.0:1789760791.711243:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734931628754 time 734948182568 state 0x2 ref 0 00000100:00100000:0.0:1789760791.721473:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760791.721567:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554863488 00000100:00100000:1.0:1789760791.721578:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554863488 00000100:00100000:1.0:1789760791.721649:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734931628754 time 734948182568 state 0x0 ref 1 00000100:00100000:1.0:1789760791.721667:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734948442984 time 734948182568 state 0x1 ref 1 00000100:00100000:1.0:1789760791.721696:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734948442984 time 734967314533 state 0x0 ref 1 00000100:00100000:1.0:1789760791.721708:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554863488:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760791.721756:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67076096, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x1ffd8 00010000:00010000:1.0:1789760791.721766:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c69fe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.721782:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:1.0:1789760791.721787:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.721794:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c69fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67076096->67080191] (req 67076096->67080191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.721807:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c69fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67076096->67080191] (req 67076096->67080191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.730239:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983685 is committed 00002000:00010000:1.0:1789760791.730356:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67076096 -> 67080192 00002000:00010000:1.0:1789760791.730362:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131032 -> 131040 00010000:00010000:1.0:1789760791.730410:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c69fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67076096->67080191] (req 67076096->67080191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.730431:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c69fe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67076096->67080191] (req 67076096->67080191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.730441:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c69fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67076096->67080191] (req 67076096->67080191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.730449:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760791.730454:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c69fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67076096->67080191] (req 67076096->67080191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.730469:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c69fe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67076096->67080191] (req 67076096->67080191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.730488:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c69fe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67076096->67080191] (req 67076096->67080191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.730495:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c69fe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67076096->67080191] (req 67076096->67080191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.730955:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554863488:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9241us (9484us total) trans 4294983685 rc 0/0 00000100:00100000:1.0:1789760791.730966:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16387 00000100:00100000:1.0:1789760791.730976:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734948442984 time 734967314533 state 0x2 ref 0 00000100:00100000:0.0:1789760791.736459:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760791.736716:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554863616 00000100:00100000:2.0:1789760791.736726:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554863616 00000100:00100000:2.0:1789760791.736782:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734948442984 time 734967314533 state 0x0 ref 1 00000100:00100000:2.0:1789760791.736800:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734967574949 time 734967314533 state 0x1 ref 1 00000100:00100000:2.0:1789760791.736832:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734967574949 time 734982451047 state 0x0 ref 1 00000100:00100000:2.0:1789760791.736846:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554863616:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760791.736890:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67080192, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x1ffe0 00010000:00010000:2.0:1789760791.736900:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6a05 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.736915:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789760791.736920:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.736935:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6a05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67080192->67084287] (req 67080192->67084287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.736951:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6a05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67080192->67084287] (req 67080192->67084287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.742746:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983686 is committed 00002000:00010000:2.0:1789760791.742925:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67080192 -> 67084288 00002000:00010000:2.0:1789760791.743046:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131040 -> 131048 00010000:00010000:2.0:1789760791.743060:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6a05 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67080192->67084287] (req 67080192->67084287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.743071:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6a05 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67080192->67084287] (req 67080192->67084287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.743080:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6a05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67080192->67084287] (req 67080192->67084287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.743087:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760791.743092:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6a05 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67080192->67084287] (req 67080192->67084287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.743100:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6a05 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67080192->67084287] (req 67080192->67084287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.743118:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6a05 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67080192->67084287] (req 67080192->67084287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.743129:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6a05 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67080192->67084287] (req 67080192->67084287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.743272:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554863616:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6423us (6816us total) trans 4294983686 rc 0/0 00000100:00100000:2.0:1789760791.743282:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16388 00000100:00100000:2.0:1789760791.743292:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734967574949 time 734982451047 state 0x2 ref 0 00000100:00100000:2.0:1789760791.748770:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760791.748875:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554863744 00000100:00100000:3.0:1789760791.748886:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554863744 00000100:00100000:3.0:1789760791.748939:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734967574949 time 734982451047 state 0x0 ref 1 00000100:00100000:3.0:1789760791.748955:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734982711463 time 734982451047 state 0x1 ref 1 00000100:00100000:3.0:1789760791.749040:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734982711463 time 734994657390 state 0x0 ref 1 00000100:00100000:3.0:1789760791.749054:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554863744:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760791.749106:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67084288, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x1ffe8 00010000:00010000:3.0:1789760791.749117:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a0c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.749134:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=734s v=(0 0) 00010000:00010000:3.0:1789760791.749140:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.749146:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67084288->67088383] (req 67084288->67088383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.749160:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67084288->67088383] (req 67084288->67088383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.755140:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983687 is committed 00002000:00010000:3.0:1789760791.755282:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67084288 -> 67088384 00002000:00010000:3.0:1789760791.755288:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131048 -> 131056 00010000:00010000:3.0:1789760791.755300:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a0c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67084288->67088383] (req 67084288->67088383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.755314:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a0c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67084288->67088383] (req 67084288->67088383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.755324:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67084288->67088383] (req 67084288->67088383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.755331:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760791.755336:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a0c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67084288->67088383] (req 67084288->67088383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.755347:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a0c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67084288->67088383] (req 67084288->67088383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.756304:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a0c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67084288->67088383] (req 67084288->67088383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.756318:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a0c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67084288->67088383] (req 67084288->67088383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.757907:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554863744:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8845us (9136us total) trans 4294983687 rc 0/0 00000100:00100000:2.0:1789760791.757921:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16389 00000100:00100000:2.0:1789760791.757931:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734982711463 time 734994657390 state 0x2 ref 0 00000100:00100000:0.0:1789760791.765332:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760791.765843:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554863872 00000100:00100000:1.0:1789760791.765854:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554863872 00000100:00100000:1.0:1789760791.765910:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734982711463 time 734994657390 state 0x0 ref 1 00000100:00100000:1.0:1789760791.765924:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734994917806 time 734994657390 state 0x1 ref 1 00000100:00100000:1.0:1789760791.766297:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734994917806 time 735011914147 state 0x0 ref 1 00000100:00100000:1.0:1789760791.766313:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554863872:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760791.766359:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67088384, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x1fff0 00010000:00010000:1.0:1789760791.766556:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6a13 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.767118:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760791.767123:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.767130:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6a13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67088384->67092479] (req 67088384->67092479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.767150:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6a13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67088384->67092479] (req 67088384->67092479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.774513:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983688 is committed 00002000:00010000:1.0:1789760791.774775:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67088384 -> 67092480 00002000:00010000:1.0:1789760791.774781:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131056 -> 131064 00010000:00010000:1.0:1789760791.774795:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6a13 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67088384->67092479] (req 67088384->67092479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.774808:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6a13 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67088384->67092479] (req 67088384->67092479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.774818:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6a13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67088384->67092479] (req 67088384->67092479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.774825:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760791.774830:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6a13 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67088384->67092479] (req 67088384->67092479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.774839:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6a13 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67088384->67092479] (req 67088384->67092479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.774859:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6a13 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67088384->67092479] (req 67088384->67092479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.774867:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6a13 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67088384->67092479] (req 67088384->67092479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.774965:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554863872:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8650us (9637us total) trans 4294983688 rc 0/0 00000100:00100000:1.0:1789760791.774975:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16390 00000100:00100000:1.0:1789760791.774993:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734994917806 time 735011914147 state 0x2 ref 0 00000100:00100000:0.0:1789760791.782086:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760791.782186:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554864000 00000100:00100000:1.0:1789760791.782196:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554864000 00000100:00100000:1.0:1789760791.782249:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 734994917806 time 735011914147 state 0x0 ref 1 00000100:00100000:1.0:1789760791.782265:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735012174563 time 735011914147 state 0x1 ref 1 00000100:00100000:1.0:1789760791.782298:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735012174563 time 735027916954 state 0x0 ref 1 00000100:00100000:1.0:1789760791.782311:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554864000:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760791.782360:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67092480, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x1fff8 00010000:00010000:1.0:1789760791.782369:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6a1a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.782416:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760791.782421:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.782428:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6a1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67092480->67096575] (req 67092480->67096575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.782442:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6a1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67092480->67096575] (req 67092480->67096575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.788120:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983689 is committed 00002000:00010000:1.0:1789760791.788237:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67092480 -> 67096576 00002000:00010000:1.0:1789760791.788243:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131064 -> 131072 00010000:00010000:1.0:1789760791.788256:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6a1a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67092480->67096575] (req 67092480->67096575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.788269:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6a1a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67092480->67096575] (req 67092480->67096575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.788279:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6a1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67092480->67096575] (req 67092480->67096575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.788287:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760791.788293:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6a1a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67092480->67096575] (req 67092480->67096575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.788302:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6a1a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67092480->67096575] (req 67092480->67096575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.788321:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6a1a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67092480->67096575] (req 67092480->67096575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.788336:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6a1a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67092480->67096575] (req 67092480->67096575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.788481:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554864000:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6167us (6401us total) trans 4294983689 rc 0/0 00000100:00100000:1.0:1789760791.788493:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16391 00000100:00100000:1.0:1789760791.788503:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735012174563 time 735027916954 state 0x2 ref 0 00000100:00100000:2.0:1789760791.793674:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760791.793788:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554864128 00000100:00100000:0.0:1789760791.793797:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554864128 00000100:00100000:0.0:1789760791.793852:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735012174563 time 735027916954 state 0x0 ref 1 00000100:00100000:0.0:1789760791.793868:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735028177370 time 735027916954 state 0x1 ref 1 00000100:00100000:0.0:1789760791.793901:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735028177370 time 735039519513 state 0x0 ref 1 00000100:00100000:0.0:1789760791.793913:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554864128:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760791.794067:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67096576, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20000 00010000:00010000:0.0:1789760791.794077:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a21 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.794092:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760791.794097:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.794104:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67096576->67100671] (req 67096576->67100671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.794118:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67096576->67100671] (req 67096576->67100671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.802243:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983690 is committed 00002000:00010000:0.0:1789760791.813800:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67096576 -> 67100672 00002000:00010000:0.0:1789760791.813811:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131072 -> 131080 00010000:00010000:0.0:1789760791.813825:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a21 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67096576->67100671] (req 67096576->67100671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.813838:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a21 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67096576->67100671] (req 67096576->67100671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.813845:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67096576->67100671] (req 67096576->67100671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.813852:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760791.813856:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a21 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67096576->67100671] (req 67096576->67100671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.813869:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a21 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67096576->67100671] (req 67096576->67100671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.813889:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a21 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67096576->67100671] (req 67096576->67100671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.813895:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a21 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67096576->67100671] (req 67096576->67100671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.814311:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554864128:12345-192.168.203.22@tcp:4:dd.0 Request processed in 20392us (20639us total) trans 4294983690 rc 0/0 00000100:00100000:0.0:1789760791.814322:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16392 00000100:00100000:0.0:1789760791.814331:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735028177370 time 735039519513 state 0x2 ref 0 00000100:00100000:0.0:1789760791.822271:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760791.824316:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554864256 00000100:00100000:2.0:1789760791.824330:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554864256 00000100:00100000:2.0:1789760791.824430:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735028177370 time 735039519513 state 0x0 ref 1 00000100:00100000:2.0:1789760791.824458:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735039779929 time 735039519513 state 0x1 ref 1 00000100:00100000:2.0:1789760791.824502:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735039779929 time 735070119616 state 0x0 ref 1 00000100:00100000:2.0:1789760791.824516:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554864256:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760791.824565:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67100672, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20008 00010000:00010000:2.0:1789760791.824575:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6a28 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.824603:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760791.824616:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.824623:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6a28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67100672->67104767] (req 67100672->67104767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.824638:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6a28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67100672->67104767] (req 67100672->67104767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.833417:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983691 is committed 00002000:00010000:2.0:1789760791.840997:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67100672 -> 67104768 00002000:00010000:2.0:1789760791.841005:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131080 -> 131088 00010000:00010000:2.0:1789760791.841018:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6a28 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67100672->67104767] (req 67100672->67104767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.841031:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6a28 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67100672->67104767] (req 67100672->67104767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.841042:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6a28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67100672->67104767] (req 67100672->67104767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.841050:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760791.841055:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6a28 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67100672->67104767] (req 67100672->67104767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.841064:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6a28 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67100672->67104767] (req 67100672->67104767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.841084:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6a28 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67100672->67104767] (req 67100672->67104767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.841092:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6a28 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67100672->67104767] (req 67100672->67104767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.841201:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554864256:12345-192.168.203.22@tcp:4:dd.0 Request processed in 16683us (18935us total) trans 4294983691 rc 0/0 00000100:00100000:2.0:1789760791.841211:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16393 00000100:00100000:2.0:1789760791.841222:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735039779929 time 735070119616 state 0x2 ref 0 00000100:00100000:2.0:1789760791.846670:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760791.846756:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554864384 00000100:00100000:1.0:1789760791.846768:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554864384 00000100:00100000:1.0:1789760791.846821:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735039779929 time 735070119616 state 0x0 ref 1 00000100:00100000:1.0:1789760791.846835:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735070380032 time 735070119616 state 0x1 ref 1 00000100:00100000:1.0:1789760791.846868:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735070380032 time 735092485755 state 0x0 ref 1 00000100:00100000:1.0:1789760791.846880:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554864384:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760791.846927:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67104768, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20010 00010000:00010000:1.0:1789760791.846937:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6a2f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.846954:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760791.846958:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.846965:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6a2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67104768->67108863] (req 67104768->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.846983:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6a2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67104768->67108863] (req 67104768->67108863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.851333:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983692 is committed 00002000:00010000:1.0:1789760791.851540:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67104768 -> 67108864 00002000:00010000:1.0:1789760791.851545:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131088 -> 131096 00010000:00010000:1.0:1789760791.851558:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6a2f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67104768->67108863] (req 67104768->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.851571:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6a2f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67104768->67108863] (req 67104768->67108863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.851580:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6a2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67104768->67108863] (req 67104768->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.851588:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655de00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760791.851592:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6a2f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67104768->67108863] (req 67104768->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.851602:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6a2f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67104768->67108863] (req 67104768->67108863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.855544:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6a2f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67104768->67108863] (req 67104768->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.855558:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6a2f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67104768->67108863] (req 67104768->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.855673:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554864384:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8790us (9008us total) trans 4294983692 rc 0/0 00000100:00100000:1.0:1789760791.855684:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16394 00000100:00100000:1.0:1789760791.855694:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735070380032 time 735092485755 state 0x2 ref 0 00000100:00100000:1.0:1789760791.859187:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760791.860165:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554864512 00000100:00100000:3.0:1789760791.860176:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554864512 00000100:00100000:3.0:1789760791.860233:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735070380032 time 735092485755 state 0x0 ref 1 00000100:00100000:3.0:1789760791.860248:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735092746171 time 735092485755 state 0x1 ref 1 00000100:00100000:3.0:1789760791.860278:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735092746171 time 735105897218 state 0x0 ref 1 00000100:00100000:3.0:1789760791.860289:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554864512:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760791.860570:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67108864, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20018 00010000:00010000:3.0:1789760791.860580:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a36 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.860596:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760791.860601:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.860618:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67108864->67112959] (req 67108864->67112959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.860633:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67108864->67112959] (req 67108864->67112959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.864822:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983693 is committed 00002000:00010000:3.0:1789760791.864944:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67108864 -> 67112960 00002000:00010000:3.0:1789760791.864950:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131096 -> 131104 00010000:00010000:3.0:1789760791.864961:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a36 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67108864->67112959] (req 67108864->67112959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.864975:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a36 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67108864->67112959] (req 67108864->67112959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.864996:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67108864->67112959] (req 67108864->67112959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.865005:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760791.865010:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a36 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67108864->67112959] (req 67108864->67112959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.865020:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a36 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67108864->67112959] (req 67108864->67112959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.865040:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a36 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67108864->67112959] (req 67108864->67112959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.865053:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a36 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67108864->67112959] (req 67108864->67112959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.865144:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554864512:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4851us (5961us total) trans 4294983693 rc 0/0 00000100:00100000:3.0:1789760791.865155:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16395 00000100:00100000:3.0:1789760791.865164:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735092746171 time 735105897218 state 0x2 ref 0 00000100:00100000:2.0:1789760791.873702:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760791.873815:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554864640 00000100:00100000:0.0:1789760791.873824:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554864640 00000100:00100000:0.0:1789760791.873880:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735092746171 time 735105897218 state 0x0 ref 1 00000100:00100000:0.0:1789760791.873897:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735106157634 time 735105897218 state 0x1 ref 1 00000100:00100000:0.0:1789760791.873923:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735106157634 time 735119542025 state 0x0 ref 1 00000100:00100000:0.0:1789760791.873948:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554864640:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760791.873993:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67112960, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20020 00010000:00010000:0.0:1789760791.874006:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a3d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.874021:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760791.874026:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.874032:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67112960->67117055] (req 67112960->67117055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.874045:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67112960->67117055] (req 67112960->67117055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.878722:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983694 is committed 00002000:00010000:0.0:1789760791.879021:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67112960 -> 67117056 00002000:00010000:0.0:1789760791.879027:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131104 -> 131112 00010000:00010000:0.0:1789760791.879040:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a3d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67112960->67117055] (req 67112960->67117055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.879053:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a3d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67112960->67117055] (req 67112960->67117055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.879062:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67112960->67117055] (req 67112960->67117055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.879068:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760791.879072:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a3d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67112960->67117055] (req 67112960->67117055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.879086:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a3d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67112960->67117055] (req 67112960->67117055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.879178:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a3d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67112960->67117055] (req 67112960->67117055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.879187:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a3d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67112960->67117055] (req 67112960->67117055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.879348:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554864640:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5396us (5649us total) trans 4294983694 rc 0/0 00000100:00100000:0.0:1789760791.879359:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16396 00000100:00100000:0.0:1789760791.879368:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735106157634 time 735119542025 state 0x2 ref 0 00000100:00100000:2.0:1789760791.890114:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760791.890337:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554864768 00000100:00100000:0.0:1789760791.890347:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554864768 00000100:00100000:0.0:1789760791.890443:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735106157634 time 735119542025 state 0x0 ref 1 00000100:00100000:0.0:1789760791.890459:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735119802441 time 735119542025 state 0x1 ref 1 00000100:00100000:0.0:1789760791.890487:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735119802441 time 735136105907 state 0x0 ref 1 00000100:00100000:0.0:1789760791.890499:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554864768:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760791.890546:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67117056, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20028 00010000:00010000:0.0:1789760791.890556:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a44 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.890573:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760791.890577:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.890584:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67117056->67121151] (req 67117056->67121151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.890597:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67117056->67121151] (req 67117056->67121151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.894915:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983695 is committed 00002000:00010000:0.0:1789760791.895314:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67117056 -> 67121152 00002000:00010000:0.0:1789760791.895319:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131112 -> 131120 00010000:00010000:0.0:1789760791.895394:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a44 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67117056->67121151] (req 67117056->67121151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.895409:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a44 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67117056->67121151] (req 67117056->67121151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.895429:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67117056->67121151] (req 67117056->67121151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.895437:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760791.895442:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a44 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67117056->67121151] (req 67117056->67121151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.895451:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a44 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67117056->67121151] (req 67117056->67121151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.895477:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a44 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67117056->67121151] (req 67117056->67121151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.895484:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6a44 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67117056->67121151] (req 67117056->67121151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.895575:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554864768:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5074us (5467us total) trans 4294983695 rc 0/0 00000100:00100000:0.0:1789760791.895586:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16397 00000100:00100000:0.0:1789760791.895594:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735119802441 time 735136105907 state 0x2 ref 0 00000100:00100000:1.0:1789760791.902333:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760791.902455:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554864896 00000100:00100000:2.0:1789760791.902466:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554864896 00000100:00100000:2.0:1789760791.902526:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735119802441 time 735136105907 state 0x0 ref 1 00000100:00100000:2.0:1789760791.902542:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735136366323 time 735136105907 state 0x1 ref 1 00000100:00100000:2.0:1789760791.902577:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735136366323 time 735148195508 state 0x0 ref 1 00000100:00100000:2.0:1789760791.902589:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a384e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554864896:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760791.902647:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67121152, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20030 00010000:00010000:2.0:1789760791.902657:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6a4b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.902674:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760791.902678:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.902684:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6a4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67121152->67125247] (req 67121152->67125247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.902698:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6a4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67121152->67125247] (req 67121152->67125247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.906992:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983696 is committed 00002000:00010000:2.0:1789760791.907480:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67121152 -> 67125248 00002000:00010000:2.0:1789760791.907485:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131120 -> 131128 00010000:00010000:2.0:1789760791.907498:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6a4b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67121152->67125247] (req 67121152->67125247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.907511:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6a4b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67121152->67125247] (req 67121152->67125247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.907520:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6a4b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67121152->67125247] (req 67121152->67125247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.907527:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760791.907532:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6a4b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67121152->67125247] (req 67121152->67125247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.907541:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6a4b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67121152->67125247] (req 67121152->67125247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.907560:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6a4b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67121152->67125247] (req 67121152->67125247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.907568:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6a4b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67121152->67125247] (req 67121152->67125247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.907674:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a384e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554864896:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5081us (5344us total) trans 4294983696 rc 0/0 00000100:00100000:2.0:1789760791.907685:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16398 00000100:00100000:2.0:1789760791.907694:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735136366323 time 735148195508 state 0x2 ref 0 00000100:00100000:2.0:1789760791.913161:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760791.913247:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554865024 00000100:00100000:3.0:1789760791.913258:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554865024 00000100:00100000:3.0:1789760791.913698:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735136366323 time 735148195508 state 0x0 ref 1 00000100:00100000:3.0:1789760791.913714:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735148455924 time 735148195508 state 0x1 ref 1 00000100:00100000:3.0:1789760791.913790:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735148455924 time 735159407411 state 0x0 ref 1 00000100:00100000:3.0:1789760791.913804:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554865024:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760791.913861:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67125248, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20038 00010000:00010000:3.0:1789760791.913871:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a52 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.913886:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760791.913890:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.913896:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67125248->67129343] (req 67125248->67129343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.913909:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67125248->67129343] (req 67125248->67129343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.918010:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983697 is committed 00002000:00010000:3.0:1789760791.918560:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67125248 -> 67129344 00002000:00010000:3.0:1789760791.918566:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131128 -> 131136 00010000:00010000:3.0:1789760791.918579:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a52 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67125248->67129343] (req 67125248->67129343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.918593:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a52 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67125248->67129343] (req 67125248->67129343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.918604:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a52 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67125248->67129343] (req 67125248->67129343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.918624:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760791.918629:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a52 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67125248->67129343] (req 67125248->67129343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.918640:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a52 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67125248->67129343] (req 67125248->67129343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.918660:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a52 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67125248->67129343] (req 67125248->67129343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.918669:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a52 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67125248->67129343] (req 67125248->67129343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760791.918810:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554865024:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5003us (5652us total) trans 4294983697 rc 0/0 00000100:00100000:3.0:1789760791.918822:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16399 00000100:00100000:3.0:1789760791.918832:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735148455924 time 735159407411 state 0x2 ref 0 00000100:00100000:0.0:1789760791.923731:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760791.923870:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554865152 00000100:00100000:1.0:1789760791.923879:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554865152 00000100:00100000:1.0:1789760791.923931:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735148455924 time 735159407411 state 0x0 ref 1 00000100:00100000:1.0:1789760791.923946:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735159667827 time 735159407411 state 0x1 ref 1 00000100:00100000:1.0:1789760791.924077:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735159667827 time 735169694045 state 0x0 ref 1 00000100:00100000:1.0:1789760791.924096:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554865152:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760791.924143:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67129344, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20040 00010000:00010000:1.0:1789760791.924152:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6a59 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.924168:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760791.924172:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.924179:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6a59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67129344->67133439] (req 67129344->67133439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.924199:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6a59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67129344->67133439] (req 67129344->67133439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760791.928313:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983698 is committed 00002000:00010000:1.0:1789760791.928444:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67129344 -> 67133440 00002000:00010000:1.0:1789760791.928449:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131136 -> 131144 00010000:00010000:1.0:1789760791.928461:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6a59 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67129344->67133439] (req 67129344->67133439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.928474:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6a59 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67129344->67133439] (req 67129344->67133439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.928484:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6a59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67129344->67133439] (req 67129344->67133439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.928491:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760791.928495:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6a59 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67129344->67133439] (req 67129344->67133439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.928504:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6a59 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67129344->67133439] (req 67129344->67133439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.928522:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6a59 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67129344->67133439] (req 67129344->67133439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.928529:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6a59 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67129344->67133439] (req 67129344->67133439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.928634:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554865152:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4541us (4908us total) trans 4294983698 rc 0/0 00000100:00100000:1.0:1789760791.928645:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16400 00000100:00100000:1.0:1789760791.928654:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735159667827 time 735169694045 state 0x2 ref 0 00000100:00100000:1.0:1789760791.932276:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760791.932776:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554865280 00000100:00100000:0.0:1789760791.932787:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554865280 00000100:00100000:0.0:1789760791.932840:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735159667827 time 735169694045 state 0x0 ref 1 00000100:00100000:0.0:1789760791.932856:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735169954461 time 735169694045 state 0x1 ref 1 00000100:00100000:0.0:1789760791.932937:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735169954461 time 735178554804 state 0x0 ref 1 00000100:00100000:0.0:1789760791.932950:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a386d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554865280:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760791.933011:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67133440, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20048 00010000:00010000:0.0:1789760791.933022:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6a60 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.933039:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760791.933043:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.933050:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6a60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67133440->67137535] (req 67133440->67137535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.933064:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6a60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67133440->67137535] (req 67133440->67137535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.940457:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983699 is committed 00002000:00010000:0.0:1789760791.941019:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67133440 -> 67137536 00002000:00010000:0.0:1789760791.941026:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131144 -> 131152 00010000:00010000:0.0:1789760791.941040:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6a60 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67133440->67137535] (req 67133440->67137535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.941053:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6a60 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67133440->67137535] (req 67133440->67137535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.941062:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6a60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67133440->67137535] (req 67133440->67137535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.941069:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760791.941074:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6a60 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67133440->67137535] (req 67133440->67137535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.941083:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6a60 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67133440->67137535] (req 67133440->67137535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.941106:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6a60 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67133440->67137535] (req 67133440->67137535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.941114:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6a60 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67133440->67137535] (req 67133440->67137535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.941198:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a386d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554865280:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8244us (8938us total) trans 4294983699 rc 0/0 00000100:00100000:0.0:1789760791.941208:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16401 00000100:00100000:0.0:1789760791.941218:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735169954461 time 735178554804 state 0x2 ref 0 00000100:00100000:2.0:1789760791.946183:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760791.946604:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554865408 00000100:00100000:1.0:1789760791.946624:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554865408 00000100:00100000:1.0:1789760791.946682:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735169954461 time 735178554804 state 0x0 ref 1 00000100:00100000:1.0:1789760791.946698:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735178815220 time 735178554804 state 0x1 ref 1 00000100:00100000:1.0:1789760791.946764:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735178815220 time 735192380914 state 0x0 ref 1 00000100:00100000:1.0:1789760791.946778:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554865408:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760791.946831:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67137536, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20050 00010000:00010000:1.0:1789760791.946841:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6a67 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.946858:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760791.946863:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.946870:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6a67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67137536->67141631] (req 67137536->67141631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.946885:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6a67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67137536->67141631] (req 67137536->67141631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.952245:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983700 is committed 00002000:00010000:1.0:1789760791.953793:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67137536 -> 67141632 00002000:00010000:1.0:1789760791.953802:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131152 -> 131160 00010000:00010000:1.0:1789760791.953817:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6a67 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67137536->67141631] (req 67137536->67141631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.953831:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6a67 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67137536->67141631] (req 67137536->67141631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.953840:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6a67 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67137536->67141631] (req 67137536->67141631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.953847:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760791.953852:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6a67 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67137536->67141631] (req 67137536->67141631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.953861:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6a67 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67137536->67141631] (req 67137536->67141631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.953882:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6a67 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67137536->67141631] (req 67137536->67141631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.953890:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6a67 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67137536->67141631] (req 67137536->67141631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760791.954088:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554865408:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7307us (7907us total) trans 4294983700 rc 0/0 00000100:00100000:1.0:1789760791.954105:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16402 00000100:00100000:1.0:1789760791.954115:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735178815220 time 735192380914 state 0x2 ref 0 00000100:00100000:2.0:1789760791.958782:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760791.958881:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554865536 00000100:00100000:3.0:1789760791.958890:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554865536 00000100:00100000:3.0:1789760791.958946:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735178815220 time 735192380914 state 0x0 ref 1 00000100:00100000:3.0:1789760791.958962:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735192641330 time 735192380914 state 0x1 ref 1 00000100:00100000:3.0:1789760791.959100:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735192641330 time 735204716698 state 0x0 ref 1 00000100:00100000:3.0:1789760791.959125:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554865536:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760791.959176:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67141632, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20058 00010000:00010000:3.0:1789760791.959187:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a6e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.959203:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760791.959220:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760791.959227:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67141632->67145727] (req 67141632->67145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760791.959242:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67141632->67145727] (req 67141632->67145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760791.963917:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983701 is committed 00002000:00010000:2.0:1789760791.964038:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67141632 -> 67145728 00002000:00010000:2.0:1789760791.964043:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131160 -> 131168 00010000:00010000:2.0:1789760791.964056:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a6e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67141632->67145727] (req 67141632->67145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.964069:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a6e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67141632->67145727] (req 67141632->67145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.964079:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a6e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67141632->67145727] (req 67141632->67145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.964091:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760791.964097:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a6e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67141632->67145727] (req 67141632->67145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.964106:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a6e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67141632->67145727] (req 67141632->67145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.964137:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a6e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67141632->67145727] (req 67141632->67145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.964146:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c6a6e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67141632->67145727] (req 67141632->67145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.964412:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554865536:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5264us (5603us total) trans 4294983701 rc 0/0 00000100:00100000:2.0:1789760791.964424:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16403 00000100:00100000:2.0:1789760791.964433:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735192641330 time 735204716698 state 0x2 ref 0 00000100:00100000:2.0:1789760791.968808:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760791.968976:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554865664 00000100:00100000:0.0:1789760791.968986:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554865664 00000100:00100000:0.0:1789760791.969042:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735192641330 time 735204716698 state 0x0 ref 1 00000100:00100000:0.0:1789760791.969057:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735204977114 time 735204716698 state 0x1 ref 1 00000100:00100000:0.0:1789760791.969089:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735204977114 time 735214707256 state 0x0 ref 1 00000100:00100000:0.0:1789760791.969101:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554865664:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760791.969161:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67145728, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20060 00010000:00010000:0.0:1789760791.969170:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6a75 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.969186:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760791.969190:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760791.969197:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6a75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67145728->67149823] (req 67145728->67149823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.969210:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6a75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67145728->67149823] (req 67145728->67149823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760791.978061:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983702 is committed 00002000:00010000:0.0:1789760791.978191:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67145728 -> 67149824 00002000:00010000:0.0:1789760791.978196:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131168 -> 131176 00010000:00010000:0.0:1789760791.978628:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6a75 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67145728->67149823] (req 67145728->67149823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.978643:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6a75 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67145728->67149823] (req 67145728->67149823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.978652:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6a75 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67145728->67149823] (req 67145728->67149823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.978659:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760791.978663:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6a75 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67145728->67149823] (req 67145728->67149823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.978671:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6a75 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67145728->67149823] (req 67145728->67149823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.978694:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6a75 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67145728->67149823] (req 67145728->67149823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760791.978707:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6a75 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67145728->67149823] (req 67145728->67149823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760791.978805:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554865664:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9702us (10003us total) trans 4294983702 rc 0/0 00000100:00100000:0.0:1789760791.978817:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16404 00000100:00100000:0.0:1789760791.978827:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735204977114 time 735214707256 state 0x2 ref 0 00000100:00100000:0.0:1789760791.983035:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760791.983421:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554865792 00000100:00100000:2.0:1789760791.983431:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554865792 00000100:00100000:2.0:1789760791.983486:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735204977114 time 735214707256 state 0x0 ref 1 00000100:00100000:2.0:1789760791.983502:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735214967672 time 735214707256 state 0x1 ref 1 00000100:00100000:2.0:1789760791.983533:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735214967672 time 735229152058 state 0x0 ref 1 00000100:00100000:2.0:1789760791.983545:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554865792:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760791.983591:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67149824, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20068 00010000:00010000:2.0:1789760791.983599:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a7c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.983771:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760791.983776:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760791.983784:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67149824->67153919] (req 67149824->67153919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.983799:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67149824->67153919] (req 67149824->67153919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760791.992659:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983703 is committed 00002000:00010000:2.0:1789760791.992762:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67149824 -> 67153920 00002000:00010000:2.0:1789760791.992767:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131176 -> 131184 00010000:00010000:2.0:1789760791.992780:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a7c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67149824->67153919] (req 67149824->67153919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.992793:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a7c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67149824->67153919] (req 67149824->67153919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.992803:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67149824->67153919] (req 67149824->67153919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.992816:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760791.992821:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a7c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67149824->67153919] (req 67149824->67153919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.992830:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a7c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67149824->67153919] (req 67149824->67153919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.992849:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a7c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67149824->67153919] (req 67149824->67153919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760791.992857:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a7c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67149824->67153919] (req 67149824->67153919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760791.992955:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554865792:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9407us (9927us total) trans 4294983703 rc 0/0 00000100:00100000:2.0:1789760791.992966:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16405 00000100:00100000:2.0:1789760791.992976:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735214967672 time 735229152058 state 0x2 ref 0 00000100:00100000:2.0:1789760791.998208:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760791.998558:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554865920 00000100:00100000:1.0:1789760791.998567:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554865920 00000100:00100000:1.0:1789760791.998682:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735214967672 time 735229152058 state 0x0 ref 1 00000100:00100000:1.0:1789760791.998699:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735229412474 time 735229152058 state 0x1 ref 1 00000100:00100000:1.0:1789760791.998773:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735229412474 time 735244391124 state 0x0 ref 1 00000100:00100000:1.0:1789760791.998787:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554865920:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760791.998829:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67153920, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20070 00010000:00010000:1.0:1789760791.998838:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6a83 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.998854:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760791.998858:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760791.998864:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6a83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67153920->67158015] (req 67153920->67158015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760791.998878:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6a83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67153920->67158015] (req 67153920->67158015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.003783:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983704 is committed 00002000:00010000:1.0:1789760792.003900:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67153920 -> 67158016 00002000:00010000:1.0:1789760792.003906:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131184 -> 131192 00010000:00010000:1.0:1789760792.003920:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6a83 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67153920->67158015] (req 67153920->67158015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.003932:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6a83 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67153920->67158015] (req 67153920->67158015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.003941:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6a83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67153920->67158015] (req 67153920->67158015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.003948:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760792.003953:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6a83 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67153920->67158015] (req 67153920->67158015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.003962:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6a83 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67153920->67158015] (req 67153920->67158015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.003989:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6a83 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67153920->67158015] (req 67153920->67158015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.003997:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6a83 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67153920->67158015] (req 67153920->67158015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.004081:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554865920:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5292us (5879us total) trans 4294983704 rc 0/0 00000100:00100000:1.0:1789760792.004091:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16406 00000100:00100000:1.0:1789760792.004101:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735229412474 time 735244391124 state 0x2 ref 0 00000100:00100000:1.0:1789760792.011772:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760792.011918:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554866048 00000100:00100000:2.0:1789760792.011928:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554866048 00000100:00100000:2.0:1789760792.011980:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735229412474 time 735244391124 state 0x0 ref 1 00000100:00100000:2.0:1789760792.012005:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735244651540 time 735244391124 state 0x1 ref 1 00000100:00100000:2.0:1789760792.012040:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735244651540 time 735257658189 state 0x0 ref 1 00000100:00100000:2.0:1789760792.012053:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554866048:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760792.012099:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67158016, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20078 00010000:00010000:2.0:1789760792.012110:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a8a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.012125:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760792.012130:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.012136:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67158016->67162111] (req 67158016->67162111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.012151:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67158016->67162111] (req 67158016->67162111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.017156:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983705 is committed 00002000:00010000:2.0:1789760792.017586:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67158016 -> 67162112 00002000:00010000:2.0:1789760792.017592:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131192 -> 131200 00010000:00010000:2.0:1789760792.017615:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a8a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67158016->67162111] (req 67158016->67162111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.017629:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a8a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67158016->67162111] (req 67158016->67162111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.017639:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67158016->67162111] (req 67158016->67162111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.017653:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760792.017658:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a8a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67158016->67162111] (req 67158016->67162111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.017668:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a8a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67158016->67162111] (req 67158016->67162111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.017688:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a8a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67158016->67162111] (req 67158016->67162111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.017697:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a8a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67158016->67162111] (req 67158016->67162111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.017839:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554866048:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5783us (6070us total) trans 4294983705 rc 0/0 00000100:00100000:2.0:1789760792.017850:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16407 00000100:00100000:2.0:1789760792.017860:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735244651540 time 735257658189 state 0x2 ref 0 00000100:00100000:0.0:1789760792.022927:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760792.023092:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554866176 00000100:00100000:1.0:1789760792.023103:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554866176 00000100:00100000:1.0:1789760792.023156:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735244651540 time 735257658189 state 0x0 ref 1 00000100:00100000:1.0:1789760792.023172:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735257918605 time 735257658189 state 0x1 ref 1 00000100:00100000:1.0:1789760792.023237:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735257918605 time 735268855207 state 0x0 ref 1 00000100:00100000:1.0:1789760792.023250:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554866176:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760792.023295:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67162112, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20080 00010000:00010000:1.0:1789760792.023304:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6a91 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.023321:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760792.023335:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.023342:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6a91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67162112->67166207] (req 67162112->67166207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.023356:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6a91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67162112->67166207] (req 67162112->67166207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.028236:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983706 is committed 00002000:00010000:0.0:1789760792.028374:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67162112 -> 67166208 00002000:00010000:0.0:1789760792.028425:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131200 -> 131208 00010000:00010000:0.0:1789760792.028439:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6a91 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67162112->67166207] (req 67162112->67166207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.028451:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6a91 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67162112->67166207] (req 67162112->67166207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.028461:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6a91 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67162112->67166207] (req 67162112->67166207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.028469:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760792.028474:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6a91 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67162112->67166207] (req 67162112->67166207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.028489:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6a91 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67162112->67166207] (req 67162112->67166207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.028510:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6a91 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67162112->67166207] (req 67162112->67166207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.028518:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6a91 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67162112->67166207] (req 67162112->67166207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.028597:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554866176:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5345us (5675us total) trans 4294983706 rc 0/0 00000100:00100000:0.0:1789760792.028618:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16408 00000100:00100000:0.0:1789760792.028628:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735257918605 time 735268855207 state 0x2 ref 0 00000100:00100000:0.0:1789760792.039922:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760792.040028:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554866304 00000100:00100000:2.0:1789760792.040036:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554866304 00000100:00100000:2.0:1789760792.040092:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735257918605 time 735268855207 state 0x0 ref 1 00000100:00100000:2.0:1789760792.040107:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735269115623 time 735268855207 state 0x1 ref 1 00000100:00100000:2.0:1789760792.040142:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735269115623 time 735285760294 state 0x0 ref 1 00000100:00100000:2.0:1789760792.040158:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554866304:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760792.040503:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67166208, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20088 00010000:00010000:2.0:1789760792.040514:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a98 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.040530:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760792.040535:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.040542:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67166208->67170303] (req 67166208->67170303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.040556:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67166208->67170303] (req 67166208->67170303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.045632:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983707 is committed 00002000:00010000:2.0:1789760792.045988:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67166208 -> 67170304 00002000:00010000:2.0:1789760792.045993:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131208 -> 131216 00010000:00010000:2.0:1789760792.046006:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a98 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67166208->67170303] (req 67166208->67170303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.046020:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a98 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67166208->67170303] (req 67166208->67170303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.046029:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67166208->67170303] (req 67166208->67170303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.046036:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760792.046041:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a98 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67166208->67170303] (req 67166208->67170303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.046049:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a98 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67166208->67170303] (req 67166208->67170303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.046068:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a98 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67166208->67170303] (req 67166208->67170303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.046077:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6a98 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67166208->67170303] (req 67166208->67170303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.046221:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554866304:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6060us (6304us total) trans 4294983707 rc 0/0 00000100:00100000:2.0:1789760792.046233:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16409 00000100:00100000:2.0:1789760792.046243:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735269115623 time 735285760294 state 0x2 ref 0 00000100:00100000:1.0:1789760792.050511:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760792.050710:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554866432 00000100:00100000:2.0:1789760792.050730:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554866432 00000100:00100000:2.0:1789760792.050781:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735269115623 time 735285760294 state 0x0 ref 1 00000100:00100000:2.0:1789760792.050796:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735286020710 time 735285760294 state 0x1 ref 1 00000100:00100000:2.0:1789760792.050923:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735286020710 time 735296446108 state 0x0 ref 1 00000100:00100000:2.0:1789760792.050938:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a387480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554866432:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760792.050985:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67170304, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20090 00010000:00010000:2.0:1789760792.050995:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6a9f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.051011:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760792.051016:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.051022:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6a9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67170304->67174399] (req 67170304->67174399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.051036:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6a9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67170304->67174399] (req 67170304->67174399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.055428:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983708 is committed 00002000:00010000:1.0:1789760792.055751:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67170304 -> 67174400 00002000:00010000:1.0:1789760792.055757:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131216 -> 131224 00010000:00010000:1.0:1789760792.055770:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6a9f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67170304->67174399] (req 67170304->67174399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.055783:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6a9f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67170304->67174399] (req 67170304->67174399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.055793:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6a9f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67170304->67174399] (req 67170304->67174399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.055800:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760792.055805:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6a9f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67170304->67174399] (req 67170304->67174399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.055814:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6a9f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67170304->67174399] (req 67170304->67174399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.055835:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6a9f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67170304->67174399] (req 67170304->67174399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.056749:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6a9f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67170304->67174399] (req 67170304->67174399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.056856:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a387480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554866432:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5915us (6350us total) trans 4294983708 rc 0/0 00000100:00100000:1.0:1789760792.056868:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16410 00000100:00100000:1.0:1789760792.056878:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735286020710 time 735296446108 state 0x2 ref 0 00000100:00100000:0.0:1789760792.062357:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760792.062539:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554866560 00000100:00100000:1.0:1789760792.062550:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554866560 00000100:00100000:1.0:1789760792.062605:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735286020710 time 735296446108 state 0x0 ref 1 00000100:00100000:1.0:1789760792.062631:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735296706524 time 735296446108 state 0x1 ref 1 00000100:00100000:1.0:1789760792.062663:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735296706524 time 735308280878 state 0x0 ref 1 00000100:00100000:1.0:1789760792.062677:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc40fb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554866560:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760792.062723:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67174400, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20098 00010000:00010000:1.0:1789760792.062732:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6aa6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.062750:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760792.062754:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.062760:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6aa6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67174400->67178495] (req 67174400->67178495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.062773:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6aa6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67174400->67178495] (req 67174400->67178495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.067506:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983709 is committed 00002000:00010000:1.0:1789760792.067597:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67174400 -> 67178496 00002000:00010000:1.0:1789760792.067601:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131224 -> 131232 00010000:00010000:1.0:1789760792.067624:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6aa6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67174400->67178495] (req 67174400->67178495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.067645:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6aa6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67174400->67178495] (req 67174400->67178495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.067654:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6aa6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67174400->67178495] (req 67174400->67178495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.067662:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760792.067667:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6aa6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67174400->67178495] (req 67174400->67178495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.067676:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6aa6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67174400->67178495] (req 67174400->67178495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.067695:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6aa6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67174400->67178495] (req 67174400->67178495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.067703:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6aa6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67174400->67178495] (req 67174400->67178495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.067802:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc40fb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554866560:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5123us (5451us total) trans 4294983709 rc 0/0 00000100:00100000:1.0:1789760792.067813:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16411 00000100:00100000:1.0:1789760792.067822:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735296706524 time 735308280878 state 0x2 ref 0 00000100:00100000:0.0:1789760792.071951:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760792.072015:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554866688 00000100:00100000:1.0:1789760792.072026:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554866688 00000100:00100000:1.0:1789760792.072081:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735296706524 time 735308280878 state 0x0 ref 1 00000100:00100000:1.0:1789760792.072098:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735308541294 time 735308280878 state 0x1 ref 1 00000100:00100000:1.0:1789760792.072132:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735308541294 time 735317750507 state 0x0 ref 1 00000100:00100000:1.0:1789760792.072144:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e93800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554866688:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760792.072235:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67178496, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x200a0 00010000:00010000:1.0:1789760792.072244:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6aad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.072260:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760792.072265:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.072271:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6aad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67178496->67182591] (req 67178496->67182591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.072286:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6aad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67178496->67182591] (req 67178496->67182591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.080873:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983710 is committed 00002000:00010000:0.0:1789760792.081046:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67178496 -> 67182592 00002000:00010000:0.0:1789760792.081052:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131232 -> 131240 00010000:00010000:0.0:1789760792.081066:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6aad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67178496->67182591] (req 67178496->67182591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.081079:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6aad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67178496->67182591] (req 67178496->67182591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.081096:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6aad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67178496->67182591] (req 67178496->67182591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.081105:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760792.081109:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6aad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67178496->67182591] (req 67178496->67182591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.081118:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6aad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67178496->67182591] (req 67178496->67182591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.081136:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6aad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67178496->67182591] (req 67178496->67182591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.081144:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6aad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67178496->67182591] (req 67178496->67182591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.081257:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e93800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554866688:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9109us (9311us total) trans 4294983710 rc 0/0 00000100:00100000:0.0:1789760792.081267:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16412 00000100:00100000:0.0:1789760792.081276:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735308541294 time 735317750507 state 0x2 ref 0 00000100:00100000:2.0:1789760792.086973:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760792.087078:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554866816 00000100:00100000:0.0:1789760792.087090:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554866816 00000100:00100000:0.0:1789760792.087141:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735308541294 time 735317750507 state 0x0 ref 1 00000100:00100000:0.0:1789760792.087156:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735318010923 time 735317750507 state 0x1 ref 1 00000100:00100000:0.0:1789760792.087241:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735318010923 time 735332858394 state 0x0 ref 1 00000100:00100000:0.0:1789760792.087254:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554866816:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760792.087302:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67182592, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x200a8 00010000:00010000:0.0:1789760792.087310:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ab4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.087337:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760792.087342:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.087349:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ab4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67182592->67186687] (req 67182592->67186687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.087363:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ab4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67182592->67186687] (req 67182592->67186687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.095649:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983711 is committed 00002000:00010000:0.0:1789760792.096074:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67182592 -> 67186688 00002000:00010000:0.0:1789760792.096080:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131240 -> 131248 00010000:00010000:0.0:1789760792.096094:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ab4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67182592->67186687] (req 67182592->67186687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.096108:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ab4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67182592->67186687] (req 67182592->67186687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.096118:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ab4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67182592->67186687] (req 67182592->67186687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.096132:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760792.096137:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ab4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67182592->67186687] (req 67182592->67186687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.096146:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ab4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67182592->67186687] (req 67182592->67186687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.096167:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ab4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67182592->67186687] (req 67182592->67186687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.096175:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ab4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67182592->67186687] (req 67182592->67186687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.096849:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554866816:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9590us (9879us total) trans 4294983711 rc 0/0 00000100:00100000:0.0:1789760792.096861:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16413 00000100:00100000:0.0:1789760792.096870:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735318010923 time 735332858394 state 0x2 ref 0 00000100:00100000:2.0:1789760792.101989:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760792.102132:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554866944 00000100:00100000:1.0:1789760792.102141:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554866944 00000100:00100000:1.0:1789760792.102196:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735318010923 time 735332858394 state 0x0 ref 1 00000100:00100000:1.0:1789760792.102211:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735333118810 time 735332858394 state 0x1 ref 1 00000100:00100000:1.0:1789760792.102243:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735333118810 time 735347861615 state 0x0 ref 1 00000100:00100000:1.0:1789760792.102255:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554866944:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760792.102303:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67186688, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x200b0 00010000:00010000:1.0:1789760792.102312:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6abb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.102328:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760792.102332:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.102337:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6abb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67186688->67190783] (req 67186688->67190783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.102350:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6abb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67186688->67190783] (req 67186688->67190783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.107854:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983712 is committed 00002000:00010000:1.0:1789760792.108307:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67186688 -> 67190784 00002000:00010000:1.0:1789760792.108312:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131248 -> 131256 00010000:00010000:1.0:1789760792.108335:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6abb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67186688->67190783] (req 67186688->67190783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.108348:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6abb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67186688->67190783] (req 67186688->67190783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.108358:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6abb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67186688->67190783] (req 67186688->67190783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.108370:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760792.108374:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6abb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67186688->67190783] (req 67186688->67190783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.108514:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6abb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67186688->67190783] (req 67186688->67190783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.108535:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6abb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67186688->67190783] (req 67186688->67190783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.108543:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6abb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67186688->67190783] (req 67186688->67190783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.108679:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554866944:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6421us (6702us total) trans 4294983712 rc 0/0 00000100:00100000:1.0:1789760792.108697:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16414 00000100:00100000:1.0:1789760792.108706:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735333118810 time 735347861615 state 0x2 ref 0 00000100:00100000:2.0:1789760792.112322:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760792.112520:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554867072 00000100:00100000:1.0:1789760792.112530:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554867072 00000100:00100000:0.0:1789760792.112664:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735333118810 time 735347861615 state 0x0 ref 1 00000100:00100000:0.0:1789760792.112678:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735348122031 time 735347861615 state 0x1 ref 1 00000100:00100000:0.0:1789760792.113277:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735348122031 time 735358893279 state 0x0 ref 1 00000100:00100000:0.0:1789760792.113293:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554867072:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760792.113339:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67190784, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x200b8 00010000:00010000:0.0:1789760792.113349:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ac2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.113374:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760792.113408:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.113415:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ac2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67190784->67194879] (req 67190784->67194879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.113429:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ac2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67190784->67194879] (req 67190784->67194879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.118742:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983713 is committed 00002000:00010000:0.0:1789760792.118878:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67190784 -> 67194880 00002000:00010000:0.0:1789760792.118962:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131256 -> 131264 00010000:00010000:0.0:1789760792.118975:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ac2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67190784->67194879] (req 67190784->67194879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.119044:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ac2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67190784->67194879] (req 67190784->67194879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.119055:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ac2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67190784->67194879] (req 67190784->67194879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.119063:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760792.119068:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ac2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67190784->67194879] (req 67190784->67194879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.119083:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ac2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67190784->67194879] (req 67190784->67194879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.119103:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ac2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67190784->67194879] (req 67190784->67194879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.119111:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ac2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67190784->67194879] (req 67190784->67194879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.119252:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554867072:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5957us (6934us total) trans 4294983713 rc 0/0 00000100:00100000:0.0:1789760792.119262:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16415 00000100:00100000:0.0:1789760792.119272:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735348122031 time 735358893279 state 0x2 ref 0 00000100:00100000:2.0:1789760792.130328:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760792.131218:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554867200 00000100:00100000:0.0:1789760792.131231:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554867200 00000100:00100000:0.0:1789760792.131291:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735348122031 time 735358893279 state 0x0 ref 1 00000100:00100000:0.0:1789760792.131308:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735359153695 time 735358893279 state 0x1 ref 1 00000100:00100000:0.0:1789760792.132127:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735359153695 time 735377740732 state 0x0 ref 1 00000100:00100000:0.0:1789760792.132146:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554867200:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760792.132194:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67194880, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x200c0 00010000:00010000:0.0:1789760792.132203:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ac9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.132219:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760792.132223:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.132230:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ac9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67194880->67198975] (req 67194880->67198975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.132244:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ac9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67194880->67198975] (req 67194880->67198975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.141286:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983714 is committed 00002000:00010000:0.0:1789760792.141410:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67194880 -> 67198976 00002000:00010000:0.0:1789760792.141415:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131264 -> 131272 00010000:00010000:0.0:1789760792.141429:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ac9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67194880->67198975] (req 67194880->67198975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.141440:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ac9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67194880->67198975] (req 67194880->67198975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.141450:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ac9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67194880->67198975] (req 67194880->67198975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.141458:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760792.141464:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ac9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67194880->67198975] (req 67194880->67198975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.141473:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ac9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67194880->67198975] (req 67194880->67198975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.141497:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ac9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67194880->67198975] (req 67194880->67198975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.141504:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ac9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67194880->67198975] (req 67194880->67198975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.141591:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554867200:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9446us (11332us total) trans 4294983714 rc 0/0 00000100:00100000:0.0:1789760792.141601:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16416 00000100:00100000:0.0:1789760792.141619:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735359153695 time 735377740732 state 0x2 ref 0 00010000:00010000:3.0:1789760792.145666:0:6508:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789760792.148086:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760792.148171:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554867328 00000100:00100000:0.0:1789760792.148179:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554867328 00000100:00100000:0.0:1789760792.148233:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735359153695 time 735377740732 state 0x0 ref 1 00000100:00100000:0.0:1789760792.148248:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735378001148 time 735377740732 state 0x1 ref 1 00000100:00100000:0.0:1789760792.148278:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735378001148 time 735393897428 state 0x0 ref 1 00000100:00100000:0.0:1789760792.148290:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554867328:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760792.148334:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67198976, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x200c8 00010000:00010000:0.0:1789760792.148343:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ad0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.148358:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760792.148372:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.148409:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ad0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67198976->67203071] (req 67198976->67203071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.148423:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ad0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67198976->67203071] (req 67198976->67203071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.157293:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983715 is committed 00002000:00010000:0.0:1789760792.157555:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67198976 -> 67203072 00002000:00010000:0.0:1789760792.157560:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131272 -> 131280 00010000:00010000:0.0:1789760792.157573:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ad0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67198976->67203071] (req 67198976->67203071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.157585:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ad0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67198976->67203071] (req 67198976->67203071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.157595:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ad0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67198976->67203071] (req 67198976->67203071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.157602:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760792.157615:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ad0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67198976->67203071] (req 67198976->67203071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.157625:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ad0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67198976->67203071] (req 67198976->67203071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.157642:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ad0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67198976->67203071] (req 67198976->67203071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.157654:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ad0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67198976->67203071] (req 67198976->67203071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.157776:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554867328:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9484us (9696us total) trans 4294983715 rc 0/0 00000100:00100000:0.0:1789760792.157787:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16417 00000100:00100000:0.0:1789760792.157796:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735378001148 time 735393897428 state 0x2 ref 0 00000100:00100000:2.0:1789760792.162347:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760792.162493:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554867456 00000100:00100000:1.0:1789760792.162502:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554867456 00000100:00100000:1.0:1789760792.162556:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735378001148 time 735393897428 state 0x0 ref 1 00000100:00100000:1.0:1789760792.162570:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735394157844 time 735393897428 state 0x1 ref 1 00000100:00100000:1.0:1789760792.162598:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735394157844 time 735408216847 state 0x0 ref 1 00000100:00100000:1.0:1789760792.162621:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554867456:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760792.162663:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67203072, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x200d0 00010000:00010000:1.0:1789760792.162671:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6ad7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.162690:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760792.162694:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.162699:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6ad7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67203072->67207167] (req 67203072->67207167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.162710:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6ad7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67203072->67207167] (req 67203072->67207167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.168225:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983716 is committed 00002000:00010000:1.0:1789760792.168903:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67203072 -> 67207168 00002000:00010000:1.0:1789760792.168910:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131280 -> 131288 00010000:00010000:1.0:1789760792.168922:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6ad7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67203072->67207167] (req 67203072->67207167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.168935:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6ad7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67203072->67207167] (req 67203072->67207167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.168943:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6ad7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67203072->67207167] (req 67203072->67207167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.168957:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760792.168962:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6ad7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67203072->67207167] (req 67203072->67207167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.168972:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6ad7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67203072->67207167] (req 67203072->67207167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.168991:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6ad7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67203072->67207167] (req 67203072->67207167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.169009:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6ad7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67203072->67207167] (req 67203072->67207167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.169105:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554867456:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6492us (6761us total) trans 4294983716 rc 0/0 00000100:00100000:1.0:1789760792.169114:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16418 00000100:00100000:1.0:1789760792.169124:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735394157844 time 735408216847 state 0x2 ref 0 00000100:00100000:2.0:1789760792.172715:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760792.172859:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554867584 00000100:00100000:0.0:1789760792.172870:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554867584 00000100:00100000:0.0:1789760792.172926:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735394157844 time 735408216847 state 0x0 ref 1 00000100:00100000:0.0:1789760792.172940:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735408477263 time 735408216847 state 0x1 ref 1 00000100:00100000:0.0:1789760792.172973:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735408477263 time 735418588431 state 0x0 ref 1 00000100:00100000:0.0:1789760792.172999:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554867584:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760792.173048:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67207168, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x200d8 00010000:00010000:0.0:1789760792.173057:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ade lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.173072:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760792.173077:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.173082:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ade lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67207168->67211263] (req 67207168->67211263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.173099:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ade lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67207168->67211263] (req 67207168->67211263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.185798:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983717 is committed 00002000:00010000:0.0:1789760792.185953:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67207168 -> 67211264 00002000:00010000:0.0:1789760792.185958:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131288 -> 131296 00010000:00010000:0.0:1789760792.185971:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ade lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67207168->67211263] (req 67207168->67211263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.186044:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ade lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67207168->67211263] (req 67207168->67211263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.186054:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ade lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67207168->67211263] (req 67207168->67211263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.186061:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760792.186066:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ade lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67207168->67211263] (req 67207168->67211263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.186075:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ade lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67207168->67211263] (req 67207168->67211263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.186095:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ade lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67207168->67211263] (req 67207168->67211263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.186103:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6ade lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67207168->67211263] (req 67207168->67211263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.186274:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554867584:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13283us (13562us total) trans 4294983717 rc 0/0 00000100:00100000:0.0:1789760792.186284:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16419 00000100:00100000:0.0:1789760792.186294:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735408477263 time 735418588431 state 0x2 ref 0 00000100:00100000:2.0:1789760792.191324:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760792.192083:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554867712 00000100:00100000:0.0:1789760792.192094:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554867712 00000100:00100000:1.0:1789760792.192344:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735408477263 time 735418588431 state 0x0 ref 1 00000100:00100000:1.0:1789760792.192499:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735418848847 time 735418588431 state 0x1 ref 1 00000100:00100000:1.0:1789760792.192574:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735418848847 time 735438192763 state 0x0 ref 1 00000100:00100000:1.0:1789760792.192589:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554867712:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760792.192757:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67211264, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x200e0 00010000:00010000:1.0:1789760792.192767:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6ae5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.192784:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760792.192788:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.192794:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6ae5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67211264->67215359] (req 67211264->67215359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.192807:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6ae5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67211264->67215359] (req 67211264->67215359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.200927:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983718 is committed 00002000:00010000:1.0:1789760792.201134:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67211264 -> 67215360 00002000:00010000:1.0:1789760792.201138:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131296 -> 131304 00010000:00010000:1.0:1789760792.201149:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6ae5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67211264->67215359] (req 67211264->67215359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.201158:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6ae5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67211264->67215359] (req 67211264->67215359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.201164:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6ae5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67211264->67215359] (req 67211264->67215359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.201169:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760792.201172:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6ae5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67211264->67215359] (req 67211264->67215359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.201183:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6ae5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67211264->67215359] (req 67211264->67215359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.201197:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6ae5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67211264->67215359] (req 67211264->67215359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.201202:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6ae5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67211264->67215359] (req 67211264->67215359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.201282:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554867712:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8693us (9964us total) trans 4294983718 rc 0/0 00000100:00100000:1.0:1789760792.201289:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16420 00000100:00100000:1.0:1789760792.201296:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735418848847 time 735438192763 state 0x2 ref 0 00000100:00100000:2.0:1789760792.205891:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760792.206039:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554867840 00000100:00100000:0.0:1789760792.206048:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554867840 00000100:00100000:0.0:1789760792.206099:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735418848847 time 735438192763 state 0x0 ref 1 00000100:00100000:0.0:1789760792.206115:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735438453179 time 735438192763 state 0x1 ref 1 00000100:00100000:0.0:1789760792.206147:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735438453179 time 735451765447 state 0x0 ref 1 00000100:00100000:0.0:1789760792.206159:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554867840:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760792.206203:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67215360, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x200e8 00010000:00010000:0.0:1789760792.206212:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6aec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.206227:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760792.206232:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.206238:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6aec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67215360->67219455] (req 67215360->67219455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.206251:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6aec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67215360->67219455] (req 67215360->67219455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.216162:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983719 is committed 00002000:00010000:0.0:1789760792.216368:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67215360 -> 67219456 00002000:00010000:0.0:1789760792.216373:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131304 -> 131312 00010000:00010000:0.0:1789760792.216402:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6aec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67215360->67219455] (req 67215360->67219455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.216416:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6aec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67215360->67219455] (req 67215360->67219455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.216426:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6aec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67215360->67219455] (req 67215360->67219455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.216433:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760792.216438:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6aec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67215360->67219455] (req 67215360->67219455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.216446:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6aec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67215360->67219455] (req 67215360->67219455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.216464:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6aec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67215360->67219455] (req 67215360->67219455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.216471:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6aec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67215360->67219455] (req 67215360->67219455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.216626:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554867840:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10464us (10739us total) trans 4294983719 rc 0/0 00000100:00100000:0.0:1789760792.216644:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16421 00000100:00100000:0.0:1789760792.216653:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735438453179 time 735451765447 state 0x2 ref 0 00000100:00100000:2.0:1789760792.221235:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760792.221758:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554867968 00000100:00100000:1.0:1789760792.221768:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554867968 00000100:00100000:1.0:1789760792.221817:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735438453179 time 735451765447 state 0x0 ref 1 00000100:00100000:1.0:1789760792.221832:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735452025863 time 735451765447 state 0x1 ref 1 00000100:00100000:1.0:1789760792.221940:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735452025863 time 735467557407 state 0x0 ref 1 00000100:00100000:1.0:1789760792.221953:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554867968:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760792.222005:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67219456, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x200f0 00010000:00010000:1.0:1789760792.222014:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6af3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.222029:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760792.222033:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.222039:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6af3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67219456->67223551] (req 67219456->67223551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.222052:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6af3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67219456->67223551] (req 67219456->67223551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.228749:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983720 is committed 00002000:00010000:1.0:1789760792.228874:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67219456 -> 67223552 00002000:00010000:1.0:1789760792.228879:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131312 -> 131320 00010000:00010000:1.0:1789760792.228892:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6af3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67219456->67223551] (req 67219456->67223551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.228907:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6af3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67219456->67223551] (req 67219456->67223551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.228916:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6af3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67219456->67223551] (req 67219456->67223551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.228924:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760792.228929:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6af3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67219456->67223551] (req 67219456->67223551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.228938:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6af3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67219456->67223551] (req 67219456->67223551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.228963:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6af3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67219456->67223551] (req 67219456->67223551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.228972:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6af3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67219456->67223551] (req 67219456->67223551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.229869:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e5bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554867968:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7909us (8634us total) trans 4294983720 rc 0/0 00000100:00100000:1.0:1789760792.229881:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16422 00000100:00100000:1.0:1789760792.229891:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735452025863 time 735467557407 state 0x2 ref 0 00000100:00100000:2.0:1789760792.238960:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760792.240800:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554868096 00000100:00100000:0.0:1789760792.240813:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554868096 00000100:00100000:0.0:1789760792.240952:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735452025863 time 735467557407 state 0x0 ref 1 00000100:00100000:0.0:1789760792.240969:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735467817823 time 735467557407 state 0x1 ref 1 00000100:00100000:0.0:1789760792.241091:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735467817823 time 735486707588 state 0x0 ref 1 00000100:00100000:0.0:1789760792.241104:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554868096:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760792.241148:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67223552, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x200f8 00010000:00010000:0.0:1789760792.241157:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6afa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.241172:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760792.241176:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.241182:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6afa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67223552->67227647] (req 67223552->67227647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.241195:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6afa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67223552->67227647] (req 67223552->67227647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.246320:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983721 is committed 00002000:00010000:0.0:1789760792.246679:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67223552 -> 67227648 00002000:00010000:0.0:1789760792.246684:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131320 -> 131328 00010000:00010000:0.0:1789760792.246697:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6afa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67223552->67227647] (req 67223552->67227647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.246710:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6afa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67223552->67227647] (req 67223552->67227647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.246719:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6afa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67223552->67227647] (req 67223552->67227647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.246727:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760792.246732:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6afa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67223552->67227647] (req 67223552->67227647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.246742:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6afa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67223552->67227647] (req 67223552->67227647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.246761:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6afa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67223552->67227647] (req 67223552->67227647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.246769:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6afa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67223552->67227647] (req 67223552->67227647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.246867:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554868096:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5761us (7912us total) trans 4294983721 rc 0/0 00000100:00100000:0.0:1789760792.246878:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16423 00000100:00100000:0.0:1789760792.246888:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735467817823 time 735486707588 state 0x2 ref 0 00000100:00100000:2.0:1789760792.259184:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760792.259288:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554868224 00000100:00100000:1.0:1789760792.259298:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554868224 00000100:00100000:1.0:1789760792.259347:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735467817823 time 735486707588 state 0x0 ref 1 00000100:00100000:1.0:1789760792.259362:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735486968004 time 735486707588 state 0x1 ref 1 00000100:00100000:1.0:1789760792.259405:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735486968004 time 735505023577 state 0x0 ref 1 00000100:00100000:1.0:1789760792.259418:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554868224:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760792.259463:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67227648, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20100 00010000:00010000:1.0:1789760792.259472:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6b01 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.259495:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760792.259499:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.259506:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6b01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67227648->67231743] (req 67227648->67231743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.259520:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6b01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67227648->67231743] (req 67227648->67231743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.269206:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983722 is committed 00002000:00010000:0.0:1789760792.279142:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67227648 -> 67231744 00002000:00010000:0.0:1789760792.279152:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131328 -> 131336 00010000:00010000:0.0:1789760792.279164:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6b01 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67227648->67231743] (req 67227648->67231743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.279178:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6b01 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67227648->67231743] (req 67227648->67231743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.279187:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6b01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67227648->67231743] (req 67227648->67231743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.279194:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760792.279199:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6b01 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67227648->67231743] (req 67227648->67231743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.279208:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6b01 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67227648->67231743] (req 67227648->67231743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.279227:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6b01 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67227648->67231743] (req 67227648->67231743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.279234:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6b01 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67227648->67231743] (req 67227648->67231743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.279421:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554868224:12345-192.168.203.22@tcp:4:dd.0 Request processed in 19999us (20240us total) trans 4294983722 rc 0/0 00000100:00100000:0.0:1789760792.279436:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16424 00000100:00100000:0.0:1789760792.279445:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735486968004 time 735505023577 state 0x2 ref 0 00000100:00100000:2.0:1789760792.290967:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760792.291199:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554868352 00000100:00100000:3.0:1789760792.291210:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554868352 00000100:00100000:3.0:1789760792.291262:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735486968004 time 735505023577 state 0x0 ref 1 00000100:00100000:3.0:1789760792.291277:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735505283993 time 735505023577 state 0x1 ref 1 00000100:00100000:3.0:1789760792.291309:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735505283993 time 735536927970 state 0x0 ref 1 00000100:00100000:3.0:1789760792.291322:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe1a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554868352:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760792.291496:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67231744, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20108 00010000:00010000:3.0:1789760792.291507:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b08 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.291522:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760792.291526:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.291532:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67231744->67235839] (req 67231744->67235839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.291545:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67231744->67235839] (req 67231744->67235839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.303666:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983723 is committed 00002000:00010000:3.0:1789760792.303771:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67231744 -> 67235840 00002000:00010000:3.0:1789760792.303776:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131336 -> 131344 00010000:00010000:3.0:1789760792.303789:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b08 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67231744->67235839] (req 67231744->67235839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.303802:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b08 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67231744->67235839] (req 67231744->67235839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.303811:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67231744->67235839] (req 67231744->67235839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.303818:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760792.303832:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b08 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67231744->67235839] (req 67231744->67235839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.303842:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b08 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67231744->67235839] (req 67231744->67235839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.303861:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b08 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67231744->67235839] (req 67231744->67235839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.303870:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b08 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67231744->67235839] (req 67231744->67235839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.303959:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe1a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554868352:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12634us (12995us total) trans 4294983723 rc 0/0 00000100:00100000:3.0:1789760792.303970:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16425 00000100:00100000:3.0:1789760792.303986:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735505283993 time 735536927970 state 0x2 ref 0 00000100:00100000:3.0:1789760792.307732:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760792.308024:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554868480 00000100:00100000:2.0:1789760792.308042:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554868480 00000100:00100000:2.0:1789760792.308093:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735505283993 time 735536927970 state 0x0 ref 1 00000100:00100000:2.0:1789760792.308108:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735537188386 time 735536927970 state 0x1 ref 1 00000100:00100000:2.0:1789760792.308141:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735537188386 time 735553759857 state 0x0 ref 1 00000100:00100000:2.0:1789760792.308153:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554868480:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760792.308196:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67235840, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20110 00010000:00010000:2.0:1789760792.308205:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6b0f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.308219:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760792.308224:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.308230:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6b0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67235840->67239935] (req 67235840->67239935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.308241:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6b0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67235840->67239935] (req 67235840->67239935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.313579:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983724 is committed 00002000:00010000:2.0:1789760792.313702:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67235840 -> 67239936 00002000:00010000:2.0:1789760792.313707:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131344 -> 131352 00010000:00010000:2.0:1789760792.313720:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6b0f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67235840->67239935] (req 67235840->67239935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.313733:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6b0f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67235840->67239935] (req 67235840->67239935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.313742:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6b0f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67235840->67239935] (req 67235840->67239935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.313749:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760792.313754:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6b0f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67235840->67239935] (req 67235840->67239935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.313764:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6b0f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67235840->67239935] (req 67235840->67239935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.313783:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6b0f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67235840->67239935] (req 67235840->67239935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.313790:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6b0f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67235840->67239935] (req 67235840->67239935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.314204:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554868480:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6048us (6476us total) trans 4294983724 rc 0/0 00000100:00100000:2.0:1789760792.314215:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16426 00000100:00100000:2.0:1789760792.314224:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735537188386 time 735553759857 state 0x2 ref 0 00000100:00100000:2.0:1789760792.326873:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760792.326946:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554868608 00000100:00100000:0.0:1789760792.326957:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554868608 00000100:00100000:0.0:1789760792.327018:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735537188386 time 735553759857 state 0x0 ref 1 00000100:00100000:0.0:1789760792.327034:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735554020273 time 735553759857 state 0x1 ref 1 00000100:00100000:0.0:1789760792.327068:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735554020273 time 735572687108 state 0x0 ref 1 00000100:00100000:0.0:1789760792.327080:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554868608:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760792.327127:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67239936, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20118 00010000:00010000:0.0:1789760792.327136:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6b16 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.327152:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760792.327156:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.327163:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6b16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67239936->67244031] (req 67239936->67244031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.327177:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6b16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67239936->67244031] (req 67239936->67244031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.335243:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983725 is committed 00002000:00010000:0.0:1789760792.335351:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67239936 -> 67244032 00002000:00010000:0.0:1789760792.335356:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131352 -> 131360 00010000:00010000:0.0:1789760792.335446:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6b16 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67239936->67244031] (req 67239936->67244031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.335461:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6b16 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67239936->67244031] (req 67239936->67244031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.335471:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6b16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67239936->67244031] (req 67239936->67244031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.335478:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760792.335483:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6b16 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67239936->67244031] (req 67239936->67244031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.335491:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6b16 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67239936->67244031] (req 67239936->67244031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.335510:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6b16 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67239936->67244031] (req 67239936->67244031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.335518:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6b16 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67239936->67244031] (req 67239936->67244031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.335654:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554868608:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8572us (8789us total) trans 4294983725 rc 0/0 00000100:00100000:0.0:1789760792.335664:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16427 00000100:00100000:0.0:1789760792.335674:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735554020273 time 735572687108 state 0x2 ref 0 00000100:00100000:1.0:1789760792.344463:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760792.344554:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554868736 00000100:00100000:0.0:1789760792.344564:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554868736 00000100:00100000:0.0:1789760792.344761:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735554020273 time 735572687108 state 0x0 ref 1 00000100:00100000:0.0:1789760792.344777:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735572947524 time 735572687108 state 0x1 ref 1 00000100:00100000:0.0:1789760792.344921:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735572947524 time 735590537886 state 0x0 ref 1 00000100:00100000:0.0:1789760792.344935:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a387b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554868736:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760792.344997:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67244032, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20120 00010000:00010000:0.0:1789760792.345007:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6b1d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.345023:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760792.345027:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.345034:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6b1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67244032->67248127] (req 67244032->67248127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.345055:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6b1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67244032->67248127] (req 67244032->67248127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.350986:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983726 is committed 00002000:00010000:0.0:1789760792.351082:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67244032 -> 67248128 00002000:00010000:0.0:1789760792.351088:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131360 -> 131368 00010000:00010000:0.0:1789760792.351100:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6b1d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67244032->67248127] (req 67244032->67248127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.351119:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6b1d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67244032->67248127] (req 67244032->67248127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.351133:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6b1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67244032->67248127] (req 67244032->67248127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.351140:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760792.351145:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6b1d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67244032->67248127] (req 67244032->67248127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.351153:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6b1d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67244032->67248127] (req 67244032->67248127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.351178:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6b1d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67244032->67248127] (req 67244032->67248127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.351186:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c6b1d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67244032->67248127] (req 67244032->67248127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.351279:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a387b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554868736:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6343us (6917us total) trans 4294983726 rc 0/0 00000100:00100000:0.0:1789760792.351290:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16428 00000100:00100000:0.0:1789760792.351299:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735572947524 time 735590537886 state 0x2 ref 0 00000100:00100000:0.0:1789760792.354644:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760792.354721:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554868864 00000100:00100000:3.0:1789760792.354731:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554868864 00000100:00100000:3.0:1789760792.354781:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735572947524 time 735590537886 state 0x0 ref 1 00000100:00100000:3.0:1789760792.354792:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735590798302 time 735590537886 state 0x1 ref 1 00000100:00100000:3.0:1789760792.354814:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735590798302 time 735600434395 state 0x0 ref 1 00000100:00100000:3.0:1789760792.354824:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554868864:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760792.354860:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67248128, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20128 00010000:00010000:3.0:1789760792.354867:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b24 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.354879:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760792.354883:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.354889:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67248128->67252223] (req 67248128->67252223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.354903:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67248128->67252223] (req 67248128->67252223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.359600:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983727 is committed 00002000:00010000:3.0:1789760792.359755:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67248128 -> 67252224 00002000:00010000:3.0:1789760792.359761:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131368 -> 131376 00010000:00010000:3.0:1789760792.359772:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b24 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67248128->67252223] (req 67248128->67252223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.359785:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b24 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67248128->67252223] (req 67248128->67252223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.359794:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67248128->67252223] (req 67248128->67252223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.359802:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760792.359806:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b24 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67248128->67252223] (req 67248128->67252223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.359816:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b24 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67248128->67252223] (req 67248128->67252223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.359834:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b24 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67248128->67252223] (req 67248128->67252223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.359842:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b24 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67248128->67252223] (req 67248128->67252223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.359972:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554868864:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5144us (5333us total) trans 4294983727 rc 0/0 00000100:00100000:3.0:1789760792.359990:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16429 00000100:00100000:3.0:1789760792.359999:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735590798302 time 735600434395 state 0x2 ref 0 00000100:00100000:2.0:1789760792.366789:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760792.366985:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554868992 00000100:00100000:1.0:1789760792.366995:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554868992 00000100:00100000:1.0:1789760792.367055:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735590798302 time 735600434395 state 0x0 ref 1 00000100:00100000:1.0:1789760792.367069:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735600694811 time 735600434395 state 0x1 ref 1 00000100:00100000:1.0:1789760792.367145:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735600694811 time 735612763495 state 0x0 ref 1 00000100:00100000:1.0:1789760792.367158:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554868992:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760792.367202:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67252224, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20130 00010000:00010000:1.0:1789760792.367211:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6b2b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.367226:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760792.367230:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.367236:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6b2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67252224->67256319] (req 67252224->67256319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.367249:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6b2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67252224->67256319] (req 67252224->67256319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.372236:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983728 is committed 00002000:00010000:1.0:1789760792.372992:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67252224 -> 67256320 00002000:00010000:1.0:1789760792.372999:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131376 -> 131384 00010000:00010000:1.0:1789760792.373011:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6b2b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67252224->67256319] (req 67252224->67256319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.373024:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6b2b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67252224->67256319] (req 67252224->67256319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.373034:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6b2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67252224->67256319] (req 67252224->67256319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.373052:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760792.373057:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6b2b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67252224->67256319] (req 67252224->67256319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.373065:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6b2b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67252224->67256319] (req 67252224->67256319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.373083:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6b2b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67252224->67256319] (req 67252224->67256319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.373090:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6b2b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67252224->67256319] (req 67252224->67256319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.373178:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554868992:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6016us (6390us total) trans 4294983728 rc 0/0 00000100:00100000:1.0:1789760792.373188:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16430 00000100:00100000:1.0:1789760792.373197:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735600694811 time 735612763495 state 0x2 ref 0 00000100:00100000:1.0:1789760792.389169:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760792.389503:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554869120 00000100:00100000:0.0:1789760792.389513:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554869120 00000100:00100000:0.0:1789760792.389568:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735600694811 time 735612763495 state 0x0 ref 1 00000100:00100000:0.0:1789760792.389583:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735613023911 time 735612763495 state 0x1 ref 1 00000100:00100000:0.0:1789760792.389628:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735613023911 time 735635245680 state 0x0 ref 1 00000100:00100000:0.0:1789760792.389641:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a384a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554869120:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760792.389698:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67256320, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20138 00010000:00010000:0.0:1789760792.389708:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6b32 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.389725:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760792.389730:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.389737:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6b32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67256320->67260415] (req 67256320->67260415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.389751:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6b32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67256320->67260415] (req 67256320->67260415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.400999:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983729 is committed 00002000:00010000:2.0:1789760792.401447:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67256320 -> 67260416 00002000:00010000:2.0:1789760792.401453:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131384 -> 131392 00010000:00010000:2.0:1789760792.401466:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6b32 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67256320->67260415] (req 67256320->67260415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.401479:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6b32 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67256320->67260415] (req 67256320->67260415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.401487:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6b32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67256320->67260415] (req 67256320->67260415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.401495:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760792.401500:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6b32 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67256320->67260415] (req 67256320->67260415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.401510:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6b32 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67256320->67260415] (req 67256320->67260415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.401530:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6b32 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67256320->67260415] (req 67256320->67260415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.401543:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6b32 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67256320->67260415] (req 67256320->67260415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.401653:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a384a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554869120:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12009us (12498us total) trans 4294983729 rc 0/0 00000100:00100000:2.0:1789760792.401664:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16431 00000100:00100000:2.0:1789760792.401673:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735613023911 time 735635245680 state 0x2 ref 0 00000100:00100000:0.0:1789760792.412105:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760792.412315:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554869248 00000100:00100000:2.0:1789760792.412910:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554869248 00000100:00100000:2.0:1789760792.412966:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735613023911 time 735635245680 state 0x0 ref 1 00000100:00100000:2.0:1789760792.413047:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735635506096 time 735635245680 state 0x1 ref 1 00000100:00100000:2.0:1789760792.413083:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735635506096 time 735658701874 state 0x0 ref 1 00000100:00100000:2.0:1789760792.413096:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554869248:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760792.413142:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67260416, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20140 00010000:00010000:2.0:1789760792.413152:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6b39 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.413167:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760792.413171:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.413177:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6b39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67260416->67264511] (req 67260416->67264511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.413196:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6b39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67260416->67264511] (req 67260416->67264511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.426484:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983730 is committed 00002000:00010000:0.0:1789760792.426644:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67260416 -> 67264512 00002000:00010000:0.0:1789760792.426650:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131392 -> 131400 00010000:00010000:0.0:1789760792.426661:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6b39 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67260416->67264511] (req 67260416->67264511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.426679:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6b39 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67260416->67264511] (req 67260416->67264511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.426688:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6b39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67260416->67264511] (req 67260416->67264511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.426696:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760792.426700:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6b39 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67260416->67264511] (req 67260416->67264511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.426857:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6b39 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67260416->67264511] (req 67260416->67264511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.426891:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6b39 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67260416->67264511] (req 67260416->67264511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.426898:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6b39 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67260416->67264511] (req 67260416->67264511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.427003:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554869248:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13904us (14902us total) trans 4294983730 rc 0/0 00000100:00100000:0.0:1789760792.427013:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16432 00000100:00100000:0.0:1789760792.427021:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735635506096 time 735658701874 state 0x2 ref 0 00000100:00100000:0.0:1789760792.433064:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760792.433413:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554869376 00000100:00100000:3.0:1789760792.433435:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554869376 00000100:00100000:3.0:1789760792.433492:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735635506096 time 735658701874 state 0x0 ref 1 00000100:00100000:3.0:1789760792.433508:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735658962290 time 735658701874 state 0x1 ref 1 00000100:00100000:3.0:1789760792.433541:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735658962290 time 735679159632 state 0x0 ref 1 00000100:00100000:3.0:1789760792.433554:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554869376:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760792.433600:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67264512, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20148 00010000:00010000:3.0:1789760792.433618:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b40 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.433634:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760792.433639:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.433646:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67264512->67268607] (req 67264512->67268607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.433661:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67264512->67268607] (req 67264512->67268607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.441808:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983731 is committed 00002000:00010000:3.0:1789760792.441925:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67264512 -> 67268608 00002000:00010000:3.0:1789760792.441931:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131400 -> 131408 00010000:00010000:3.0:1789760792.441944:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b40 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67264512->67268607] (req 67264512->67268607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.441962:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b40 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67264512->67268607] (req 67264512->67268607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.441971:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67264512->67268607] (req 67264512->67268607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.441990:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760792.441996:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b40 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67264512->67268607] (req 67264512->67268607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.442004:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b40 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67264512->67268607] (req 67264512->67268607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.442023:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b40 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67264512->67268607] (req 67264512->67268607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.442031:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b40 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67264512->67268607] (req 67264512->67268607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.442160:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554869376:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8604us (9102us total) trans 4294983731 rc 0/0 00000100:00100000:3.0:1789760792.442181:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16433 00000100:00100000:3.0:1789760792.442191:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735658962290 time 735679159632 state 0x2 ref 0 00000100:00100000:2.0:1789760792.448724:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760792.449108:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554869504 00000100:00100000:1.0:1789760792.449199:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554869504 00000100:00100000:1.0:1789760792.449258:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735658962290 time 735679159632 state 0x0 ref 1 00000100:00100000:1.0:1789760792.449274:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735679420048 time 735679159632 state 0x1 ref 1 00000100:00100000:1.0:1789760792.449310:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735679420048 time 735694928939 state 0x0 ref 1 00000100:00100000:1.0:1789760792.449322:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554869504:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760792.449369:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67268608, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20150 00010000:00010000:1.0:1789760792.449408:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6b47 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.449424:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760792.449429:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.449435:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6b47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67268608->67272703] (req 67268608->67272703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.449448:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6b47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67268608->67272703] (req 67268608->67272703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.457279:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983732 is committed 00002000:00010000:1.0:1789760792.457502:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67268608 -> 67272704 00002000:00010000:1.0:1789760792.457507:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131408 -> 131416 00010000:00010000:1.0:1789760792.457521:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6b47 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67268608->67272703] (req 67268608->67272703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.457534:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6b47 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67268608->67272703] (req 67268608->67272703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.457543:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6b47 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67268608->67272703] (req 67268608->67272703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.457551:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760792.457556:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6b47 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67268608->67272703] (req 67268608->67272703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.457565:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6b47 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67268608->67272703] (req 67268608->67272703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.457583:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6b47 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67268608->67272703] (req 67268608->67272703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.457601:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6b47 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67268608->67272703] (req 67268608->67272703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.457745:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554869504:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8421us (9025us total) trans 4294983732 rc 0/0 00000100:00100000:1.0:1789760792.457756:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16434 00000100:00100000:1.0:1789760792.457766:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735679420048 time 735694928939 state 0x2 ref 0 00000100:00100000:3.0:1789760792.468334:0:3647:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03e90000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3647:1876699573921536:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760792.468437:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760792.468712:0:11526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573921536 00000100:00100000:1.0:1789760792.468720:0:11526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573921536 00000100:00100000:1.0:1789760792.468777:0:11526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 370 00000100:00100000:1.0:1789760792.468787:0:11526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3647:x1876699573921536:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789760792.468815:0:11526:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:1.0:1789760792.468858:0:11526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3647:x1876699573921536:12345-0@lo:13:osp-pre-1-0.0 Request processed in 73us (421us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760792.468865:0:11526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 370 00000100:00080000:1.0:1789760792.468886:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789760792.468963:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760792.468987:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554869632 00000100:00100000:1.0:1789760792.468991:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554869632 00000100:00100000:1.0:1789760792.469023:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735679420048 time 735694928939 state 0x0 ref 1 00000100:00100000:1.0:1789760792.469036:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735695189355 time 735694928939 state 0x1 ref 1 00000100:00100000:1.0:1789760792.469055:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735695189355 time 735714673820 state 0x0 ref 1 00000100:00100000:1.0:1789760792.469065:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a386a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554869632:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760792.469102:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67272704, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20158 00010000:00010000:1.0:1789760792.469111:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6b4e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.469132:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760792.469136:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.469142:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6b4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67272704->67276799] (req 67272704->67276799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.469155:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6b4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67272704->67276799] (req 67272704->67276799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760792.469940:0:3647:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03e90000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3647:1876699573921536:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789760792.469963:0:3647:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789760792.469971:0:3647:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cc3a385880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3647:1876699573921408:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760792.470009:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760792.471539:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03e93800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573921920:0@lo:400:kworker.0 00000100:00100000:2.0:1789760792.471728:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03e92a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:5711db52-8702-4d6b-b42c-8543e9bf1cb2:3648:1876699573921664:0@lo:400:kworker.0 00000100:00100000:2.0:1789760792.471765:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760792.471796:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03de4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3648:1876699573922176:0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.471846:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760792.471883:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03de5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3646:1876699573922432:0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.471908:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760792.471923:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03de5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3646:1876699573922944:0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.471942:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760792.472029:0:3646:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1789760792.472037:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03de6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3646:1876699573922048:0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.472062:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760792.472077:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03de7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573922560:0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.472096:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760792.472125:0:8438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573922560 00000100:00100000:0.0:1789760792.472132:0:8438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573922560 00000100:00100000:0.0:1789760792.472180:0:8438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 303 00000100:00100000:0.0:1789760792.472190:0:8438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06403480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3646:x1876699573922560:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.472237:0:8438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06403480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3646:x1876699573922560:12345-0@lo:400:kworker.0 Request processed in 48us (138us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760792.472244:0:8438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 303 00000100:00100000:0.0:1789760792.472272:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760792.472291:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03de4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3648:1876699573922688:0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.472319:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760792.472338:0:3648:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:0.0:1789760792.472344:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03e90000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3648:1876699573921792:0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.472363:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760792.472402:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03de6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3648:1876699573922304:0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.472427:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760792.472442:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03de7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3648:1876699573922816:0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.472463:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760792.472481:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573921920 00000100:00100000:0.0:1789760792.472484:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573921920 00000100:00100000:0.0:1789760792.472511:0:6525:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573922176 00000100:00100000:0.0:1789760792.472514:0:6525:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573922176 00000100:00100000:0.0:1789760792.472526:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 394 00000100:00100000:0.0:1789760792.472532:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3646:x1876699573921920:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.472564:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3646:x1876699573921920:12345-0@lo:400:kworker.0 Request processed in 32us (973us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760792.472569:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 394 00000100:00100000:0.0:1789760792.472581:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 395 00000100:00100000:0.0:1789760792.472586:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06402a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3648:x1876699573922176:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760792.472739:0:6513:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573921664 00000100:00100000:3.0:1789760792.472743:0:6513:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573921664 00000100:00100000:3.0:1789760792.472773:0:6513:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 475 00000100:00100000:3.0:1789760792.472780:0:6513:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc320b4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5711db52-8702-4d6b-b42c-8543e9bf1cb2+24:3648:x1876699573921664:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760792.472823:0:6513:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc320b4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5711db52-8702-4d6b-b42c-8543e9bf1cb2+24:3648:x1876699573921664:12345-0@lo:400:kworker.0 Request processed in 44us (1058us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760792.472829:0:6513:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 475 00000100:00100000:3.0:1789760792.472856:0:11089:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573922432 00000100:00100000:3.0:1789760792.472860:0:11089:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573922432 00000100:00100000:3.0:1789760792.472877:0:11089:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573922944 00000100:00100000:3.0:1789760792.472879:0:11089:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573922944 00000100:00100000:3.0:1789760792.472889:0:11089:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573922048 00000100:00100000:3.0:1789760792.472891:0:11089:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573922048 00000100:00100000:3.0:1789760792.472899:0:11089:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573921792 00000100:00100000:3.0:1789760792.472901:0:11089:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573921792 00000100:00100000:3.0:1789760792.472908:0:11089:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573922304 00000100:00100000:3.0:1789760792.472910:0:11089:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573922304 00000100:00100000:3.0:1789760792.472918:0:11089:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573922816 00000100:00100000:3.0:1789760792.472920:0:11089:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573922816 00000100:00100000:3.0:1789760792.472934:0:11089:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 899 00000100:00100000:3.0:1789760792.472940:0:11089:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3646:x1876699573922432:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760792.474270:0:8440:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573921408 00000100:00100000:1.0:1789760792.474286:0:8440:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573921408 00000100:00100000:1.0:1789760792.474335:0:8440:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 371 00000100:00100000:1.0:1789760792.474343:0:8440:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3647:x1876699573921408:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1789760792.474362:0:8440:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:1.0:1789760792.474449:0:8440:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3647:x1876699573921408:12345-0@lo:13:osp-pre-1-1.0 Request processed in 106us (4438us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760792.474456:0:8440:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 371 00000100:00080000:1.0:1789760792.474478:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789760792.474482:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789760792.474484:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:1.0:1789760792.474486:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760792.474522:0:7470:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 900 00000100:00100000:1.0:1789760792.474530:0:7470:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3646:x1876699573922944:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760792.474564:0:7470:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3646:x1876699573922944:12345-0@lo:400:kworker.0 Request processed in 37us (2622us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760792.474571:0:7470:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 900 00000100:00100000:1.0:1789760792.474584:0:7470:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 901 00000100:00100000:1.0:1789760792.474591:0:7470:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06401f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3646:x1876699573922048:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760792.474720:0:11089:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3646:x1876699573922432:12345-0@lo:400:kworker.0 Request processed in 1779us (2810us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760792.474731:0:11089:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 899 00000100:00100000:3.0:1789760792.474746:0:11089:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 902 00000100:00100000:3.0:1789760792.474753:0:11089:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06403b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3648:x1876699573921792:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760792.474785:0:11089:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06403b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3648:x1876699573921792:12345-0@lo:400:kworker.0 Request processed in 33us (2420us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760792.474790:0:11089:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 902 00000100:00100000:3.0:1789760792.474802:0:11089:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 903 00000100:00100000:3.0:1789760792.474809:0:11089:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06402300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3648:x1876699573922304:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760792.474836:0:11089:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06402300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3648:x1876699573922304:12345-0@lo:400:kworker.0 Request processed in 28us (2408us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760792.474843:0:11089:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 903 00000100:00100000:3.0:1789760792.474858:0:11089:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 904 00000100:00100000:3.0:1789760792.474864:0:11089:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06403800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3648:x1876699573922816:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760792.474894:0:11089:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06403800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3648:x1876699573922816:12345-0@lo:400:kworker.0 Request processed in 31us (2430us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760792.474899:0:11089:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 904 00000001:00080000:2.0:1789760792.475821:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983733 is committed 00002000:00010000:2.0:1789760792.475877:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67272704 -> 67276800 00002000:00010000:2.0:1789760792.475891:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131416 -> 131424 00010000:00010000:2.0:1789760792.475905:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6b4e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67272704->67276799] (req 67272704->67276799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.475918:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6b4e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67272704->67276799] (req 67272704->67276799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.475928:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6b4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67272704->67276799] (req 67272704->67276799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.475935:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760792.475940:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6b4e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67272704->67276799] (req 67272704->67276799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.475954:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6b4e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67272704->67276799] (req 67272704->67276799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.475974:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6b4e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67272704->67276799] (req 67272704->67276799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.476086:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6b4e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67272704->67276799] (req 67272704->67276799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.476174:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06402a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3648:x1876699573922176:12345-0@lo:400:kworker.0 Request processed in 3585us (3898us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760792.476183:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 395 00000100:00100000:0.0:1789760792.476244:0:15103:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573922688 00000100:00100000:0.0:1789760792.476251:0:15103:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573922688 00000100:00100000:2.0:1789760792.476293:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a386a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554869632:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7225us (7327us total) trans 4294983733 rc 0/0 00000100:00100000:0.0:1789760792.476298:0:15103:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 304 00000100:00100000:0.0:1789760792.476312:0:15103:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06400e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3648:x1876699573922688:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760792.476312:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16435 00000100:00100000:2.0:1789760792.476326:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735695189355 time 735714673820 state 0x2 ref 0 00000100:00100000:0.0:1789760792.476352:0:15103:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06400e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3648:x1876699573922688:12345-0@lo:400:kworker.0 Request processed in 44us (4032us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760792.476359:0:15103:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 304 00000100:00100000:0.0:1789760792.476432:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03de7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573922560:0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.476445:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03e93800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573921920:0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.476457:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03de5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3646:1876699573922432:0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.476468:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03de5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3646:1876699573922944:0@lo:400:kworker.0 00000100:00100000:0.0:1789760792.476489:0:3646:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760792.476501:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03ee7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573923072:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789760792.476536:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760792.476569:0:7831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573923072 00000100:00100000:0.0:1789760792.476573:0:7831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573923072 00000100:00100000:1.0:1789760792.476804:0:7470:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06401f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3646:x1876699573922048:12345-0@lo:400:kworker.0 Request processed in 2213us (4739us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760792.476811:0:7470:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 901 00000100:00080000:1.0:1789760792.476829:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760792.476856:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03de6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3646:1876699573922048:0@lo:400:kworker.0 00000100:00100000:1.0:1789760792.476882:0:6525:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 396 00000100:00100000:1.0:1789760792.476889:0:6525:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06401180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3646:x1876699573923072:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1789760792.476913:0:6525:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446227968 avail 1299988480 00000100:00100000:1.0:1789760792.476945:0:6525:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06401180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3646:x1876699573923072:12345-0@lo:41:osp-pre-0-1.0 Request processed in 58us (409us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760792.477129:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03e90000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3648:1876699573921792:0@lo:400:kworker.0 00000100:00100000:3.0:1789760792.478221:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03de6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3648:1876699573922304:0@lo:400:kworker.0 00000100:00100000:3.0:1789760792.478246:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03de7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3648:1876699573922816:0@lo:400:kworker.0 00000100:00100000:3.0:1789760792.478256:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03e92a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:5711db52-8702-4d6b-b42c-8543e9bf1cb2:3648:1876699573921664:0@lo:400:kworker.0 00000100:00100000:3.0:1789760792.478265:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03de4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3648:1876699573922176:0@lo:400:kworker.0 00000100:00100000:3.0:1789760792.478276:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03de4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3648:1876699573922688:0@lo:400:kworker.0 00000100:00100000:3.0:1789760792.478335:0:3647:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cc3a385880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3647:1876699573921408:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1789760792.478451:0:6525:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 396 00000100:00100000:1.0:1789760792.478493:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03ee7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3646:1876699573923072:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1789760792.484866:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760792.485262:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554869760 00000100:00100000:0.0:1789760792.485273:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554869760 00000100:00100000:0.0:1789760792.485330:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735695189355 time 735714673820 state 0x0 ref 1 00000100:00100000:0.0:1789760792.485346:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735714934236 time 735714673820 state 0x1 ref 1 00000100:00100000:0.0:1789760792.485418:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735714934236 time 735730995559 state 0x0 ref 1 00000100:00100000:0.0:1789760792.485433:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a386300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554869760:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760792.485481:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67276800, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20160 00010000:00010000:0.0:1789760792.485490:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6b55 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.485505:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760792.485509:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.485516:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6b55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67276800->67280895] (req 67276800->67280895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.485530:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6b55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67276800->67280895] (req 67276800->67280895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.493185:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983734 is committed 00002000:00010000:0.0:1789760792.493926:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67276800 -> 67280896 00002000:00010000:0.0:1789760792.493933:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131424 -> 131432 00010000:00010000:0.0:1789760792.493945:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6b55 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67276800->67280895] (req 67276800->67280895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.494284:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6b55 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67276800->67280895] (req 67276800->67280895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.494293:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6b55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67276800->67280895] (req 67276800->67280895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.494299:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760792.494304:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6b55 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67276800->67280895] (req 67276800->67280895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.494312:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6b55 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67276800->67280895] (req 67276800->67280895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.494330:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6b55 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67276800->67280895] (req 67276800->67280895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.494338:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6b55 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67276800->67280895] (req 67276800->67280895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.494816:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a386300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554869760:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9379us (9954us total) trans 4294983734 rc 0/0 00000100:00100000:2.0:1789760792.494827:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16436 00000100:00100000:2.0:1789760792.494837:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735714934236 time 735730995559 state 0x2 ref 0 00000100:00100000:2.0:1789760792.503677:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760792.503794:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554869888 00000100:00100000:1.0:1789760792.503803:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554869888 00000100:00100000:1.0:1789760792.503857:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735714934236 time 735730995559 state 0x0 ref 1 00000100:00100000:1.0:1789760792.503873:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735731255975 time 735730995559 state 0x1 ref 1 00000100:00100000:1.0:1789760792.503943:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735731255975 time 735749561104 state 0x0 ref 1 00000100:00100000:1.0:1789760792.503956:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554869888:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760792.504005:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67280896, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20168 00010000:00010000:1.0:1789760792.504014:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6b5c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.504041:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760792.504046:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.504052:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6b5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67280896->67284991] (req 67280896->67284991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.504065:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6b5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67280896->67284991] (req 67280896->67284991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.509668:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983735 is committed 00002000:00010000:1.0:1789760792.509900:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67280896 -> 67284992 00002000:00010000:1.0:1789760792.509905:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131432 -> 131440 00010000:00010000:1.0:1789760792.509917:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6b5c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67280896->67284991] (req 67280896->67284991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.509927:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6b5c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67280896->67284991] (req 67280896->67284991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.509936:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6b5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67280896->67284991] (req 67280896->67284991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.509942:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760792.509947:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6b5c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67280896->67284991] (req 67280896->67284991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.509956:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6b5c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67280896->67284991] (req 67280896->67284991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.509984:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6b5c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67280896->67284991] (req 67280896->67284991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.509992:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6b5c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67280896->67284991] (req 67280896->67284991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.510253:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554869888:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6294us (6580us total) trans 4294983735 rc 0/0 00000100:00100000:1.0:1789760792.510263:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16437 00000100:00100000:1.0:1789760792.510273:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735731255975 time 735749561104 state 0x2 ref 0 00000100:00100000:1.0:1789760792.520310:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760792.520480:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554870016 00000100:00100000:2.0:1789760792.520489:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554870016 00000100:00100000:2.0:1789760792.520615:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735731255975 time 735749561104 state 0x0 ref 1 00000100:00100000:2.0:1789760792.520635:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735749821520 time 735749561104 state 0x1 ref 1 00000100:00100000:2.0:1789760792.520667:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735749821520 time 735766286316 state 0x0 ref 1 00000100:00100000:2.0:1789760792.520680:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554870016:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760792.520722:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67284992, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20170 00010000:00010000:2.0:1789760792.520731:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6b63 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.522157:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760792.522168:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.522179:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6b63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67284992->67289087] (req 67284992->67289087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.522197:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6b63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67284992->67289087] (req 67284992->67289087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.532206:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983736 is committed 00002000:00010000:1.0:1789760792.532451:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67284992 -> 67289088 00002000:00010000:1.0:1789760792.532456:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131440 -> 131448 00010000:00010000:1.0:1789760792.532470:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6b63 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67284992->67289087] (req 67284992->67289087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.532482:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6b63 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67284992->67289087] (req 67284992->67289087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.532491:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6b63 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67284992->67289087] (req 67284992->67289087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.532498:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760792.532503:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6b63 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67284992->67289087] (req 67284992->67289087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.532511:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6b63 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67284992->67289087] (req 67284992->67289087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.532530:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6b63 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67284992->67289087] (req 67284992->67289087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.532538:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6b63 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67284992->67289087] (req 67284992->67289087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.532774:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554870016:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12091us (12469us total) trans 4294983736 rc 0/0 00000100:00100000:1.0:1789760792.532785:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16438 00000100:00100000:1.0:1789760792.532796:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735749821520 time 735766286316 state 0x2 ref 0 00000100:00100000:2.0:1789760792.545279:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760792.545501:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554870144 00000100:00100000:0.0:1789760792.545512:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554870144 00000100:00100000:0.0:1789760792.545567:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735749821520 time 735766286316 state 0x0 ref 1 00000100:00100000:0.0:1789760792.545582:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735766546732 time 735766286316 state 0x1 ref 1 00000100:00100000:0.0:1789760792.545669:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735766546732 time 735791286646 state 0x0 ref 1 00000100:00100000:0.0:1789760792.545684:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554870144:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760792.545731:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67289088, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20178 00010000:00010000:0.0:1789760792.545741:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6b6a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.545756:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760792.545761:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.545767:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6b6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67289088->67293183] (req 67289088->67293183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.545782:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6b6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67289088->67293183] (req 67289088->67293183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.551320:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983737 is committed 00002000:00010000:0.0:1789760792.551676:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67289088 -> 67293184 00002000:00010000:0.0:1789760792.551698:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131448 -> 131456 00010000:00010000:0.0:1789760792.551712:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6b6a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67289088->67293183] (req 67289088->67293183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.551725:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6b6a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67289088->67293183] (req 67289088->67293183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.551741:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6b6a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67289088->67293183] (req 67289088->67293183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.551749:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760792.551754:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6b6a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67289088->67293183] (req 67289088->67293183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.551763:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6b6a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67289088->67293183] (req 67289088->67293183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.551781:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6b6a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67289088->67293183] (req 67289088->67293183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.551789:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6b6a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67289088->67293183] (req 67289088->67293183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.551889:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554870144:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6203us (6615us total) trans 4294983737 rc 0/0 00000100:00100000:0.0:1789760792.551899:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16439 00000100:00100000:0.0:1789760792.551908:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735766546732 time 735791286646 state 0x2 ref 0 00000100:00100000:2.0:1789760792.556759:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760792.557015:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554870272 00000100:00100000:0.0:1789760792.557026:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554870272 00000100:00100000:0.0:1789760792.557090:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735766546732 time 735791286646 state 0x0 ref 1 00000100:00100000:0.0:1789760792.557118:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735791547062 time 735791286646 state 0x1 ref 1 00000100:00100000:0.0:1789760792.557196:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735791547062 time 735802814416 state 0x0 ref 1 00000100:00100000:0.0:1789760792.557208:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554870272:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760792.557260:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67293184, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20180 00010000:00010000:0.0:1789760792.557269:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6b71 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.557286:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760792.557291:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.557297:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6b71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67293184->67297279] (req 67293184->67297279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.557312:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6b71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67293184->67297279] (req 67293184->67297279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.565896:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983738 is committed 00002000:00010000:0.0:1789760792.567048:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67293184 -> 67297280 00002000:00010000:0.0:1789760792.567055:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131456 -> 131464 00010000:00010000:0.0:1789760792.567070:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6b71 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67293184->67297279] (req 67293184->67297279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.567084:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6b71 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67293184->67297279] (req 67293184->67297279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.567093:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6b71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67293184->67297279] (req 67293184->67297279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.567106:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760792.567111:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6b71 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67293184->67297279] (req 67293184->67297279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.567119:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6b71 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67293184->67297279] (req 67293184->67297279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.567137:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6b71 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67293184->67297279] (req 67293184->67297279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.567144:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6b71 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67293184->67297279] (req 67293184->67297279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.567241:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554870272:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10030us (10488us total) trans 4294983738 rc 0/0 00000100:00100000:0.0:1789760792.567252:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16440 00000100:00100000:0.0:1789760792.567262:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735791547062 time 735802814416 state 0x2 ref 0 00000100:00100000:1.0:1789760792.575149:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760792.579484:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554870400 00000100:00100000:3.0:1789760792.579501:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554870400 00000100:00100000:3.0:1789760792.579578:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735791547062 time 735802814416 state 0x0 ref 1 00000100:00100000:3.0:1789760792.579596:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735803074832 time 735802814416 state 0x1 ref 1 00000100:00100000:3.0:1789760792.579693:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735803074832 time 735825310468 state 0x0 ref 1 00000100:00100000:3.0:1789760792.579707:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a386680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554870400:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760792.579753:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67297280, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20188 00010000:00010000:3.0:1789760792.579764:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b78 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.579779:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:3.0:1789760792.579783:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760792.579790:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67297280->67301375] (req 67297280->67301375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760792.579804:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67297280->67301375] (req 67297280->67301375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.587227:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983739 is committed 00002000:00010000:1.0:1789760792.587482:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67297280 -> 67301376 00002000:00010000:1.0:1789760792.587488:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131464 -> 131472 00010000:00010000:1.0:1789760792.587501:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b78 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67297280->67301375] (req 67297280->67301375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.587512:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b78 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67297280->67301375] (req 67297280->67301375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.587522:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67297280->67301375] (req 67297280->67301375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.587530:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760792.587536:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b78 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67297280->67301375] (req 67297280->67301375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.587545:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b78 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67297280->67301375] (req 67297280->67301375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.587564:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b78 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67297280->67301375] (req 67297280->67301375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.587571:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6b78 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67297280->67301375] (req 67297280->67301375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.587717:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a386680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554870400:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8009us (12573us total) trans 4294983739 rc 0/0 00000100:00100000:1.0:1789760792.587728:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16441 00000100:00100000:1.0:1789760792.587737:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735803074832 time 735825310468 state 0x2 ref 0 00000100:00100000:2.0:1789760792.591464:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760792.591554:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554870528 00000100:00100000:1.0:1789760792.591563:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554870528 00000100:00100000:1.0:1789760792.591622:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735803074832 time 735825310468 state 0x0 ref 1 00000100:00100000:1.0:1789760792.591636:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735825570884 time 735825310468 state 0x1 ref 1 00000100:00100000:1.0:1789760792.591658:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735825570884 time 735837276533 state 0x0 ref 1 00000100:00100000:1.0:1789760792.591673:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554870528:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760792.591718:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67301376, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20190 00010000:00010000:1.0:1789760792.591727:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6b7f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.591744:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760792.591747:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.591753:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6b7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67301376->67305471] (req 67301376->67305471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.591766:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6b7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67301376->67305471] (req 67301376->67305471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.596531:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983740 is committed 00002000:00010000:1.0:1789760792.596738:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67301376 -> 67305472 00002000:00010000:1.0:1789760792.596744:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131472 -> 131480 00010000:00010000:1.0:1789760792.596757:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6b7f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67301376->67305471] (req 67301376->67305471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.596771:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6b7f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67301376->67305471] (req 67301376->67305471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.596780:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6b7f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67301376->67305471] (req 67301376->67305471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.596788:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655de00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760792.596794:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6b7f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67301376->67305471] (req 67301376->67305471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.596803:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6b7f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67301376->67305471] (req 67301376->67305471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.596822:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6b7f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67301376->67305471] (req 67301376->67305471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.596829:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6b7f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67301376->67305471] (req 67301376->67305471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.597164:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554870528:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5486us (5713us total) trans 4294983740 rc 0/0 00000100:00100000:1.0:1789760792.597176:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16442 00000100:00100000:1.0:1789760792.597185:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735825570884 time 735837276533 state 0x2 ref 0 00000100:00100000:2.0:1789760792.604917:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760792.605027:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554870656 00000100:00100000:0.0:1789760792.605037:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554870656 00000100:00100000:0.0:1789760792.605090:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735825570884 time 735837276533 state 0x0 ref 1 00000100:00100000:0.0:1789760792.605105:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735837536949 time 735837276533 state 0x1 ref 1 00000100:00100000:0.0:1789760792.605144:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735837536949 time 735850762929 state 0x0 ref 1 00000100:00100000:0.0:1789760792.605160:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554870656:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760792.605208:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67305472, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x20198 00010000:00010000:0.0:1789760792.605216:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6b86 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.605231:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:0.0:1789760792.605236:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.605242:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6b86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67305472->67309567] (req 67305472->67309567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.605255:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6b86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67305472->67309567] (req 67305472->67309567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.611180:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983741 is committed 00002000:00010000:2.0:1789760792.611262:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67305472 -> 67309568 00002000:00010000:2.0:1789760792.611266:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131480 -> 131488 00010000:00010000:2.0:1789760792.611277:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6b86 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67305472->67309567] (req 67305472->67309567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.611285:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6b86 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67305472->67309567] (req 67305472->67309567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.611290:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6b86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67305472->67309567] (req 67305472->67309567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.611294:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760792.611298:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6b86 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67305472->67309567] (req 67305472->67309567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.611304:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6b86 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67305472->67309567] (req 67305472->67309567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.611318:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6b86 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67305472->67309567] (req 67305472->67309567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.611322:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6b86 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67305472->67309567] (req 67305472->67309567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.611426:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554870656:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6264us (6516us total) trans 4294983741 rc 0/0 00000100:00100000:2.0:1789760792.611436:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16443 00000100:00100000:2.0:1789760792.611445:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735837536949 time 735850762929 state 0x2 ref 0 00000100:00100000:1.0:1789760792.615076:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760792.615590:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554870784 00000100:00100000:0.0:1789760792.615601:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554870784 00000100:00100000:1.0:1789760792.615745:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735837536949 time 735850762929 state 0x0 ref 1 00000100:00100000:1.0:1789760792.615760:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735851023345 time 735850762929 state 0x1 ref 1 00000100:00100000:1.0:1789760792.615826:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735851023345 time 735861443515 state 0x0 ref 1 00000100:00100000:1.0:1789760792.615839:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554870784:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760792.615898:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67309568, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x201a0 00010000:00010000:1.0:1789760792.615907:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6b8d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.615924:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760792.615928:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.615934:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6b8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67309568->67313663] (req 67309568->67313663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.615948:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6b8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67309568->67313663] (req 67309568->67313663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.621078:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983742 is committed 00002000:00010000:1.0:1789760792.622663:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67309568 -> 67313664 00002000:00010000:1.0:1789760792.622671:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131488 -> 131496 00010000:00010000:1.0:1789760792.622745:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6b8d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67309568->67313663] (req 67309568->67313663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.622760:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6b8d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67309568->67313663] (req 67309568->67313663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.622770:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6b8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67309568->67313663] (req 67309568->67313663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.622778:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760792.622784:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6b8d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67309568->67313663] (req 67309568->67313663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.622795:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6b8d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67309568->67313663] (req 67309568->67313663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.622815:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6b8d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67309568->67313663] (req 67309568->67313663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.622823:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6b8d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67309568->67313663] (req 67309568->67313663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.623753:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554870784:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7908us (8677us total) trans 4294983742 rc 0/0 00000100:00100000:1.0:1789760792.623765:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16444 00000100:00100000:1.0:1789760792.623775:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735851023345 time 735861443515 state 0x2 ref 0 00000100:00100000:1.0:1789760792.630655:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760792.631047:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554870912 00000100:00100000:2.0:1789760792.631057:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554870912 00000100:00100000:2.0:1789760792.631108:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735851023345 time 735861443515 state 0x0 ref 1 00000100:00100000:2.0:1789760792.631123:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735861703931 time 735861443515 state 0x1 ref 1 00000100:00100000:2.0:1789760792.631153:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735861703931 time 735876771978 state 0x0 ref 1 00000100:00100000:2.0:1789760792.631165:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554870912:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760792.631211:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67313664, mtime: 0x6aad9517, atime: 0x0, ctime: 0x6aad9517, blocks: 0x201a8 00010000:00010000:2.0:1789760792.631221:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6b94 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.631235:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760792.631240:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.631246:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6b94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67313664->67317759] (req 67313664->67317759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.631259:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6b94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67313664->67317759] (req 67313664->67317759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760792.639458:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983743 is committed 00002000:00010000:2.0:1789760792.639733:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67313664 -> 67317760 00002000:00010000:2.0:1789760792.639737:0:8442:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760791 -> 1789760792 00002000:00010000:2.0:1789760792.639739:0:8442:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760791 -> 1789760792 00002000:00010000:2.0:1789760792.639741:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131496 -> 131504 00010000:00010000:2.0:1789760792.639756:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6b94 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67313664->67317759] (req 67313664->67317759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.639778:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6b94 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67313664->67317759] (req 67313664->67317759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.639787:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6b94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67313664->67317759] (req 67313664->67317759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.639793:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760792.639798:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6b94 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67313664->67317759] (req 67313664->67317759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.639806:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6b94 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67313664->67317759] (req 67313664->67317759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.639824:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6b94 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67313664->67317759] (req 67313664->67317759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.639841:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6b94 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67313664->67317759] (req 67313664->67317759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.639932:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554870912:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8764us (9281us total) trans 4294983743 rc 0/0 00000100:00100000:2.0:1789760792.639943:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16445 00000100:00100000:2.0:1789760792.639954:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735861703931 time 735876771978 state 0x2 ref 0 00000100:00100000:2.0:1789760792.647559:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760792.647742:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554871040 00000100:00100000:1.0:1789760792.647752:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554871040 00000100:00100000:1.0:1789760792.647817:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735861703931 time 735876771978 state 0x0 ref 1 00000100:00100000:1.0:1789760792.647831:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735877032394 time 735876771978 state 0x1 ref 1 00000100:00100000:1.0:1789760792.647862:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735877032394 time 735893480057 state 0x0 ref 1 00000100:00100000:1.0:1789760792.647874:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554871040:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760792.647920:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67317760, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x201b0 00010000:00010000:1.0:1789760792.647929:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6b9b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.647950:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760792.647954:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.647960:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6b9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67317760->67321855] (req 67317760->67321855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.647973:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6b9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67317760->67321855] (req 67317760->67321855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.662186:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983744 is committed 00002000:00010000:1.0:1789760792.662306:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67317760 -> 67321856 00002000:00010000:1.0:1789760792.662312:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131504 -> 131512 00010000:00010000:1.0:1789760792.662335:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6b9b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67317760->67321855] (req 67317760->67321855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.662347:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6b9b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67317760->67321855] (req 67317760->67321855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.662356:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6b9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67317760->67321855] (req 67317760->67321855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.662362:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760792.662367:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6b9b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67317760->67321855] (req 67317760->67321855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.662413:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6b9b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67317760->67321855] (req 67317760->67321855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.662439:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6b9b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67317760->67321855] (req 67317760->67321855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.662446:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6b9b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67317760->67321855] (req 67317760->67321855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.662562:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554871040:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14684us (15007us total) trans 4294983744 rc 0/0 00000100:00100000:1.0:1789760792.662572:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16446 00000100:00100000:1.0:1789760792.662581:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735877032394 time 735893480057 state 0x2 ref 0 00000100:00100000:1.0:1789760792.681179:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760792.681407:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554871168 00000100:00100000:2.0:1789760792.681417:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554871168 00000100:00100000:2.0:1789760792.681465:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735877032394 time 735893480057 state 0x0 ref 1 00000100:00100000:2.0:1789760792.681480:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735893740473 time 735893480057 state 0x1 ref 1 00000100:00100000:2.0:1789760792.681511:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735893740473 time 735927129543 state 0x0 ref 1 00000100:00100000:2.0:1789760792.681522:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a384e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554871168:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760792.681578:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67321856, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x201b8 00010000:00010000:2.0:1789760792.681587:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6ba2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.681612:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789760792.681616:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.681622:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6ba2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67321856->67325951] (req 67321856->67325951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.681635:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6ba2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67321856->67325951] (req 67321856->67325951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.698464:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983745 is committed 00002000:00010000:2.0:1789760792.698818:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67321856 -> 67325952 00002000:00010000:2.0:1789760792.698823:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131512 -> 131520 00010000:00010000:2.0:1789760792.698837:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6ba2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67321856->67325951] (req 67321856->67325951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.698856:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6ba2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67321856->67325951] (req 67321856->67325951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.698865:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6ba2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67321856->67325951] (req 67321856->67325951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.698872:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760792.698877:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6ba2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67321856->67325951] (req 67321856->67325951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.698886:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6ba2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67321856->67325951] (req 67321856->67325951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.698905:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6ba2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67321856->67325951] (req 67321856->67325951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.698912:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6ba2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67321856->67325951] (req 67321856->67325951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.699027:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a384e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554871168:12345-192.168.203.22@tcp:4:dd.0 Request processed in 17499us (17852us total) trans 4294983745 rc 0/0 00000100:00100000:2.0:1789760792.699038:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16447 00000100:00100000:2.0:1789760792.699048:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735893740473 time 735927129543 state 0x2 ref 0 00000100:00100000:2.0:1789760792.708806:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760792.708953:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554871296 00000100:00100000:1.0:1789760792.708963:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554871296 00000100:00100000:1.0:1789760792.709022:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735893740473 time 735927129543 state 0x0 ref 1 00000100:00100000:1.0:1789760792.709038:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735927389959 time 735927129543 state 0x1 ref 1 00000100:00100000:1.0:1789760792.709065:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735927389959 time 735954683643 state 0x0 ref 1 00000100:00100000:1.0:1789760792.709076:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554871296:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760792.709121:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67325952, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x201c0 00010000:00010000:1.0:1789760792.709131:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6ba9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.709151:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760792.709155:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.709162:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6ba9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67325952->67330047] (req 67325952->67330047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.709176:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6ba9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67325952->67330047] (req 67325952->67330047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.717640:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983746 is committed 00002000:00010000:1.0:1789760792.717783:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67325952 -> 67330048 00002000:00010000:1.0:1789760792.717788:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131520 -> 131528 00010000:00010000:1.0:1789760792.717801:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6ba9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67325952->67330047] (req 67325952->67330047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.717811:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6ba9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67325952->67330047] (req 67325952->67330047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.717820:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6ba9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67325952->67330047] (req 67325952->67330047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.717827:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760792.717832:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6ba9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67325952->67330047] (req 67325952->67330047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.717840:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6ba9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67325952->67330047] (req 67325952->67330047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.717864:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6ba9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67325952->67330047] (req 67325952->67330047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.717872:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6ba9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67325952->67330047] (req 67325952->67330047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.718270:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554871296:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9189us (9468us total) trans 4294983746 rc 0/0 00000100:00100000:1.0:1789760792.718281:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16448 00000100:00100000:1.0:1789760792.718290:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735927389959 time 735954683643 state 0x2 ref 0 00000100:00100000:0.0:1789760792.732438:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760792.732513:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554871424 00000100:00100000:1.0:1789760792.732523:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554871424 00000100:00100000:1.0:1789760792.732573:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735927389959 time 735954683643 state 0x0 ref 1 00000100:00100000:1.0:1789760792.732588:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735954944059 time 735954683643 state 0x1 ref 1 00000100:00100000:1.0:1789760792.732862:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735954944059 time 735978478254 state 0x0 ref 1 00000100:00100000:1.0:1789760792.733008:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06401c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554871424:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760792.733288:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67330048, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x201c8 00010000:00010000:1.0:1789760792.733297:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6bb0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.733313:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789760792.733319:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.733325:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6bb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67330048->67334143] (req 67330048->67334143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.733341:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6bb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67330048->67334143] (req 67330048->67334143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.748469:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983747 is committed 00002000:00010000:1.0:1789760792.752404:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67330048 -> 67334144 00002000:00010000:1.0:1789760792.752413:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131528 -> 131536 00010000:00010000:1.0:1789760792.752426:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6bb0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67330048->67334143] (req 67330048->67334143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.752438:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6bb0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67330048->67334143] (req 67330048->67334143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.752448:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6bb0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67330048->67334143] (req 67330048->67334143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.752455:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760792.752460:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6bb0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67330048->67334143] (req 67330048->67334143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.752469:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6bb0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67330048->67334143] (req 67330048->67334143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.752490:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6bb0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67330048->67334143] (req 67330048->67334143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.752504:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6bb0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67330048->67334143] (req 67330048->67334143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.752817:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06401c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554871424:12345-192.168.203.22@tcp:4:dd.0 Request processed in 19938us (20750us total) trans 4294983747 rc 0/0 00000100:00100000:2.0:1789760792.752828:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16449 00000100:00100000:2.0:1789760792.752838:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735954944059 time 735978478254 state 0x2 ref 0 00000100:00100000:0.0:1789760792.760328:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760792.764542:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554871552 00000100:00100000:2.0:1789760792.764558:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554871552 00000100:00100000:2.0:1789760792.764636:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735954944059 time 735978478254 state 0x0 ref 1 00000100:00100000:2.0:1789760792.764650:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735978738670 time 735978478254 state 0x1 ref 1 00000100:00100000:2.0:1789760792.764685:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735978738670 time 736010303417 state 0x0 ref 1 00000100:00100000:2.0:1789760792.764698:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06402d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554871552:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760792.764741:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67334144, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x201d0 00010000:00010000:2.0:1789760792.764750:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6bb7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.764765:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760792.764770:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.764776:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6bb7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67334144->67338239] (req 67334144->67338239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.764792:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6bb7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67334144->67338239] (req 67334144->67338239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.778595:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983748 is committed 00002000:00010000:2.0:1789760792.778942:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67334144 -> 67338240 00002000:00010000:2.0:1789760792.778948:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131536 -> 131544 00010000:00010000:2.0:1789760792.778965:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6bb7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67334144->67338239] (req 67334144->67338239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.778978:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6bb7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67334144->67338239] (req 67334144->67338239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.778987:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6bb7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67334144->67338239] (req 67334144->67338239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.778994:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760792.779000:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6bb7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67334144->67338239] (req 67334144->67338239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.779010:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6bb7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67334144->67338239] (req 67334144->67338239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.779031:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6bb7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67334144->67338239] (req 67334144->67338239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.779039:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6bb7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67334144->67338239] (req 67334144->67338239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.779152:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06402d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554871552:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14451us (18827us total) trans 4294983748 rc 0/0 00000100:00100000:2.0:1789760792.779164:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16450 00000100:00100000:2.0:1789760792.779175:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735978738670 time 736010303417 state 0x2 ref 0 00000100:00100000:1.0:1789760792.784165:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760792.784481:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554871680 00000100:00100000:2.0:1789760792.784493:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554871680 00000100:00100000:2.0:1789760792.784545:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 735978738670 time 736010303417 state 0x0 ref 1 00000100:00100000:2.0:1789760792.784560:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736010563833 time 736010303417 state 0x1 ref 1 00000100:00100000:2.0:1789760792.784762:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736010563833 time 736030379058 state 0x0 ref 1 00000100:00100000:2.0:1789760792.784776:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a386d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554871680:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760792.784823:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67338240, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x201d8 00010000:00010000:2.0:1789760792.784831:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6bbe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.784846:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760792.784850:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.784856:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6bbe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67338240->67342335] (req 67338240->67342335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.784871:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6bbe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67338240->67342335] (req 67338240->67342335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.793245:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983749 is committed 00002000:00010000:0.0:1789760792.793476:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67338240 -> 67342336 00002000:00010000:0.0:1789760792.793481:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131544 -> 131552 00010000:00010000:0.0:1789760792.793496:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6bbe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67338240->67342335] (req 67338240->67342335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.793510:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6bbe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67338240->67342335] (req 67338240->67342335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.793520:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6bbe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67338240->67342335] (req 67338240->67342335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.793528:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760792.793533:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6bbe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67338240->67342335] (req 67338240->67342335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.793556:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6bbe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67338240->67342335] (req 67338240->67342335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.793578:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6bbe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67338240->67342335] (req 67338240->67342335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.793585:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6bbe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67338240->67342335] (req 67338240->67342335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.793871:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a386d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554871680:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9091us (9709us total) trans 4294983749 rc 0/0 00000100:00100000:0.0:1789760792.793884:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16451 00000100:00100000:0.0:1789760792.793897:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736010563833 time 736030379058 state 0x2 ref 0 00000100:00100000:0.0:1789760792.798826:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760792.799297:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554871808 00000100:00100000:1.0:1789760792.799308:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554871808 00000100:00100000:1.0:1789760792.799366:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736010563833 time 736030379058 state 0x0 ref 1 00000100:00100000:1.0:1789760792.799413:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736030639474 time 736030379058 state 0x1 ref 1 00000100:00100000:1.0:1789760792.799448:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736030639474 time 736045066197 state 0x0 ref 1 00000100:00100000:1.0:1789760792.799460:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06400000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554871808:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760792.799509:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67342336, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x201e0 00010000:00010000:1.0:1789760792.799518:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bc5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.799530:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760792.799534:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.799546:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bc5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67342336->67346431] (req 67342336->67346431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.799558:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bc5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67342336->67346431] (req 67342336->67346431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.809288:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983750 is committed 00002000:00010000:1.0:1789760792.809744:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67342336 -> 67346432 00002000:00010000:1.0:1789760792.809749:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131552 -> 131560 00010000:00010000:1.0:1789760792.809763:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bc5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67342336->67346431] (req 67342336->67346431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.809777:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bc5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67342336->67346431] (req 67342336->67346431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.809785:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bc5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67342336->67346431] (req 67342336->67346431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.809793:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760792.809798:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bc5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67342336->67346431] (req 67342336->67346431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.809807:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bc5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67342336->67346431] (req 67342336->67346431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.809827:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bc5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67342336->67346431] (req 67342336->67346431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.809835:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bc5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67342336->67346431] (req 67342336->67346431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.810081:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06400000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554871808:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10617us (11260us total) trans 4294983750 rc 0/0 00000100:00100000:1.0:1789760792.810093:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16452 00000100:00100000:1.0:1789760792.810104:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736030639474 time 736045066197 state 0x2 ref 0 00000100:00100000:0.0:1789760792.822811:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760792.822997:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554871936 00000100:00100000:2.0:1789760792.823006:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554871936 00000100:00100000:2.0:1789760792.823061:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736030639474 time 736045066197 state 0x0 ref 1 00000100:00100000:2.0:1789760792.823075:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736045326613 time 736045066197 state 0x1 ref 1 00000100:00100000:2.0:1789760792.823112:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736045326613 time 736068729957 state 0x0 ref 1 00000100:00100000:2.0:1789760792.823124:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06403100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554871936:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760792.823183:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67346432, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x201e8 00010000:00010000:2.0:1789760792.823192:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6bcc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.823207:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760792.823212:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.823218:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6bcc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67346432->67350527] (req 67346432->67350527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.823231:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6bcc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67346432->67350527] (req 67346432->67350527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.830154:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983751 is committed 00002000:00010000:2.0:1789760792.830547:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67346432 -> 67350528 00002000:00010000:2.0:1789760792.830553:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131560 -> 131568 00010000:00010000:2.0:1789760792.830566:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6bcc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67346432->67350527] (req 67346432->67350527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.830579:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6bcc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67346432->67350527] (req 67346432->67350527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.830587:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6bcc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67346432->67350527] (req 67346432->67350527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.830599:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760792.830748:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6bcc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67346432->67350527] (req 67346432->67350527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.830761:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6bcc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67346432->67350527] (req 67346432->67350527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.830781:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6bcc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67346432->67350527] (req 67346432->67350527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.830790:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6bcc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67346432->67350527] (req 67346432->67350527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.830934:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06403100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554871936:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7805us (8125us total) trans 4294983751 rc 0/0 00000100:00100000:2.0:1789760792.830945:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16453 00000100:00100000:2.0:1789760792.830954:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736045326613 time 736068729957 state 0x2 ref 0 00000100:00100000:2.0:1789760792.836917:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760792.837192:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554872064 00000100:00100000:1.0:1789760792.837201:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554872064 00000100:00100000:1.0:1789760792.837256:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736045326613 time 736068729957 state 0x0 ref 1 00000100:00100000:1.0:1789760792.837271:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736068990373 time 736068729957 state 0x1 ref 1 00000100:00100000:1.0:1789760792.837338:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736068990373 time 736082954976 state 0x0 ref 1 00000100:00100000:1.0:1789760792.837351:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554872064:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760792.837427:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67350528, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x201f0 00010000:00010000:1.0:1789760792.837436:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bd3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.837451:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760792.837456:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.837462:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bd3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67350528->67354623] (req 67350528->67354623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.837475:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bd3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67350528->67354623] (req 67350528->67354623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.846341:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983752 is committed 00002000:00010000:1.0:1789760792.847883:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67350528 -> 67354624 00002000:00010000:1.0:1789760792.847892:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131568 -> 131576 00010000:00010000:1.0:1789760792.847906:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bd3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67350528->67354623] (req 67350528->67354623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.847919:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bd3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67350528->67354623] (req 67350528->67354623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.847928:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bd3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67350528->67354623] (req 67350528->67354623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.847936:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760792.847941:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bd3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67350528->67354623] (req 67350528->67354623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.847950:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bd3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67350528->67354623] (req 67350528->67354623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.847970:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bd3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67350528->67354623] (req 67350528->67354623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.848093:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bd3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67350528->67354623] (req 67350528->67354623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.848243:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554872064:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10889us (11331us total) trans 4294983752 rc 0/0 00000100:00100000:1.0:1789760792.848254:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16454 00000100:00100000:1.0:1789760792.848266:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736068990373 time 736082954976 state 0x2 ref 0 00000100:00100000:0.0:1789760792.860142:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760792.860463:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554872192 00000100:00100000:1.0:1789760792.860474:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554872192 00000100:00100000:1.0:1789760792.860532:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736068990373 time 736082954976 state 0x0 ref 1 00000100:00100000:1.0:1789760792.860559:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736083215392 time 736082954976 state 0x1 ref 1 00000100:00100000:1.0:1789760792.860590:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736083215392 time 736106208997 state 0x0 ref 1 00000100:00100000:1.0:1789760792.860614:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06400a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554872192:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760792.860660:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67354624, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x201f8 00010000:00010000:1.0:1789760792.860670:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bda lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.860685:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760792.860690:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.860696:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67354624->67358719] (req 67354624->67358719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.860710:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67354624->67358719] (req 67354624->67358719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.866734:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983753 is committed 00002000:00010000:1.0:1789760792.866853:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67354624 -> 67358720 00002000:00010000:1.0:1789760792.866859:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131576 -> 131584 00010000:00010000:1.0:1789760792.866873:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bda lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67354624->67358719] (req 67354624->67358719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.866893:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bda lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67354624->67358719] (req 67354624->67358719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.866903:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bda lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67354624->67358719] (req 67354624->67358719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.866911:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760792.866917:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bda lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67354624->67358719] (req 67354624->67358719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.866926:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bda lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67354624->67358719] (req 67354624->67358719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.866947:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bda lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67354624->67358719] (req 67354624->67358719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.866956:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6bda lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67354624->67358719] (req 67354624->67358719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.867197:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06400a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554872192:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6591us (7060us total) trans 4294983753 rc 0/0 00000100:00100000:1.0:1789760792.867210:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16455 00000100:00100000:1.0:1789760792.867221:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736083215392 time 736106208997 state 0x2 ref 0 00000100:00100000:0.0:1789760792.872338:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760792.872550:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554872320 00000100:00100000:2.0:1789760792.872560:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554872320 00000100:00100000:2.0:1789760792.872623:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736083215392 time 736106208997 state 0x0 ref 1 00000100:00100000:2.0:1789760792.872638:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736106469413 time 736106208997 state 0x1 ref 1 00000100:00100000:2.0:1789760792.872701:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736106469413 time 736118318610 state 0x0 ref 1 00000100:00100000:2.0:1789760792.872713:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06402680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554872320:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760792.872760:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67358720, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20200 00010000:00010000:2.0:1789760792.872769:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6be1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.872784:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760792.872788:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.872794:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6be1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67358720->67362815] (req 67358720->67362815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.872806:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6be1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67358720->67362815] (req 67358720->67362815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760792.881786:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983754 is committed 00002000:00010000:1.0:1789760792.881907:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67358720 -> 67362816 00002000:00010000:1.0:1789760792.881914:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131584 -> 131592 00010000:00010000:1.0:1789760792.881927:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6be1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67358720->67362815] (req 67358720->67362815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.881940:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6be1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67358720->67362815] (req 67358720->67362815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.881958:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6be1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67358720->67362815] (req 67358720->67362815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.881966:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760792.881971:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6be1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67358720->67362815] (req 67358720->67362815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.882205:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6be1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67358720->67362815] (req 67358720->67362815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.882225:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6be1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67358720->67362815] (req 67358720->67362815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.882234:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6be1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67358720->67362815] (req 67358720->67362815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.882330:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06402680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554872320:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9614us (9996us total) trans 4294983754 rc 0/0 00000100:00100000:1.0:1789760792.882342:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16456 00000100:00100000:1.0:1789760792.882351:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736106469413 time 736118318610 state 0x2 ref 0 00000100:00100000:2.0:1789760792.887905:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760792.889517:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554872448 00000100:00100000:0.0:1789760792.889531:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554872448 00000100:00100000:0.0:1789760792.889593:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736106469413 time 736118318610 state 0x0 ref 1 00000100:00100000:0.0:1789760792.889622:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736118579026 time 736118318610 state 0x1 ref 1 00000100:00100000:0.0:1789760792.889814:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736118579026 time 736135431060 state 0x0 ref 1 00000100:00100000:0.0:1789760792.889829:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554872448:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760792.889880:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67362816, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20208 00010000:00010000:0.0:1789760792.889889:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6be8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.889906:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760792.889911:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.889917:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6be8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67362816->67366911] (req 67362816->67366911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.889931:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6be8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67362816->67366911] (req 67362816->67366911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.895570:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983755 is committed 00002000:00010000:0.0:1789760792.895795:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67362816 -> 67366912 00002000:00010000:0.0:1789760792.895801:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131592 -> 131600 00010000:00010000:0.0:1789760792.895813:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6be8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67362816->67366911] (req 67362816->67366911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.895830:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6be8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67362816->67366911] (req 67362816->67366911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.895838:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6be8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67362816->67366911] (req 67362816->67366911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.895845:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760792.895848:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6be8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67362816->67366911] (req 67362816->67366911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.895857:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6be8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67362816->67366911] (req 67362816->67366911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.895877:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6be8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67362816->67366911] (req 67362816->67366911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.895886:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6be8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67362816->67366911] (req 67362816->67366911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.896000:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554872448:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6169us (8100us total) trans 4294983755 rc 0/0 00000100:00100000:0.0:1789760792.896011:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16457 00000100:00100000:0.0:1789760792.896021:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736118579026 time 736135431060 state 0x2 ref 0 00000100:00100000:0.0:1789760792.904859:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760792.904960:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554872576 00000100:00100000:1.0:1789760792.904971:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554872576 00000100:00100000:1.0:1789760792.905023:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736118579026 time 736135431060 state 0x0 ref 1 00000100:00100000:1.0:1789760792.905039:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736135691476 time 736135431060 state 0x1 ref 1 00000100:00100000:1.0:1789760792.905108:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736135691476 time 736150726631 state 0x0 ref 1 00000100:00100000:1.0:1789760792.905121:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06401500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554872576:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760792.905165:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67366912, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20210 00010000:00010000:1.0:1789760792.905174:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6bef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.905191:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760792.905195:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.905201:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6bef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67366912->67371007] (req 67366912->67371007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.905215:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6bef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67366912->67371007] (req 67366912->67371007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760792.918330:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983756 is committed 00002000:00010000:1.0:1789760792.918669:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67366912 -> 67371008 00002000:00010000:1.0:1789760792.918674:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131600 -> 131608 00010000:00010000:1.0:1789760792.918687:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6bef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67366912->67371007] (req 67366912->67371007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.918707:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6bef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67366912->67371007] (req 67366912->67371007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.918717:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6bef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67366912->67371007] (req 67366912->67371007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.918724:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760792.918728:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6bef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67366912->67371007] (req 67366912->67371007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.918737:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6bef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67366912->67371007] (req 67366912->67371007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.918757:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6bef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67366912->67371007] (req 67366912->67371007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.918765:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6bef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67366912->67371007] (req 67366912->67371007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.918855:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06401500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554872576:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13732us (14001us total) trans 4294983756 rc 0/0 00000100:00100000:1.0:1789760792.918865:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16458 00000100:00100000:1.0:1789760792.918873:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736135691476 time 736150726631 state 0x2 ref 0 00000100:00100000:1.0:1789760792.922167:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760792.922690:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554872704 00000100:00100000:0.0:1789760792.922700:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554872704 00000100:00100000:0.0:1789760792.922755:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736135691476 time 736150726631 state 0x0 ref 1 00000100:00100000:0.0:1789760792.922770:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736150987047 time 736150726631 state 0x1 ref 1 00000100:00100000:0.0:1789760792.922802:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736150987047 time 736168420072 state 0x0 ref 1 00000100:00100000:0.0:1789760792.922814:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554872704:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760792.922860:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67371008, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20218 00010000:00010000:0.0:1789760792.922869:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6bf6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.922886:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760792.922891:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.922897:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6bf6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67371008->67375103] (req 67371008->67375103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.922911:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6bf6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67371008->67375103] (req 67371008->67375103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.934027:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983757 is committed 00002000:00010000:0.0:1789760792.934254:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67371008 -> 67375104 00002000:00010000:0.0:1789760792.934260:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131608 -> 131616 00010000:00010000:0.0:1789760792.934273:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6bf6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67371008->67375103] (req 67371008->67375103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.934286:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6bf6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67371008->67375103] (req 67371008->67375103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.934295:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6bf6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67371008->67375103] (req 67371008->67375103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.934303:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760792.934308:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6bf6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67371008->67375103] (req 67371008->67375103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.934317:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6bf6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67371008->67375103] (req 67371008->67375103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.934336:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6bf6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67371008->67375103] (req 67371008->67375103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.934343:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6bf6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67371008->67375103] (req 67371008->67375103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.934541:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554872704:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11724us (12379us total) trans 4294983757 rc 0/0 00000100:00100000:0.0:1789760792.934553:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16459 00000100:00100000:0.0:1789760792.934562:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736150987047 time 736168420072 state 0x2 ref 0 00000100:00100000:2.0:1789760792.938090:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760792.938538:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554872832 00000100:00100000:0.0:1789760792.938548:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554872832 00000100:00100000:0.0:1789760792.938668:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736150987047 time 736168420072 state 0x0 ref 1 00000100:00100000:0.0:1789760792.938684:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736168680488 time 736168420072 state 0x1 ref 1 00000100:00100000:0.0:1789760792.938752:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736168680488 time 736184369460 state 0x0 ref 1 00000100:00100000:0.0:1789760792.938766:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554872832:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760792.938812:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67375104, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20220 00010000:00010000:0.0:1789760792.938821:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6bfd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.938838:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760792.938843:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.938849:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6bfd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67375104->67379199] (req 67375104->67379199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.938862:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6bfd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67375104->67379199] (req 67375104->67379199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.944521:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983758 is committed 00002000:00010000:0.0:1789760792.944933:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67375104 -> 67379200 00002000:00010000:0.0:1789760792.944939:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131616 -> 131624 00010000:00010000:0.0:1789760792.944952:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6bfd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67375104->67379199] (req 67375104->67379199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.944965:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6bfd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67375104->67379199] (req 67375104->67379199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.944973:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6bfd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67375104->67379199] (req 67375104->67379199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.945063:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760792.945069:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6bfd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67375104->67379199] (req 67375104->67379199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.945523:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6bfd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67375104->67379199] (req 67375104->67379199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.945548:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6bfd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67375104->67379199] (req 67375104->67379199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.945557:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6bfd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67375104->67379199] (req 67375104->67379199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.945712:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554872832:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6944us (7627us total) trans 4294983758 rc 0/0 00000100:00100000:0.0:1789760792.945722:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16460 00000100:00100000:0.0:1789760792.945731:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736168680488 time 736184369460 state 0x2 ref 0 00000100:00100000:2.0:1789760792.953295:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760792.954002:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554872960 00000100:00100000:0.0:1789760792.954013:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554872960 00000100:00100000:0.0:1789760792.954071:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736168680488 time 736184369460 state 0x0 ref 1 00000100:00100000:0.0:1789760792.954089:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736184629876 time 736184369460 state 0x1 ref 1 00000100:00100000:0.0:1789760792.954932:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736184629876 time 736200544397 state 0x0 ref 1 00000100:00100000:0.0:1789760792.954957:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554872960:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760792.955014:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67379200, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20228 00010000:00010000:0.0:1789760792.955024:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6c04 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.955041:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760792.955045:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760792.955051:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6c04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67379200->67383295] (req 67379200->67383295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.955066:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6c04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67379200->67383295] (req 67379200->67383295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.960916:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983759 is committed 00002000:00010000:0.0:1789760792.961256:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67379200 -> 67383296 00002000:00010000:0.0:1789760792.961262:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131624 -> 131632 00010000:00010000:0.0:1789760792.961276:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6c04 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67379200->67383295] (req 67379200->67383295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.961290:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6c04 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67379200->67383295] (req 67379200->67383295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.961298:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6c04 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67379200->67383295] (req 67379200->67383295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.961305:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760792.961310:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6c04 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67379200->67383295] (req 67379200->67383295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.961318:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6c04 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67379200->67383295] (req 67379200->67383295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.961338:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6c04 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67379200->67383295] (req 67379200->67383295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760792.961346:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6c04 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67379200->67383295] (req 67379200->67383295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760792.961921:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554872960:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6964us (8629us total) trans 4294983759 rc 0/0 00000100:00100000:0.0:1789760792.961932:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16461 00000100:00100000:0.0:1789760792.961942:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736184629876 time 736200544397 state 0x2 ref 0 00000100:00100000:1.0:1789760792.967772:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760792.967861:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554873088 00000100:00100000:2.0:1789760792.967871:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554873088 00000100:00100000:2.0:1789760792.967928:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736184629876 time 736200544397 state 0x0 ref 1 00000100:00100000:2.0:1789760792.967944:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736200804813 time 736200544397 state 0x1 ref 1 00000100:00100000:2.0:1789760792.968078:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736200804813 time 736213596755 state 0x0 ref 1 00000100:00100000:2.0:1789760792.968094:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a387480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554873088:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760792.968152:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67383296, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20230 00010000:00010000:2.0:1789760792.968161:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6c0b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.968186:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760792.968190:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760792.968197:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6c0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67383296->67387391] (req 67383296->67387391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.968211:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6c0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67383296->67387391] (req 67383296->67387391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.973306:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983760 is committed 00002000:00010000:2.0:1789760792.973437:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67383296 -> 67387392 00002000:00010000:2.0:1789760792.973443:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131632 -> 131640 00010000:00010000:2.0:1789760792.973456:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6c0b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67383296->67387391] (req 67383296->67387391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.973469:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6c0b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67383296->67387391] (req 67383296->67387391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.973477:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6c0b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67383296->67387391] (req 67383296->67387391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.973483:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760792.973488:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6c0b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67383296->67387391] (req 67383296->67387391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.973497:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6c0b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67383296->67387391] (req 67383296->67387391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.973515:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6c0b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67383296->67387391] (req 67383296->67387391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760792.973523:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6c0b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67383296->67387391] (req 67383296->67387391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760792.973629:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a387480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554873088:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5532us (5860us total) trans 4294983760 rc 0/0 00000100:00100000:2.0:1789760792.973640:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16462 00000100:00100000:2.0:1789760792.973649:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736200804813 time 736213596755 state 0x2 ref 0 00000100:00100000:2.0:1789760792.979362:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760792.979621:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554873216 00000100:00100000:1.0:1789760792.979632:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554873216 00000100:00100000:1.0:1789760792.979688:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736200804813 time 736213596755 state 0x0 ref 1 00000100:00100000:1.0:1789760792.979704:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736213857171 time 736213596755 state 0x1 ref 1 00000100:00100000:1.0:1789760792.979773:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736213857171 time 736225390183 state 0x0 ref 1 00000100:00100000:1.0:1789760792.979786:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554873216:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760792.979832:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67387392, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20238 00010000:00010000:1.0:1789760792.979841:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6c12 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.979858:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760792.979862:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.979869:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6c12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67387392->67391487] (req 67387392->67391487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.979891:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6c12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67387392->67391487] (req 67387392->67391487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.988146:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983761 is committed 00002000:00010000:1.0:1789760792.988522:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67387392 -> 67391488 00002000:00010000:1.0:1789760792.988528:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131640 -> 131648 00010000:00010000:1.0:1789760792.988542:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6c12 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67387392->67391487] (req 67387392->67391487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.988562:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6c12 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67387392->67391487] (req 67387392->67391487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.988571:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6c12 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67387392->67391487] (req 67387392->67391487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.988578:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760792.988584:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6c12 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67387392->67391487] (req 67387392->67391487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.988593:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6c12 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67387392->67391487] (req 67387392->67391487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.988621:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6c12 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67387392->67391487] (req 67387392->67391487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.988629:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c6c12 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67387392->67391487] (req 67387392->67391487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.988724:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554873216:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8936us (9368us total) trans 4294983761 rc 0/0 00000100:00100000:1.0:1789760792.988735:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16463 00000100:00100000:1.0:1789760792.988744:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736213857171 time 736225390183 state 0x2 ref 0 00000100:00100000:0.0:1789760792.992971:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760792.993058:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554873344 00000100:00100000:1.0:1789760792.993070:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554873344 00000100:00100000:1.0:1789760792.993122:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736213857171 time 736225390183 state 0x0 ref 1 00000100:00100000:1.0:1789760792.993137:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736225650599 time 736225390183 state 0x1 ref 1 00000100:00100000:1.0:1789760792.993179:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736225650599 time 736238788386 state 0x0 ref 1 00000100:00100000:1.0:1789760792.993191:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06401880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554873344:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760792.993237:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67391488, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20240 00010000:00010000:1.0:1789760792.993246:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c19 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.993261:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760792.993265:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760792.993271:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67391488->67395583] (req 67391488->67395583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.993284:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67391488->67395583] (req 67391488->67395583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760792.998580:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983762 is committed 00002000:00010000:1.0:1789760792.998704:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67391488 -> 67395584 00002000:00010000:1.0:1789760792.998710:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131648 -> 131656 00010000:00010000:1.0:1789760792.998722:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c19 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67391488->67395583] (req 67391488->67395583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.998734:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c19 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67391488->67395583] (req 67391488->67395583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.998758:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c19 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67391488->67395583] (req 67391488->67395583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.998766:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760792.998771:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c19 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67391488->67395583] (req 67391488->67395583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.998780:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c19 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67391488->67395583] (req 67391488->67395583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.998805:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c19 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67391488->67395583] (req 67391488->67395583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760792.998812:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c19 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67391488->67395583] (req 67391488->67395583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760792.998946:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06401880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554873344:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5752us (5980us total) trans 4294983762 rc 0/0 00000100:00100000:1.0:1789760792.998957:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16464 00000100:00100000:1.0:1789760792.998967:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736225650599 time 736238788386 state 0x2 ref 0 00000100:00100000:0.0:1789760793.009923:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760793.010084:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554873472 00000100:00100000:2.0:1789760793.010094:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554873472 00000100:00100000:2.0:1789760793.010165:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736225650599 time 736238788386 state 0x0 ref 1 00000100:00100000:2.0:1789760793.010194:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736239048802 time 736238788386 state 0x1 ref 1 00000100:00100000:2.0:1789760793.010226:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736239048802 time 736255844038 state 0x0 ref 1 00000100:00100000:2.0:1789760793.010238:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e92a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554873472:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760793.010286:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67395584, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20248 00010000:00010000:2.0:1789760793.010295:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6c20 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.010311:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760793.010316:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.010322:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6c20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67395584->67399679] (req 67395584->67399679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.010335:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6c20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67395584->67399679] (req 67395584->67399679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789760793.010924:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789760793.016070:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983763 is committed 00002000:00010000:2.0:1789760793.017569:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67395584 -> 67399680 00002000:00010000:2.0:1789760793.017577:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131656 -> 131664 00010000:00010000:2.0:1789760793.017591:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6c20 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67395584->67399679] (req 67395584->67399679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.017674:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6c20 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67395584->67399679] (req 67395584->67399679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.017683:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6c20 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67395584->67399679] (req 67395584->67399679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.017691:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760793.017705:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6c20 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67395584->67399679] (req 67395584->67399679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.017720:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6c20 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67395584->67399679] (req 67395584->67399679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.017740:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6c20 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67395584->67399679] (req 67395584->67399679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.017748:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6c20 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67395584->67399679] (req 67395584->67399679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.017848:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e92a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554873472:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7608us (7931us total) trans 4294983763 rc 0/0 00000100:00100000:2.0:1789760793.017859:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16465 00000100:00100000:2.0:1789760793.017868:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736239048802 time 736255844038 state 0x2 ref 0 00000100:00100000:2.0:1789760793.022626:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760793.022729:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554873600 00000100:00100000:1.0:1789760793.022739:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554873600 00000100:00100000:1.0:1789760793.022793:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736239048802 time 736255844038 state 0x0 ref 1 00000100:00100000:1.0:1789760793.022812:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736256104454 time 736255844038 state 0x1 ref 1 00000100:00100000:1.0:1789760793.022844:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736256104454 time 736268462142 state 0x0 ref 1 00000100:00100000:1.0:1789760793.022856:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554873600:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760793.023020:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67399680, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20250 00010000:00010000:1.0:1789760793.023030:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c27 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.023046:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760793.023051:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.023057:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67399680->67403775] (req 67399680->67403775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.023071:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67399680->67403775] (req 67399680->67403775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.029704:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983764 is committed 00002000:00010000:1.0:1789760793.029885:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67399680 -> 67403776 00002000:00010000:1.0:1789760793.029891:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131664 -> 131672 00010000:00010000:1.0:1789760793.029905:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c27 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67399680->67403775] (req 67399680->67403775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.029919:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c27 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67399680->67403775] (req 67399680->67403775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.029927:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c27 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67399680->67403775] (req 67399680->67403775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.029935:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.029941:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c27 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67399680->67403775] (req 67399680->67403775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.029950:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c27 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67399680->67403775] (req 67399680->67403775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.029970:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c27 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67399680->67403775] (req 67399680->67403775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.030041:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c27 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67399680->67403775] (req 67399680->67403775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.030304:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554873600:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7444us (7681us total) trans 4294983764 rc 0/0 00000100:00100000:1.0:1789760793.030315:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16466 00000100:00100000:1.0:1789760793.030325:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736256104454 time 736268462142 state 0x2 ref 0 00000100:00100000:0.0:1789760793.047253:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760793.047368:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554873728 00000100:00100000:1.0:1789760793.047407:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554873728 00000100:00100000:1.0:1789760793.047464:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736256104454 time 736268462142 state 0x0 ref 1 00000100:00100000:1.0:1789760793.047479:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736268722558 time 736268462142 state 0x1 ref 1 00000100:00100000:1.0:1789760793.047511:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736268722558 time 736293129613 state 0x0 ref 1 00000100:00100000:1.0:1789760793.047527:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554873728:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760793.047575:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67403776, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20258 00010000:00010000:1.0:1789760793.047584:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c2e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.047613:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760793.047618:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.047624:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67403776->67407871] (req 67403776->67407871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.047639:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67403776->67407871] (req 67403776->67407871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.053728:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983765 is committed 00002000:00010000:1.0:1789760793.053851:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67403776 -> 67407872 00002000:00010000:1.0:1789760793.053856:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131672 -> 131680 00010000:00010000:1.0:1789760793.053870:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c2e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67403776->67407871] (req 67403776->67407871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.053892:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c2e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67403776->67407871] (req 67403776->67407871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.053901:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67403776->67407871] (req 67403776->67407871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.053908:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.053913:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c2e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67403776->67407871] (req 67403776->67407871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.053922:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c2e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67403776->67407871] (req 67403776->67407871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.053941:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c2e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67403776->67407871] (req 67403776->67407871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.053949:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c2e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67403776->67407871] (req 67403776->67407871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.054047:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554873728:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6518us (6800us total) trans 4294983765 rc 0/0 00000100:00100000:1.0:1789760793.054059:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16467 00000100:00100000:1.0:1789760793.054068:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736268722558 time 736293129613 state 0x2 ref 0 00000100:00100000:0.0:1789760793.060254:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760793.060342:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554873856 00000100:00100000:1.0:1789760793.060352:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554873856 00000100:00100000:1.0:1789760793.060448:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736268722558 time 736293129613 state 0x0 ref 1 00000100:00100000:1.0:1789760793.060466:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736293390029 time 736293129613 state 0x1 ref 1 00000100:00100000:1.0:1789760793.060500:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736293390029 time 736306118493 state 0x0 ref 1 00000100:00100000:1.0:1789760793.060512:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e93800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554873856:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760793.060559:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67407872, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20260 00010000:00010000:1.0:1789760793.060569:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c35 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.060586:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760793.060591:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.060597:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67407872->67411967] (req 67407872->67411967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.060623:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67407872->67411967] (req 67407872->67411967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.065945:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983766 is committed 00002000:00010000:1.0:1789760793.066454:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67407872 -> 67411968 00002000:00010000:1.0:1789760793.066460:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131680 -> 131688 00010000:00010000:1.0:1789760793.066474:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c35 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67407872->67411967] (req 67407872->67411967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.066487:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c35 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67407872->67411967] (req 67407872->67411967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.066495:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c35 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67407872->67411967] (req 67407872->67411967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.066502:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.066507:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c35 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67407872->67411967] (req 67407872->67411967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.066515:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c35 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67407872->67411967] (req 67407872->67411967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.066535:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c35 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67407872->67411967] (req 67407872->67411967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.066544:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c6c35 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67407872->67411967] (req 67407872->67411967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.066637:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e93800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554873856:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6121us (6389us total) trans 4294983766 rc 0/0 00000100:00100000:1.0:1789760793.066647:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16468 00000100:00100000:1.0:1789760793.066656:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736293390029 time 736306118493 state 0x2 ref 0 00000100:00100000:2.0:1789760793.073353:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760793.073551:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554873984 00000100:00100000:0.0:1789760793.073562:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554873984 00000100:00100000:0.0:1789760793.073625:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736293390029 time 736306118493 state 0x0 ref 1 00000100:00100000:0.0:1789760793.073643:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736306378909 time 736306118493 state 0x1 ref 1 00000100:00100000:0.0:1789760793.073674:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736306378909 time 736319293116 state 0x0 ref 1 00000100:00100000:0.0:1789760793.073685:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554873984:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760793.073726:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67411968, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20268 00010000:00010000:0.0:1789760793.073735:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c3c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.074312:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760793.074319:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.074326:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c3c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67411968->67416063] (req 67411968->67416063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.074342:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c3c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67411968->67416063] (req 67411968->67416063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.080409:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983767 is committed 00002000:00010000:0.0:1789760793.080523:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67411968 -> 67416064 00002000:00010000:0.0:1789760793.080528:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131688 -> 131696 00010000:00010000:0.0:1789760793.080542:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c3c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67411968->67416063] (req 67411968->67416063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.080555:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c3c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67411968->67416063] (req 67411968->67416063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.080565:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c3c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67411968->67416063] (req 67411968->67416063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.080573:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760793.080578:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c3c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67411968->67416063] (req 67411968->67416063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.080587:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c3c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67411968->67416063] (req 67411968->67416063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.080617:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c3c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67411968->67416063] (req 67411968->67416063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.080627:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c3c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67411968->67416063] (req 67411968->67416063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.080721:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554873984:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7032us (7372us total) trans 4294983767 rc 0/0 00000100:00100000:0.0:1789760793.080731:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16469 00000100:00100000:0.0:1789760793.080742:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736306378909 time 736319293116 state 0x2 ref 0 00000100:00100000:0.0:1789760793.088850:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760793.089090:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554874112 00000100:00100000:2.0:1789760793.089101:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554874112 00000100:00100000:2.0:1789760793.089232:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736306378909 time 736319293116 state 0x0 ref 1 00000100:00100000:2.0:1789760793.089251:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736319553532 time 736319293116 state 0x1 ref 1 00000100:00100000:2.0:1789760793.089289:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736319553532 time 736334906735 state 0x0 ref 1 00000100:00100000:2.0:1789760793.089303:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554874112:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760793.089902:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67416064, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20270 00010000:00010000:2.0:1789760793.089915:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6c43 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.089932:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760793.089936:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.089943:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6c43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67416064->67420159] (req 67416064->67420159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.089957:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6c43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67416064->67420159] (req 67416064->67420159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.101043:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983768 is committed 00002000:00010000:2.0:1789760793.101250:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67416064 -> 67420160 00002000:00010000:2.0:1789760793.101256:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131696 -> 131704 00010000:00010000:2.0:1789760793.101269:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6c43 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67416064->67420159] (req 67416064->67420159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.101282:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6c43 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67416064->67420159] (req 67416064->67420159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.101291:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6c43 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67416064->67420159] (req 67416064->67420159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.101298:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760793.101303:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6c43 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67416064->67420159] (req 67416064->67420159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.101316:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6c43 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67416064->67420159] (req 67416064->67420159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.101334:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6c43 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67416064->67420159] (req 67416064->67420159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.101341:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6c43 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67416064->67420159] (req 67416064->67420159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.101491:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554874112:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12186us (12646us total) trans 4294983768 rc 0/0 00000100:00100000:2.0:1789760793.101502:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16470 00000100:00100000:2.0:1789760793.101511:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736319553532 time 736334906735 state 0x2 ref 0 00000100:00100000:1.0:1789760793.110950:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760793.111046:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554874240 00000100:00100000:0.0:1789760793.111057:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554874240 00000100:00100000:0.0:1789760793.111111:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736319553532 time 736334906735 state 0x0 ref 1 00000100:00100000:0.0:1789760793.112357:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736335167151 time 736334906735 state 0x1 ref 1 00000100:00100000:0.0:1789760793.112430:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736335167151 time 736358045265 state 0x0 ref 1 00000100:00100000:0.0:1789760793.112448:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554874240:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760793.112501:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67420160, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20278 00010000:00010000:0.0:1789760793.112511:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c4a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.112528:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760793.112532:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.112538:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67420160->67424255] (req 67420160->67424255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.112754:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67420160->67424255] (req 67420160->67424255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.128414:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983769 is committed 00002000:00010000:0.0:1789760793.128527:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67420160 -> 67424256 00002000:00010000:0.0:1789760793.128533:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131704 -> 131712 00010000:00010000:0.0:1789760793.128545:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c4a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67420160->67424255] (req 67420160->67424255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.128557:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c4a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67420160->67424255] (req 67420160->67424255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.128570:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c4a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67420160->67424255] (req 67420160->67424255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.128583:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760793.128588:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c4a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67420160->67424255] (req 67420160->67424255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.128597:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c4a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67420160->67424255] (req 67420160->67424255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.128626:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c4a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67420160->67424255] (req 67420160->67424255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.128633:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c4a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67420160->67424255] (req 67420160->67424255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.128770:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554874240:12345-192.168.203.22@tcp:4:dd.0 Request processed in 16323us (17824us total) trans 4294983769 rc 0/0 00000100:00100000:0.0:1789760793.128781:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16471 00000100:00100000:0.0:1789760793.128800:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736335167151 time 736358045265 state 0x2 ref 0 00000100:00100000:1.0:1789760793.135718:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760793.135808:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554874368 00000100:00100000:0.0:1789760793.135818:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554874368 00000100:00100000:0.0:1789760793.135875:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736335167151 time 736358045265 state 0x0 ref 1 00000100:00100000:0.0:1789760793.135890:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736358305681 time 736358045265 state 0x1 ref 1 00000100:00100000:0.0:1789760793.135923:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736358305681 time 736381541448 state 0x0 ref 1 00000100:00100000:0.0:1789760793.135938:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554874368:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760793.135986:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67424256, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20280 00010000:00010000:0.0:1789760793.135995:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c51 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.136012:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760793.136016:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.136023:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67424256->67428351] (req 67424256->67428351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.136036:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67424256->67428351] (req 67424256->67428351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.144543:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983770 is committed 00002000:00010000:0.0:1789760793.144684:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67424256 -> 67428352 00002000:00010000:0.0:1789760793.144689:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131712 -> 131720 00010000:00010000:0.0:1789760793.144702:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c51 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67424256->67428351] (req 67424256->67428351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.144715:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c51 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67424256->67428351] (req 67424256->67428351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.144726:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c51 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67424256->67428351] (req 67424256->67428351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.144733:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760793.144739:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c51 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67424256->67428351] (req 67424256->67428351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.144748:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c51 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67424256->67428351] (req 67424256->67428351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.144767:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c51 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67424256->67428351] (req 67424256->67428351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.144775:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c51 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67424256->67428351] (req 67424256->67428351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.144919:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554874368:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8978us (9205us total) trans 4294983770 rc 0/0 00000100:00100000:0.0:1789760793.144930:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16472 00000100:00100000:0.0:1789760793.144940:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736358305681 time 736381541448 state 0x2 ref 0 00000100:00100000:2.0:1789760793.150612:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760793.152390:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554874496 00000100:00100000:0.0:1789760793.152406:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554874496 00000100:00100000:0.0:1789760793.152467:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736358305681 time 736381541448 state 0x0 ref 1 00000100:00100000:0.0:1789760793.152481:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736381801864 time 736381541448 state 0x1 ref 1 00000100:00100000:0.0:1789760793.152506:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736381801864 time 736398124495 state 0x0 ref 1 00000100:00100000:0.0:1789760793.152518:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554874496:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760793.152563:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67428352, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20288 00010000:00010000:0.0:1789760793.152572:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c58 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.152586:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760793.152591:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.152597:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67428352->67432447] (req 67428352->67432447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.152932:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67428352->67432447] (req 67428352->67432447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.162442:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983771 is committed 00002000:00010000:0.0:1789760793.162547:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67428352 -> 67432448 00002000:00010000:0.0:1789760793.162554:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131720 -> 131728 00010000:00010000:0.0:1789760793.162566:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c58 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67428352->67432447] (req 67428352->67432447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.162579:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c58 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67428352->67432447] (req 67428352->67432447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.162587:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67428352->67432447] (req 67428352->67432447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.162594:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760793.162608:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c58 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67428352->67432447] (req 67428352->67432447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.162617:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c58 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67428352->67432447] (req 67428352->67432447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.162638:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c58 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67428352->67432447] (req 67428352->67432447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.162649:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c58 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67428352->67432447] (req 67428352->67432447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.162751:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554874496:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10230us (12157us total) trans 4294983771 rc 0/0 00000100:00100000:0.0:1789760793.162760:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16473 00000100:00100000:0.0:1789760793.162769:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736381801864 time 736398124495 state 0x2 ref 0 00010000:00010000:3.0:1789760793.170488:0:8453:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789760793.172560:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760793.172768:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554874624 00000100:00100000:2.0:1789760793.172778:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554874624 00000100:00100000:2.0:1789760793.172830:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736381801864 time 736398124495 state 0x0 ref 1 00000100:00100000:2.0:1789760793.172844:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736398384911 time 736398124495 state 0x1 ref 1 00000100:00100000:2.0:1789760793.172875:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736398384911 time 736418494243 state 0x0 ref 1 00000100:00100000:2.0:1789760793.172886:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554874624:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760793.172932:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67432448, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20290 00010000:00010000:2.0:1789760793.172942:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6c5f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.172957:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760793.172962:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.172968:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6c5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67432448->67436543] (req 67432448->67436543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.173933:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6c5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67432448->67436543] (req 67432448->67436543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.191420:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983772 is committed 00002000:00010000:1.0:1789760793.191745:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67432448 -> 67436544 00002000:00010000:1.0:1789760793.191750:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131728 -> 131736 00010000:00010000:1.0:1789760793.191763:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6c5f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67432448->67436543] (req 67432448->67436543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.191775:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6c5f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67432448->67436543] (req 67432448->67436543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.191784:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6c5f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67432448->67436543] (req 67432448->67436543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.191791:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.191796:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6c5f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67432448->67436543] (req 67432448->67436543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.191806:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6c5f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67432448->67436543] (req 67432448->67436543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.191825:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6c5f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67432448->67436543] (req 67432448->67436543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.191833:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c6c5f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67432448->67436543] (req 67432448->67436543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.191967:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554874624:12345-192.168.203.22@tcp:4:dd.0 Request processed in 19077us (19412us total) trans 4294983772 rc 0/0 00000100:00100000:1.0:1789760793.191978:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16474 00000100:00100000:1.0:1789760793.191987:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736398384911 time 736418494243 state 0x2 ref 0 00000100:00100000:1.0:1789760793.201858:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760793.202024:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554874752 00000100:00100000:0.0:1789760793.202045:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554874752 00000100:00100000:0.0:1789760793.202118:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736398384911 time 736418494243 state 0x0 ref 1 00000100:00100000:0.0:1789760793.202135:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736418754659 time 736418494243 state 0x1 ref 1 00000100:00100000:0.0:1789760793.202168:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736418754659 time 736447786306 state 0x0 ref 1 00000100:00100000:0.0:1789760793.202180:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554874752:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760793.202235:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67436544, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20298 00010000:00010000:0.0:1789760793.202244:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c66 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.202260:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760793.202265:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.202271:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67436544->67440639] (req 67436544->67440639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.202292:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67436544->67440639] (req 67436544->67440639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.212362:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983773 is committed 00002000:00010000:1.0:1789760793.213026:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67436544 -> 67440640 00002000:00010000:1.0:1789760793.213041:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131736 -> 131744 00010000:00010000:1.0:1789760793.213055:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c66 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67436544->67440639] (req 67436544->67440639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.213069:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c66 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67436544->67440639] (req 67436544->67440639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.213079:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c66 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67436544->67440639] (req 67436544->67440639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.213088:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.213098:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c66 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67436544->67440639] (req 67436544->67440639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.213107:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c66 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67436544->67440639] (req 67436544->67440639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.213127:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c66 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67436544->67440639] (req 67436544->67440639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.213136:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6c66 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67436544->67440639] (req 67436544->67440639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.213235:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554874752:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11051us (11383us total) trans 4294983773 rc 0/0 00000100:00100000:1.0:1789760793.213247:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16475 00000100:00100000:1.0:1789760793.213257:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736418754659 time 736447786306 state 0x2 ref 0 00000100:00100000:2.0:1789760793.222422:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760793.222544:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554874880 00000100:00100000:0.0:1789760793.222555:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554874880 00000100:00100000:0.0:1789760793.222625:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736418754659 time 736447786306 state 0x0 ref 1 00000100:00100000:0.0:1789760793.222640:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736448046722 time 736447786306 state 0x1 ref 1 00000100:00100000:0.0:1789760793.222674:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736448046722 time 736468292726 state 0x0 ref 1 00000100:00100000:0.0:1789760793.222686:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554874880:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760793.222732:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67440640, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x202a0 00010000:00010000:0.0:1789760793.222742:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6c6d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.222756:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760793.222760:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.222767:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6c6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67440640->67444735] (req 67440640->67444735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.222782:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6c6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67440640->67444735] (req 67440640->67444735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.230156:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983774 is committed 00002000:00010000:0.0:1789760793.233887:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67440640 -> 67444736 00002000:00010000:0.0:1789760793.233896:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131744 -> 131752 00010000:00010000:0.0:1789760793.233910:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6c6d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67440640->67444735] (req 67440640->67444735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.233924:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6c6d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67440640->67444735] (req 67440640->67444735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.233934:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6c6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67440640->67444735] (req 67440640->67444735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.233940:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760793.233946:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6c6d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67440640->67444735] (req 67440640->67444735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.233955:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6c6d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67440640->67444735] (req 67440640->67444735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.233975:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6c6d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67440640->67444735] (req 67440640->67444735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.233982:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6c6d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67440640->67444735] (req 67440640->67444735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.236538:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554874880:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13840us (14116us total) trans 4294983774 rc 0/0 00000100:00100000:0.0:1789760793.236554:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16476 00000100:00100000:0.0:1789760793.236564:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736448046722 time 736468292726 state 0x2 ref 0 00000100:00100000:0.0:1789760793.242572:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760793.243254:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554875008 00000100:00100000:2.0:1789760793.243265:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554875008 00000100:00100000:2.0:1789760793.243323:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736448046722 time 736468292726 state 0x0 ref 1 00000100:00100000:2.0:1789760793.243338:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736468553142 time 736468292726 state 0x1 ref 1 00000100:00100000:1.0:1789760793.248724:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736468553142 time 736491288328 state 0x0 ref 1 00000100:00100000:1.0:1789760793.248768:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554875008:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760793.248831:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67444736, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x202a8 00010000:00010000:1.0:1789760793.248841:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c74 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.248863:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760793.248868:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.248874:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67444736->67448831] (req 67444736->67448831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.248888:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67444736->67448831] (req 67444736->67448831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.255951:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983775 is committed 00002000:00010000:1.0:1789760793.256866:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67444736 -> 67448832 00002000:00010000:1.0:1789760793.256874:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131752 -> 131760 00010000:00010000:1.0:1789760793.256893:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c74 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67444736->67448831] (req 67444736->67448831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.256907:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c74 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67444736->67448831] (req 67444736->67448831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.256918:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c74 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67444736->67448831] (req 67444736->67448831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.256926:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.256930:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c74 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67444736->67448831] (req 67444736->67448831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.256939:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c74 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67444736->67448831] (req 67444736->67448831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.256975:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c74 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67444736->67448831] (req 67444736->67448831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.256983:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c74 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67444736->67448831] (req 67444736->67448831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.257090:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554875008:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8327us (14516us total) trans 4294983775 rc 0/0 00000100:00100000:1.0:1789760793.257102:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16477 00000100:00100000:1.0:1789760793.257112:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736468553142 time 736491288328 state 0x2 ref 0 00000100:00100000:2.0:1789760793.264094:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760793.264219:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554875136 00000100:00100000:1.0:1789760793.264229:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554875136 00000100:00100000:1.0:1789760793.264283:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736468553142 time 736491288328 state 0x0 ref 1 00000100:00100000:1.0:1789760793.264298:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736491548744 time 736491288328 state 0x1 ref 1 00000100:00100000:1.0:1789760793.264425:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736491548744 time 736510042011 state 0x0 ref 1 00000100:00100000:1.0:1789760793.264448:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06449880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554875136:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760793.264505:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67448832, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x202b0 00010000:00010000:1.0:1789760793.264514:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c7b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.264530:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760793.264535:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.264542:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67448832->67452927] (req 67448832->67452927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.264560:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67448832->67452927] (req 67448832->67452927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.271166:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983776 is committed 00002000:00010000:1.0:1789760793.271279:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67448832 -> 67452928 00002000:00010000:1.0:1789760793.271285:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131760 -> 131768 00010000:00010000:1.0:1789760793.271301:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c7b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67448832->67452927] (req 67448832->67452927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.271316:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c7b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67448832->67452927] (req 67448832->67452927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.271326:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67448832->67452927] (req 67448832->67452927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.271334:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.271339:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c7b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67448832->67452927] (req 67448832->67452927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.271349:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c7b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67448832->67452927] (req 67448832->67452927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.271370:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c7b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67448832->67452927] (req 67448832->67452927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.271405:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c7b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67448832->67452927] (req 67448832->67452927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.271519:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06449880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554875136:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7069us (7429us total) trans 4294983776 rc 0/0 00000100:00100000:1.0:1789760793.271531:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16478 00000100:00100000:1.0:1789760793.271541:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736491548744 time 736510042011 state 0x2 ref 0 00000100:00100000:2.0:1789760793.275993:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760793.276076:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554875264 00000100:00100000:3.0:1789760793.276085:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554875264 00000100:00100000:3.0:1789760793.276134:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736491548744 time 736510042011 state 0x0 ref 1 00000100:00100000:3.0:1789760793.276830:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736510302427 time 736510042011 state 0x1 ref 1 00000100:00100000:3.0:1789760793.276902:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736510302427 time 736522519864 state 0x0 ref 1 00000100:00100000:3.0:1789760793.276920:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06448e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554875264:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760793.276970:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67452928, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x202b8 00010000:00010000:3.0:1789760793.276980:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6c82 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.276996:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760793.277000:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.277008:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6c82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67452928->67457023] (req 67452928->67457023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.277021:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6c82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67452928->67457023] (req 67452928->67457023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.284871:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983777 is committed 00002000:00010000:3.0:1789760793.285000:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67452928 -> 67457024 00002000:00010000:3.0:1789760793.285005:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131768 -> 131776 00010000:00010000:3.0:1789760793.285018:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6c82 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67452928->67457023] (req 67452928->67457023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.285031:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6c82 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67452928->67457023] (req 67452928->67457023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.285039:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6c82 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67452928->67457023] (req 67452928->67457023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.285046:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760793.285051:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6c82 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67452928->67457023] (req 67452928->67457023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.285059:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6c82 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67452928->67457023] (req 67452928->67457023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.285083:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6c82 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67452928->67457023] (req 67452928->67457023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.285096:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6c82 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67452928->67457023] (req 67452928->67457023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.285185:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06448e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554875264:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8267us (9199us total) trans 4294983777 rc 0/0 00000100:00100000:3.0:1789760793.285196:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16479 00000100:00100000:3.0:1789760793.285205:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736510302427 time 736522519864 state 0x2 ref 0 00000100:00100000:2.0:1789760793.292802:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760793.296798:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554875392 00000100:00100000:0.0:1789760793.296814:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554875392 00000100:00100000:0.0:1789760793.296877:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736510302427 time 736522519864 state 0x0 ref 1 00000100:00100000:0.0:1789760793.296893:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736522780280 time 736522519864 state 0x1 ref 1 00000100:00100000:0.0:1789760793.296976:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736522780280 time 736542593499 state 0x0 ref 1 00000100:00100000:0.0:1789760793.296989:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06449500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554875392:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760793.297036:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67457024, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x202c0 00010000:00010000:0.0:1789760793.297047:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6c89 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.297063:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760793.297067:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.297074:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6c89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67457024->67461119] (req 67457024->67461119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.297087:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6c89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67457024->67461119] (req 67457024->67461119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.302481:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983778 is committed 00002000:00010000:1.0:1789760793.302608:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67457024 -> 67461120 00002000:00010000:1.0:1789760793.302613:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131776 -> 131784 00010000:00010000:1.0:1789760793.302627:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6c89 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67457024->67461119] (req 67457024->67461119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.302640:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6c89 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67457024->67461119] (req 67457024->67461119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.302649:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6c89 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67457024->67461119] (req 67457024->67461119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.302657:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.302662:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6c89 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67457024->67461119] (req 67457024->67461119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.302672:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6c89 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67457024->67461119] (req 67457024->67461119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.302698:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6c89 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67457024->67461119] (req 67457024->67461119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.302844:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6c89 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67457024->67461119] (req 67457024->67461119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.302932:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06449500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554875392:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5940us (10136us total) trans 4294983778 rc 0/0 00000100:00100000:1.0:1789760793.302942:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16480 00000100:00100000:1.0:1789760793.302951:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736522780280 time 736542593499 state 0x2 ref 0 00000100:00100000:2.0:1789760793.307150:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760793.308072:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554875520 00000100:00100000:1.0:1789760793.308081:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554875520 00000100:00100000:1.0:1789760793.308125:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736522780280 time 736542593499 state 0x0 ref 1 00000100:00100000:1.0:1789760793.308135:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736542853915 time 736542593499 state 0x1 ref 1 00000100:00100000:1.0:1789760793.308208:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736542853915 time 736553775311 state 0x0 ref 1 00000100:00100000:1.0:1789760793.308220:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0644b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554875520:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760793.308338:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67461120, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x202c8 00010000:00010000:2.0:1789760793.308348:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6c90 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.308361:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760793.308365:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.308372:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6c90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67461120->67465215] (req 67461120->67465215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.308446:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6c90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67461120->67465215] (req 67461120->67465215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.315289:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983779 is committed 00002000:00010000:2.0:1789760793.316294:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67461120 -> 67465216 00002000:00010000:2.0:1789760793.316300:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131784 -> 131792 00010000:00010000:2.0:1789760793.316313:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6c90 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67461120->67465215] (req 67461120->67465215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.316325:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6c90 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67461120->67465215] (req 67461120->67465215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.316334:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6c90 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67461120->67465215] (req 67461120->67465215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.316341:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760793.316346:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6c90 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67461120->67465215] (req 67461120->67465215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.316359:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6c90 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67461120->67465215] (req 67461120->67465215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.316419:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6c90 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67461120->67465215] (req 67461120->67465215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.316427:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6c90 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67461120->67465215] (req 67461120->67465215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.316515:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0644b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554875520:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8292us (9369us total) trans 4294983779 rc 0/0 00000100:00100000:2.0:1789760793.316526:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16481 00000100:00100000:2.0:1789760793.316534:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736542853915 time 736553775311 state 0x2 ref 0 00000100:00100000:2.0:1789760793.320243:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760793.320343:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554875648 00000100:00100000:1.0:1789760793.320354:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554875648 00000100:00100000:1.0:1789760793.320814:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736542853915 time 736553775311 state 0x0 ref 1 00000100:00100000:1.0:1789760793.320831:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736554035727 time 736553775311 state 0x1 ref 1 00000100:00100000:1.0:1789760793.320937:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736554035727 time 736566554202 state 0x0 ref 1 00000100:00100000:1.0:1789760793.320953:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1fe1ac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554875648:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760793.321012:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67465216, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x202d0 00010000:00010000:1.0:1789760793.321021:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c97 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.321039:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760793.321044:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.321050:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67465216->67469311] (req 67465216->67469311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.321063:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67465216->67469311] (req 67465216->67469311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.325694:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983780 is committed 00002000:00010000:1.0:1789760793.325820:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67465216 -> 67469312 00002000:00010000:1.0:1789760793.325826:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131792 -> 131800 00010000:00010000:1.0:1789760793.325839:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c97 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67465216->67469311] (req 67465216->67469311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.325853:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c97 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67465216->67469311] (req 67465216->67469311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.325862:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c97 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67465216->67469311] (req 67465216->67469311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.325875:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.325880:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c97 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67465216->67469311] (req 67465216->67469311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.325889:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c97 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67465216->67469311] (req 67465216->67469311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.325908:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c97 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67465216->67469311] (req 67465216->67469311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.325917:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6c97 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67465216->67469311] (req 67465216->67469311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.326009:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1fe1ac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554875648:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5056us (5772us total) trans 4294983780 rc 0/0 00000100:00100000:1.0:1789760793.326019:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16482 00000100:00100000:1.0:1789760793.326029:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736554035727 time 736566554202 state 0x2 ref 0 00000100:00100000:0.0:1789760793.330724:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760793.330960:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554875776 00000100:00100000:3.0:1789760793.330970:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554875776 00000100:00100000:3.0:1789760793.331091:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736554035727 time 736566554202 state 0x0 ref 1 00000100:00100000:3.0:1789760793.331105:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736566814618 time 736566554202 state 0x1 ref 1 00000100:00100000:3.0:1789760793.331137:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736566814618 time 736576755457 state 0x0 ref 1 00000100:00100000:3.0:1789760793.331149:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554875776:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760793.331195:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67469312, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x202d8 00010000:00010000:3.0:1789760793.331204:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6c9e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.331219:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760793.331223:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.331229:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6c9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67469312->67473407] (req 67469312->67473407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.331247:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6c9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67469312->67473407] (req 67469312->67473407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.339582:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983781 is committed 00002000:00010000:3.0:1789760793.339706:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67469312 -> 67473408 00002000:00010000:3.0:1789760793.339711:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131800 -> 131808 00010000:00010000:3.0:1789760793.339724:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6c9e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67469312->67473407] (req 67469312->67473407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.339736:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6c9e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67469312->67473407] (req 67469312->67473407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.339746:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6c9e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67469312->67473407] (req 67469312->67473407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.339753:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760793.339758:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6c9e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67469312->67473407] (req 67469312->67473407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.339766:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6c9e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67469312->67473407] (req 67469312->67473407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.339785:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6c9e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67469312->67473407] (req 67469312->67473407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.339793:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6c9e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67469312->67473407] (req 67469312->67473407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.339885:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554875776:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8733us (9166us total) trans 4294983781 rc 0/0 00000100:00100000:3.0:1789760793.339900:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16483 00000100:00100000:3.0:1789760793.339910:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736566814618 time 736576755457 state 0x2 ref 0 00000100:00100000:2.0:1789760793.347264:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760793.347360:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554875904 00000100:00100000:0.0:1789760793.347371:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554875904 00000100:00100000:0.0:1789760793.347466:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736566814618 time 736576755457 state 0x0 ref 1 00000100:00100000:0.0:1789760793.347481:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736577015873 time 736576755457 state 0x1 ref 1 00000100:00100000:0.0:1789760793.347501:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736577015873 time 736593120206 state 0x0 ref 1 00000100:00100000:0.0:1789760793.347513:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06448a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554875904:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760793.347556:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67473408, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x202e0 00010000:00010000:0.0:1789760793.347566:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6ca5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.347582:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760793.347587:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.347593:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6ca5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67473408->67477503] (req 67473408->67477503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.347619:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6ca5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67473408->67477503] (req 67473408->67477503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.355093:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983782 is committed 00002000:00010000:0.0:1789760793.355239:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67473408 -> 67477504 00002000:00010000:0.0:1789760793.355244:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131808 -> 131816 00010000:00010000:0.0:1789760793.355258:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6ca5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67473408->67477503] (req 67473408->67477503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.355273:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6ca5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67473408->67477503] (req 67473408->67477503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.355282:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6ca5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67473408->67477503] (req 67473408->67477503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.355289:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760793.355294:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6ca5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67473408->67477503] (req 67473408->67477503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.355302:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6ca5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67473408->67477503] (req 67473408->67477503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.355333:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6ca5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67473408->67477503] (req 67473408->67477503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.355341:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6ca5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67473408->67477503] (req 67473408->67477503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.355479:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06448a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554875904:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7963us (8220us total) trans 4294983782 rc 0/0 00000100:00100000:0.0:1789760793.355489:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16484 00000100:00100000:0.0:1789760793.355498:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736577015873 time 736593120206 state 0x2 ref 0 00000100:00100000:2.0:1789760793.361883:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760793.362283:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554876032 00000100:00100000:1.0:1789760793.362294:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554876032 00000100:00100000:1.0:1789760793.362353:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736577015873 time 736593120206 state 0x0 ref 1 00000100:00100000:1.0:1789760793.362368:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736593380622 time 736593120206 state 0x1 ref 1 00000100:00100000:1.0:1789760793.362541:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736593380622 time 736608158016 state 0x0 ref 1 00000100:00100000:1.0:1789760793.362556:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06449f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554876032:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760793.362631:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67477504, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x202e8 00010000:00010000:1.0:1789760793.362647:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.362671:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760793.362676:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.362682:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67477504->67481599] (req 67477504->67481599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.362696:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67477504->67481599] (req 67477504->67481599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.371021:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983783 is committed 00002000:00010000:1.0:1789760793.371141:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67477504 -> 67481600 00002000:00010000:1.0:1789760793.371147:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131816 -> 131824 00010000:00010000:1.0:1789760793.371160:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67477504->67481599] (req 67477504->67481599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.371173:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67477504->67481599] (req 67477504->67481599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.371181:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67477504->67481599] (req 67477504->67481599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.371195:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.371199:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67477504->67481599] (req 67477504->67481599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.371209:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67477504->67481599] (req 67477504->67481599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.371227:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67477504->67481599] (req 67477504->67481599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.371236:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67477504->67481599] (req 67477504->67481599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.371330:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06449f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554876032:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8772us (9453us total) trans 4294983783 rc 0/0 00000100:00100000:1.0:1789760793.371341:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16485 00000100:00100000:1.0:1789760793.371352:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736593380622 time 736608158016 state 0x2 ref 0 00000100:00100000:2.0:1789760793.379658:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760793.379764:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554876160 00000100:00100000:1.0:1789760793.379773:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554876160 00000100:00100000:1.0:1789760793.379834:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736593380622 time 736608158016 state 0x0 ref 1 00000100:00100000:1.0:1789760793.379849:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736608418432 time 736608158016 state 0x1 ref 1 00000100:00100000:1.0:1789760793.379885:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736608418432 time 736625503027 state 0x0 ref 1 00000100:00100000:1.0:1789760793.379903:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0644b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554876160:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760793.379959:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67481600, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x202f0 00010000:00010000:1.0:1789760793.379968:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cb3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.379994:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760793.379999:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.380010:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cb3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67481600->67485695] (req 67481600->67485695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.380026:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cb3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67481600->67485695] (req 67481600->67485695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.388663:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983784 is committed 00002000:00010000:1.0:1789760793.388836:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67481600 -> 67485696 00002000:00010000:1.0:1789760793.388841:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131824 -> 131832 00010000:00010000:1.0:1789760793.388855:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cb3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67481600->67485695] (req 67481600->67485695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.388868:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cb3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67481600->67485695] (req 67481600->67485695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.388878:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cb3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67481600->67485695] (req 67481600->67485695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.388886:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.388891:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cb3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67481600->67485695] (req 67481600->67485695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.388900:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cb3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67481600->67485695] (req 67481600->67485695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.388918:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cb3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67481600->67485695] (req 67481600->67485695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.388926:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cb3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67481600->67485695] (req 67481600->67485695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.389027:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0644b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554876160:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9123us (9375us total) trans 4294983784 rc 0/0 00000100:00100000:1.0:1789760793.389038:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16486 00000100:00100000:1.0:1789760793.389047:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736608418432 time 736625503027 state 0x2 ref 0 00000100:00100000:2.0:1789760793.401800:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760793.401885:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554876288 00000100:00100000:3.0:1789760793.401894:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554876288 00000100:00100000:3.0:1789760793.404143:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736608418432 time 736625503027 state 0x0 ref 1 00000100:00100000:3.0:1789760793.404162:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736625763443 time 736625503027 state 0x1 ref 1 00000100:00100000:3.0:1789760793.404205:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736625763443 time 736649823181 state 0x0 ref 1 00000100:00100000:3.0:1789760793.404276:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06448380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554876288:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760793.404323:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67485696, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x202f8 00010000:00010000:3.0:1789760793.404332:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6cba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.404346:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760793.404351:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.404358:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6cba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67485696->67489791] (req 67485696->67489791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.404372:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6cba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67485696->67489791] (req 67485696->67489791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.411081:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983785 is committed 00002000:00010000:3.0:1789760793.411307:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67485696 -> 67489792 00002000:00010000:3.0:1789760793.411312:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131832 -> 131840 00010000:00010000:3.0:1789760793.411325:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6cba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67485696->67489791] (req 67485696->67489791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.411337:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6cba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67485696->67489791] (req 67485696->67489791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.411346:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6cba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67485696->67489791] (req 67485696->67489791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.411353:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760793.411358:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6cba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67485696->67489791] (req 67485696->67489791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.411367:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6cba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67485696->67489791] (req 67485696->67489791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.412097:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6cba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67485696->67489791] (req 67485696->67489791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.412112:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6cba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67485696->67489791] (req 67485696->67489791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.412210:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06448380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554876288:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7938us (10414us total) trans 4294983785 rc 0/0 00000100:00100000:3.0:1789760793.412222:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16487 00000100:00100000:3.0:1789760793.412231:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736625763443 time 736649823181 state 0x2 ref 0 00000100:00100000:2.0:1789760793.417263:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760793.419902:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554876416 00000100:00100000:0.0:1789760793.419917:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554876416 00000100:00100000:0.0:1789760793.420037:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736625763443 time 736649823181 state 0x0 ref 1 00000100:00100000:0.0:1789760793.420052:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736650083597 time 736649823181 state 0x1 ref 1 00000100:00100000:0.0:1789760793.420188:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736650083597 time 736665805028 state 0x0 ref 1 00000100:00100000:0.0:1789760793.420202:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0644a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554876416:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760793.420248:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67489792, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20300 00010000:00010000:0.0:1789760793.420257:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6cc1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.420272:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760793.420277:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.420284:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6cc1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67489792->67493887] (req 67489792->67493887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.420298:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6cc1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67489792->67493887] (req 67489792->67493887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.429475:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983786 is committed 00002000:00010000:0.0:1789760793.430840:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67489792 -> 67493888 00002000:00010000:0.0:1789760793.430848:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131840 -> 131848 00010000:00010000:0.0:1789760793.430862:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6cc1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67489792->67493887] (req 67489792->67493887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.430876:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6cc1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67489792->67493887] (req 67489792->67493887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.430890:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6cc1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67489792->67493887] (req 67489792->67493887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.430898:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760793.430904:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6cc1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67489792->67493887] (req 67489792->67493887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.430913:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6cc1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67489792->67493887] (req 67489792->67493887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.430932:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6cc1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67489792->67493887] (req 67489792->67493887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.430941:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6cc1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67489792->67493887] (req 67489792->67493887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.431171:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0644a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554876416:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10966us (13912us total) trans 4294983786 rc 0/0 00000100:00100000:0.0:1789760793.431181:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16488 00000100:00100000:0.0:1789760793.431190:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736650083597 time 736665805028 state 0x2 ref 0 00000100:00100000:2.0:1789760793.434846:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760793.435155:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554876544 00000100:00100000:1.0:1789760793.435164:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554876544 00000100:00100000:1.0:1789760793.435221:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736650083597 time 736665805028 state 0x0 ref 1 00000100:00100000:1.0:1789760793.435238:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736666065444 time 736665805028 state 0x1 ref 1 00000100:00100000:1.0:1789760793.435270:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736666065444 time 736680888875 state 0x0 ref 1 00000100:00100000:1.0:1789760793.435283:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06449c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554876544:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760793.435329:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67493888, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20308 00010000:00010000:1.0:1789760793.435344:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cc8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.435360:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760793.435364:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.435371:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cc8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67493888->67497983] (req 67493888->67497983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.435423:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cc8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67493888->67497983] (req 67493888->67497983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.441853:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983787 is committed 00002000:00010000:1.0:1789760793.442479:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67493888 -> 67497984 00002000:00010000:1.0:1789760793.442486:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131848 -> 131856 00010000:00010000:1.0:1789760793.442499:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cc8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67493888->67497983] (req 67493888->67497983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.442513:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cc8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67493888->67497983] (req 67493888->67497983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.442523:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cc8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67493888->67497983] (req 67493888->67497983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.442531:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.442536:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cc8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67493888->67497983] (req 67493888->67497983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.442551:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cc8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67493888->67497983] (req 67493888->67497983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.442713:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cc8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67493888->67497983] (req 67493888->67497983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.442724:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cc8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67493888->67497983] (req 67493888->67497983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.442856:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06449c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554876544:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7571us (8015us total) trans 4294983787 rc 0/0 00000100:00100000:1.0:1789760793.442867:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16489 00000100:00100000:1.0:1789760793.442876:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736666065444 time 736680888875 state 0x2 ref 0 00000100:00100000:2.0:1789760793.446818:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760793.446867:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554876672 00000100:00100000:1.0:1789760793.446877:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554876672 00000100:00100000:1.0:1789760793.446931:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736666065444 time 736680888875 state 0x0 ref 1 00000100:00100000:1.0:1789760793.446947:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736681149291 time 736680888875 state 0x1 ref 1 00000100:00100000:1.0:1789760793.447088:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736681149291 time 736692705392 state 0x0 ref 1 00000100:00100000:1.0:1789760793.447102:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06449880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554876672:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760793.447153:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67497984, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20310 00010000:00010000:1.0:1789760793.447162:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6ccf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.447181:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760793.447186:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.447193:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6ccf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67497984->67502079] (req 67497984->67502079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.447208:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6ccf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67497984->67502079] (req 67497984->67502079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.460149:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983788 is committed 00002000:00010000:2.0:1789760793.460346:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67497984 -> 67502080 00002000:00010000:2.0:1789760793.460352:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131856 -> 131864 00010000:00010000:2.0:1789760793.460366:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6ccf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67497984->67502079] (req 67497984->67502079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.460408:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6ccf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67497984->67502079] (req 67497984->67502079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.460418:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6ccf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67497984->67502079] (req 67497984->67502079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.460426:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760793.460431:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6ccf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67497984->67502079] (req 67497984->67502079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.460440:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6ccf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67497984->67502079] (req 67497984->67502079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.460460:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6ccf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67497984->67502079] (req 67497984->67502079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.460468:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6ccf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67497984->67502079] (req 67497984->67502079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.460558:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06449880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554876672:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13454us (13744us total) trans 4294983788 rc 0/0 00000100:00100000:2.0:1789760793.460570:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16490 00000100:00100000:2.0:1789760793.460579:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736681149291 time 736692705392 state 0x2 ref 0 00000100:00100000:2.0:1789760793.470850:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760793.471111:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554876800 00000100:00100000:3.0:1789760793.471120:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554876800 00000100:00100000:3.0:1789760793.471188:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736681149291 time 736692705392 state 0x0 ref 1 00000100:00100000:3.0:1789760793.471203:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736692965808 time 736692705392 state 0x1 ref 1 00000100:00100000:3.0:1789760793.471236:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736692965808 time 736716854272 state 0x0 ref 1 00000100:00100000:3.0:1789760793.471248:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06448e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554876800:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760793.471291:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67502080, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20318 00010000:00010000:3.0:1789760793.471299:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6cd6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.471314:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789760793.471318:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760793.471324:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6cd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67502080->67506175] (req 67502080->67506175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.471337:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6cd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67502080->67506175] (req 67502080->67506175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.478665:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983789 is committed 00002000:00010000:3.0:1789760793.479280:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67502080 -> 67506176 00002000:00010000:3.0:1789760793.479287:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131864 -> 131872 00010000:00010000:3.0:1789760793.479301:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6cd6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67502080->67506175] (req 67502080->67506175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.479314:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6cd6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67502080->67506175] (req 67502080->67506175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.479324:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6cd6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67502080->67506175] (req 67502080->67506175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.479331:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760793.479336:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6cd6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67502080->67506175] (req 67502080->67506175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.479344:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6cd6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67502080->67506175] (req 67502080->67506175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.479452:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6cd6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67502080->67506175] (req 67502080->67506175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760793.479461:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6cd6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67502080->67506175] (req 67502080->67506175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760793.479621:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06448e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554876800:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8369us (8775us total) trans 4294983789 rc 0/0 00000100:00100000:3.0:1789760793.479630:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16491 00000100:00100000:3.0:1789760793.479639:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736692965808 time 736716854272 state 0x2 ref 0 00000100:00100000:2.0:1789760793.492514:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760793.492608:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554876928 00000100:00100000:1.0:1789760793.492619:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554876928 00000100:00100000:1.0:1789760793.492673:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736692965808 time 736716854272 state 0x0 ref 1 00000100:00100000:1.0:1789760793.492689:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736717114688 time 736716854272 state 0x1 ref 1 00000100:00100000:1.0:1789760793.492720:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736717114688 time 736738338276 state 0x0 ref 1 00000100:00100000:1.0:1789760793.492733:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06449500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554876928:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760793.492779:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67506176, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20320 00010000:00010000:1.0:1789760793.492789:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cdd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.492805:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760793.492810:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.492817:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cdd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67506176->67510271] (req 67506176->67510271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.492830:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cdd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67506176->67510271] (req 67506176->67510271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760793.507342:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983790 is committed 00002000:00010000:0.0:1789760793.507557:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67506176 -> 67510272 00002000:00010000:0.0:1789760793.507562:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131872 -> 131880 00010000:00010000:0.0:1789760793.507575:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cdd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67506176->67510271] (req 67506176->67510271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.507609:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cdd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67506176->67510271] (req 67506176->67510271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.507620:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cdd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67506176->67510271] (req 67506176->67510271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.507627:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760793.507632:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cdd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67506176->67510271] (req 67506176->67510271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.507641:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cdd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67506176->67510271] (req 67506176->67510271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.507659:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cdd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67506176->67510271] (req 67506176->67510271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.507667:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c6cdd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67506176->67510271] (req 67506176->67510271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.507906:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06449500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554876928:12345-192.168.203.22@tcp:4:dd.0 Request processed in 15169us (15394us total) trans 4294983790 rc 0/0 00000100:00100000:0.0:1789760793.507917:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16492 00000100:00100000:0.0:1789760793.507927:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736717114688 time 736738338276 state 0x2 ref 0 00000100:00100000:2.0:1789760793.512547:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760793.512783:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554877056 00000100:00100000:1.0:1789760793.512794:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554877056 00000100:00100000:1.0:1789760793.512858:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736717114688 time 736738338276 state 0x0 ref 1 00000100:00100000:1.0:1789760793.512875:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736738598692 time 736738338276 state 0x1 ref 1 00000100:00100000:1.0:1789760793.512905:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736738598692 time 736758523229 state 0x0 ref 1 00000100:00100000:1.0:1789760793.512917:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0644b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554877056:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760793.512963:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67510272, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20328 00010000:00010000:1.0:1789760793.512982:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ce4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.512999:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760793.513005:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.513012:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ce4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67510272->67514367] (req 67510272->67514367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.513028:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ce4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67510272->67514367] (req 67510272->67514367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.519518:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983791 is committed 00002000:00010000:1.0:1789760793.522252:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67510272 -> 67514368 00002000:00010000:1.0:1789760793.522262:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131880 -> 131888 00010000:00010000:1.0:1789760793.522275:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ce4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67510272->67514367] (req 67510272->67514367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.522287:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ce4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67510272->67514367] (req 67510272->67514367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.522294:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ce4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67510272->67514367] (req 67510272->67514367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.522301:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655de00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.522306:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ce4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67510272->67514367] (req 67510272->67514367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.522468:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ce4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67510272->67514367] (req 67510272->67514367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.522492:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ce4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67510272->67514367] (req 67510272->67514367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.522500:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ce4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67510272->67514367] (req 67510272->67514367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.522603:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0644b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554877056:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9675us (10055us total) trans 4294983791 rc 0/0 00000100:00100000:1.0:1789760793.522615:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16493 00000100:00100000:1.0:1789760793.522623:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736738598692 time 736758523229 state 0x2 ref 0 00000100:00100000:2.0:1789760793.535345:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760793.540185:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554877184 00000100:00100000:1.0:1789760793.540200:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554877184 00000100:00100000:1.0:1789760793.540261:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736738598692 time 736758523229 state 0x0 ref 1 00000100:00100000:1.0:1789760793.540276:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736758783645 time 736758523229 state 0x1 ref 1 00000100:00100000:1.0:1789760793.540304:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736758783645 time 736785923556 state 0x0 ref 1 00000100:00100000:1.0:1789760793.540317:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554877184:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760793.540362:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67514368, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20330 00010000:00010000:1.0:1789760793.540370:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ceb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.540414:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760793.540419:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.540425:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ceb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67514368->67518463] (req 67514368->67518463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.540439:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ceb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67514368->67518463] (req 67514368->67518463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.546824:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983792 is committed 00002000:00010000:1.0:1789760793.547058:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67514368 -> 67518464 00002000:00010000:1.0:1789760793.547064:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131888 -> 131896 00010000:00010000:1.0:1789760793.547078:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ceb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67514368->67518463] (req 67514368->67518463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.547092:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ceb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67514368->67518463] (req 67514368->67518463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.547101:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ceb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67514368->67518463] (req 67514368->67518463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.547108:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655de00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.547112:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ceb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67514368->67518463] (req 67514368->67518463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.547120:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ceb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67514368->67518463] (req 67514368->67518463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.547140:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ceb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67514368->67518463] (req 67514368->67518463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.547220:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ceb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67514368->67518463] (req 67514368->67518463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.547358:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554877184:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7031us (12012us total) trans 4294983792 rc 0/0 00000100:00100000:1.0:1789760793.547369:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16494 00000100:00100000:1.0:1789760793.547433:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736758783645 time 736785923556 state 0x2 ref 0 00000100:00100000:2.0:1789760793.550828:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760793.551301:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554877312 00000100:00100000:3.0:1789760793.551313:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554877312 00000100:00100000:3.0:1789760793.551829:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736758783645 time 736785923556 state 0x0 ref 1 00000100:00100000:3.0:1789760793.551846:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736786183972 time 736785923556 state 0x1 ref 1 00000100:00100000:2.0:1789760793.551988:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736786183972 time 736797604189 state 0x0 ref 1 00000100:00100000:2.0:1789760793.552005:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554877312:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760793.552052:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67518464, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20338 00010000:00010000:2.0:1789760793.552061:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6cf2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.552076:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760793.552080:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.552086:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6cf2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67518464->67522559] (req 67518464->67522559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.552100:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6cf2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67518464->67522559] (req 67518464->67522559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.558941:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983793 is committed 00002000:00010000:2.0:1789760793.559112:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67518464 -> 67522560 00002000:00010000:2.0:1789760793.559117:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131896 -> 131904 00010000:00010000:2.0:1789760793.559129:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6cf2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67518464->67522559] (req 67518464->67522559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.559146:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6cf2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67518464->67522559] (req 67518464->67522559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.559155:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6cf2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67518464->67522559] (req 67518464->67522559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.559162:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760793.559167:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6cf2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67518464->67522559] (req 67518464->67522559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.559176:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6cf2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67518464->67522559] (req 67518464->67522559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.559196:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6cf2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67518464->67522559] (req 67518464->67522559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.559204:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6cf2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67518464->67522559] (req 67518464->67522559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.559300:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554877312:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7296us (8475us total) trans 4294983793 rc 0/0 00000100:00100000:2.0:1789760793.559311:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16495 00000100:00100000:2.0:1789760793.559320:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736786183972 time 736797604189 state 0x2 ref 0 00000100:00100000:2.0:1789760793.565045:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760793.565359:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554877440 00000100:00100000:0.0:1789760793.565370:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554877440 00000100:00100000:0.0:1789760793.565473:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736786183972 time 736797604189 state 0x0 ref 1 00000100:00100000:0.0:1789760793.565489:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736797864605 time 736797604189 state 0x1 ref 1 00000100:00100000:0.0:1789760793.565515:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736797864605 time 736811134184 state 0x0 ref 1 00000100:00100000:0.0:1789760793.565529:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554877440:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760793.565855:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67522560, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20340 00010000:00010000:0.0:1789760793.565870:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6cf9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.565887:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760793.565892:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.565900:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6cf9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67522560->67526655] (req 67522560->67526655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.565915:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6cf9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67522560->67526655] (req 67522560->67526655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.571314:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983794 is committed 00002000:00010000:0.0:1789760793.571552:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67522560 -> 67526656 00002000:00010000:0.0:1789760793.571558:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131904 -> 131912 00010000:00010000:0.0:1789760793.571571:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6cf9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67522560->67526655] (req 67522560->67526655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.571646:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6cf9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67522560->67526655] (req 67522560->67526655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.571657:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6cf9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67522560->67526655] (req 67522560->67526655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.571664:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760793.571669:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6cf9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67522560->67526655] (req 67522560->67526655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.571677:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6cf9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67522560->67526655] (req 67522560->67526655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.571696:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6cf9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67522560->67526655] (req 67522560->67526655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.571775:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6cf9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67522560->67526655] (req 67522560->67526655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.571917:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554877440:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6386us (6876us total) trans 4294983794 rc 0/0 00000100:00100000:0.0:1789760793.571928:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16496 00000100:00100000:0.0:1789760793.571936:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736797864605 time 736811134184 state 0x2 ref 0 00000100:00100000:0.0:1789760793.575083:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760793.575162:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554877568 00000100:00100000:1.0:1789760793.575171:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554877568 00000100:00100000:1.0:1789760793.575221:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736797864605 time 736811134184 state 0x0 ref 1 00000100:00100000:1.0:1789760793.575237:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736811394600 time 736811134184 state 0x1 ref 1 00000100:00100000:1.0:1789760793.575315:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736811394600 time 736820932178 state 0x0 ref 1 00000100:00100000:1.0:1789760793.575328:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554877568:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760793.575374:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67526656, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20348 00010000:00010000:1.0:1789760793.575416:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6d00 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.575440:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760793.575445:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.575451:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6d00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67526656->67530751] (req 67526656->67530751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.575463:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6d00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67526656->67530751] (req 67526656->67530751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.580968:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983795 is committed 00002000:00010000:1.0:1789760793.581185:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67526656 -> 67530752 00002000:00010000:1.0:1789760793.581199:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131912 -> 131920 00010000:00010000:1.0:1789760793.581212:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6d00 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67526656->67530751] (req 67526656->67530751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.581231:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6d00 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67526656->67530751] (req 67526656->67530751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.581239:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6d00 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67526656->67530751] (req 67526656->67530751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.581245:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655de00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.581249:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6d00 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67526656->67530751] (req 67526656->67530751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.581258:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6d00 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67526656->67530751] (req 67526656->67530751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.581275:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6d00 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67526656->67530751] (req 67526656->67530751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.581282:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6d00 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67526656->67530751] (req 67526656->67530751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.581367:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554877568:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6037us (6286us total) trans 4294983795 rc 0/0 00000100:00100000:1.0:1789760793.581770:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16497 00000100:00100000:1.0:1789760793.581783:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736811394600 time 736820932178 state 0x2 ref 0 00000100:00100000:2.0:1789760793.590783:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760793.592105:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554877696 00000100:00100000:1.0:1789760793.592118:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554877696 00000100:00100000:1.0:1789760793.592179:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736811394600 time 736820932178 state 0x0 ref 1 00000100:00100000:1.0:1789760793.592194:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736821192594 time 736820932178 state 0x1 ref 1 00000100:00100000:1.0:1789760793.592236:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736821192594 time 736837854520 state 0x0 ref 1 00000100:00100000:1.0:1789760793.592250:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554877696:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760793.592299:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67530752, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20350 00010000:00010000:1.0:1789760793.592308:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6d07 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.592324:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760793.592328:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.592335:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6d07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67530752->67534847] (req 67530752->67534847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.592349:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6d07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67530752->67534847] (req 67530752->67534847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.598847:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983796 is committed 00002000:00010000:0.0:1789760793.600228:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67530752 -> 67534848 00002000:00010000:0.0:1789760793.600236:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131920 -> 131928 00010000:00010000:0.0:1789760793.600249:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6d07 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67530752->67534847] (req 67530752->67534847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.600261:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6d07 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67530752->67534847] (req 67530752->67534847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.600270:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6d07 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67530752->67534847] (req 67530752->67534847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.600278:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655de00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760793.600282:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6d07 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67530752->67534847] (req 67530752->67534847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.600291:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6d07 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67530752->67534847] (req 67530752->67534847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.600312:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6d07 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67530752->67534847] (req 67530752->67534847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.600320:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6d07 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67530752->67534847] (req 67530752->67534847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.600579:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554877696:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8324us (9799us total) trans 4294983796 rc 0/0 00000100:00100000:0.0:1789760793.600660:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16498 00000100:00100000:0.0:1789760793.600669:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736821192594 time 736837854520 state 0x2 ref 0 00000100:00100000:2.0:1789760793.606274:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760793.607503:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554877824 00000100:00100000:0.0:1789760793.607515:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554877824 00000100:00100000:0.0:1789760793.607575:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736821192594 time 736837854520 state 0x0 ref 1 00000100:00100000:0.0:1789760793.607590:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736838114936 time 736837854520 state 0x1 ref 1 00000100:00100000:0.0:1789760793.607670:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736838114936 time 736853287641 state 0x0 ref 1 00000100:00100000:0.0:1789760793.607692:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554877824:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760793.607750:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67534848, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20358 00010000:00010000:0.0:1789760793.607759:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d0e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.607775:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760793.607779:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.607785:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67534848->67538943] (req 67534848->67538943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.607799:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67534848->67538943] (req 67534848->67538943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.614284:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983797 is committed 00002000:00010000:0.0:1789760793.614566:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67534848 -> 67538944 00002000:00010000:0.0:1789760793.614582:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131928 -> 131936 00010000:00010000:0.0:1789760793.614603:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d0e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67534848->67538943] (req 67534848->67538943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.614615:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d0e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67534848->67538943] (req 67534848->67538943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.614629:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67534848->67538943] (req 67534848->67538943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.614637:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760793.614642:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d0e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67534848->67538943] (req 67534848->67538943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.614650:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d0e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67534848->67538943] (req 67534848->67538943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.614670:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d0e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67534848->67538943] (req 67534848->67538943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.614679:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d0e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67534848->67538943] (req 67534848->67538943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.614777:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554877824:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7091us (8506us total) trans 4294983797 rc 0/0 00000100:00100000:0.0:1789760793.614787:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16499 00000100:00100000:0.0:1789760793.614797:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736838114936 time 736853287641 state 0x2 ref 0 00000100:00100000:2.0:1789760793.627930:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760793.628349:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554877952 00000100:00100000:0.0:1789760793.628359:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554877952 00000100:00100000:0.0:1789760793.628447:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736838114936 time 736853287641 state 0x0 ref 1 00000100:00100000:0.0:1789760793.628464:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736853548057 time 736853287641 state 0x1 ref 1 00000100:00100000:0.0:1789760793.628499:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736853548057 time 736874117092 state 0x0 ref 1 00000100:00100000:0.0:1789760793.628512:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554877952:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760793.628559:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67538944, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20360 00010000:00010000:0.0:1789760793.628568:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d15 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.628584:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760793.628589:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.628606:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67538944->67543039] (req 67538944->67543039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.628621:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67538944->67543039] (req 67538944->67543039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.635814:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983798 is committed 00002000:00010000:0.0:1789760793.636320:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67538944 -> 67543040 00002000:00010000:0.0:1789760793.636327:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131936 -> 131944 00010000:00010000:0.0:1789760793.636339:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d15 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67538944->67543039] (req 67538944->67543039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.636352:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d15 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67538944->67543039] (req 67538944->67543039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.636361:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67538944->67543039] (req 67538944->67543039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.636374:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760793.636407:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d15 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67538944->67543039] (req 67538944->67543039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.636417:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d15 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67538944->67543039] (req 67538944->67543039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.636437:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d15 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67538944->67543039] (req 67538944->67543039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.636444:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d15 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67538944->67543039] (req 67538944->67543039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.636533:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554877952:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8018us (8608us total) trans 4294983798 rc 0/0 00000100:00100000:0.0:1789760793.636543:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16500 00000100:00100000:0.0:1789760793.636552:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736853548057 time 736874117092 state 0x2 ref 0 00000100:00100000:2.0:1789760793.641098:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760793.641656:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554878080 00000100:00100000:1.0:1789760793.641667:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554878080 00000100:00100000:1.0:1789760793.641802:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736853548057 time 736874117092 state 0x0 ref 1 00000100:00100000:1.0:1789760793.641819:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736874377508 time 736874117092 state 0x1 ref 1 00000100:00100000:1.0:1789760793.641853:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736874377508 time 736887470379 state 0x0 ref 1 00000100:00100000:1.0:1789760793.641870:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554878080:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760793.641929:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67543040, mtime: 0x6aad9518, atime: 0x0, ctime: 0x6aad9518, blocks: 0x20368 00010000:00010000:1.0:1789760793.641939:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6d1c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.641955:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760793.641960:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.641965:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6d1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67543040->67547135] (req 67543040->67547135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.642073:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6d1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67543040->67547135] (req 67543040->67547135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.647928:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983799 is committed 00002000:00010000:1.0:1789760793.648034:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67543040 -> 67547136 00002000:00010000:1.0:1789760793.648039:0:15101:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760792 -> 1789760793 00002000:00010000:1.0:1789760793.648041:0:15101:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760792 -> 1789760793 00002000:00010000:1.0:1789760793.648044:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131944 -> 131952 00010000:00010000:1.0:1789760793.648057:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6d1c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67543040->67547135] (req 67543040->67547135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.648069:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6d1c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67543040->67547135] (req 67543040->67547135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.648077:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6d1c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67543040->67547135] (req 67543040->67547135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.648089:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.648093:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6d1c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67543040->67547135] (req 67543040->67547135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.648102:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6d1c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67543040->67547135] (req 67543040->67547135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.648122:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6d1c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67543040->67547135] (req 67543040->67547135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.648130:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6d1c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67543040->67547135] (req 67543040->67547135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.648229:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554878080:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6357us (7136us total) trans 4294983799 rc 0/0 00000100:00100000:1.0:1789760793.648240:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16501 00000100:00100000:1.0:1789760793.648250:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736874377508 time 736887470379 state 0x2 ref 0 00000100:00100000:2.0:1789760793.659073:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760793.659191:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554878208 00000100:00100000:0.0:1789760793.659200:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554878208 00000100:00100000:0.0:1789760793.659255:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736874377508 time 736887470379 state 0x0 ref 1 00000100:00100000:0.0:1789760793.659271:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736887730795 time 736887470379 state 0x1 ref 1 00000100:00100000:0.0:1789760793.659363:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736887730795 time 736904971048 state 0x0 ref 1 00000100:00100000:0.0:1789760793.659508:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554878208:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760793.659566:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67547136, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20370 00010000:00010000:0.0:1789760793.659575:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d23 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.659591:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760793.659603:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.659610:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67547136->67551231] (req 67547136->67551231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.659624:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67547136->67551231] (req 67547136->67551231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.665041:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983800 is committed 00002000:00010000:0.0:1789760793.666243:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67547136 -> 67551232 00002000:00010000:0.0:1789760793.666251:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131952 -> 131960 00010000:00010000:0.0:1789760793.666264:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d23 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67547136->67551231] (req 67547136->67551231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.666277:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d23 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67547136->67551231] (req 67547136->67551231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.666286:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d23 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67547136->67551231] (req 67547136->67551231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.666293:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760793.666299:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d23 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67547136->67551231] (req 67547136->67551231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.666308:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d23 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67547136->67551231] (req 67547136->67551231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.666326:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d23 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67547136->67551231] (req 67547136->67551231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.666334:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6d23 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67547136->67551231] (req 67547136->67551231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.667221:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554878208:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7839us (8268us total) trans 4294983800 rc 0/0 00000100:00100000:0.0:1789760793.667233:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16502 00000100:00100000:0.0:1789760793.667242:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736887730795 time 736904971048 state 0x2 ref 0 00000100:00100000:2.0:1789760793.671573:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760793.671722:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554878336 00000100:00100000:1.0:1789760793.671733:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554878336 00000100:00100000:1.0:1789760793.671797:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736887730795 time 736904971048 state 0x0 ref 1 00000100:00100000:1.0:1789760793.671813:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736905231464 time 736904971048 state 0x1 ref 1 00000100:00100000:1.0:1789760793.671843:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736905231464 time 736917461414 state 0x0 ref 1 00000100:00100000:1.0:1789760793.671859:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554878336:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760793.671909:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67551232, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20378 00010000:00010000:1.0:1789760793.671918:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6d2a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.671934:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760793.671948:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.671955:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6d2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67551232->67555327] (req 67551232->67555327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.671969:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6d2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67551232->67555327] (req 67551232->67555327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.677343:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983801 is committed 00002000:00010000:1.0:1789760793.677544:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67551232 -> 67555328 00002000:00010000:1.0:1789760793.677550:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131960 -> 131968 00010000:00010000:1.0:1789760793.677563:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6d2a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67551232->67555327] (req 67551232->67555327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.677577:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6d2a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67551232->67555327] (req 67551232->67555327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.677587:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6d2a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67551232->67555327] (req 67551232->67555327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.677608:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.677614:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6d2a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67551232->67555327] (req 67551232->67555327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.677624:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6d2a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67551232->67555327] (req 67551232->67555327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.677644:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6d2a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67551232->67555327] (req 67551232->67555327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.677652:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6d2a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67551232->67555327] (req 67551232->67555327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.677790:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554878336:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5918us (6213us total) trans 4294983801 rc 0/0 00000100:00100000:1.0:1789760793.677803:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16503 00000100:00100000:1.0:1789760793.677814:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736905231464 time 736917461414 state 0x2 ref 0 00000100:00100000:1.0:1789760793.686482:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760793.686647:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554878464 00000100:00100000:2.0:1789760793.686656:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554878464 00000100:00100000:2.0:1789760793.686707:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736905231464 time 736917461414 state 0x0 ref 1 00000100:00100000:2.0:1789760793.686722:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736917721830 time 736917461414 state 0x1 ref 1 00000100:00100000:2.0:1789760793.686756:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736917721830 time 736932373907 state 0x0 ref 1 00000100:00100000:2.0:1789760793.686769:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554878464:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760793.686816:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67555328, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20380 00010000:00010000:2.0:1789760793.686826:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6d31 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.686843:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789760793.686847:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.686854:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6d31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67555328->67559423] (req 67555328->67559423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.686868:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6d31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67555328->67559423] (req 67555328->67559423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.690769:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983802 is committed 00002000:00010000:2.0:1789760793.690869:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67555328 -> 67559424 00002000:00010000:2.0:1789760793.690874:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131968 -> 131976 00010000:00010000:2.0:1789760793.690887:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6d31 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67555328->67559423] (req 67555328->67559423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.690900:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6d31 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67555328->67559423] (req 67555328->67559423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.690909:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6d31 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67555328->67559423] (req 67555328->67559423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.690916:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760793.690921:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6d31 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67555328->67559423] (req 67555328->67559423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.690930:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6d31 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67555328->67559423] (req 67555328->67559423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.690950:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6d31 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67555328->67559423] (req 67555328->67559423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.690958:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6d31 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67555328->67559423] (req 67555328->67559423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.691058:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554878464:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4286us (4580us total) trans 4294983802 rc 0/0 00000100:00100000:2.0:1789760793.691069:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16504 00000100:00100000:2.0:1789760793.691078:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736917721830 time 736932373907 state 0x2 ref 0 00000100:00100000:2.0:1789760793.698302:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760793.698628:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554878592 00000100:00100000:1.0:1789760793.698637:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554878592 00000100:00100000:1.0:1789760793.698687:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736917721830 time 736932373907 state 0x0 ref 1 00000100:00100000:1.0:1789760793.698701:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736932634323 time 736932373907 state 0x1 ref 1 00000100:00100000:1.0:1789760793.698783:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736932634323 time 736944400895 state 0x0 ref 1 00000100:00100000:1.0:1789760793.698796:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554878592:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760793.699621:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67559424, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20388 00010000:00010000:1.0:1789760793.699634:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6d38 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.699651:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760793.699655:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.699662:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6d38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67559424->67563519] (req 67559424->67563519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.699677:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6d38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67559424->67563519] (req 67559424->67563519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.704417:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983803 is committed 00002000:00010000:0.0:1789760793.704538:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67559424 -> 67563520 00002000:00010000:0.0:1789760793.704543:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131976 -> 131984 00010000:00010000:0.0:1789760793.704557:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6d38 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67559424->67563519] (req 67559424->67563519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.704571:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6d38 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67559424->67563519] (req 67559424->67563519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.704580:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6d38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67559424->67563519] (req 67559424->67563519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.704588:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760793.704605:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6d38 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67559424->67563519] (req 67559424->67563519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.704615:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6d38 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67559424->67563519] (req 67559424->67563519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.704635:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6d38 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67559424->67563519] (req 67559424->67563519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.704643:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6d38 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67559424->67563519] (req 67559424->67563519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.704733:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554878592:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5935us (6436us total) trans 4294983803 rc 0/0 00000100:00100000:0.0:1789760793.704744:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16505 00000100:00100000:0.0:1789760793.704753:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736932634323 time 736944400895 state 0x2 ref 0 00000100:00100000:2.0:1789760793.711747:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760793.711849:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554878720 00000100:00100000:1.0:1789760793.711859:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554878720 00000100:00100000:1.0:1789760793.711914:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736932634323 time 736944400895 state 0x0 ref 1 00000100:00100000:1.0:1789760793.711929:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736944661311 time 736944400895 state 0x1 ref 1 00000100:00100000:1.0:1789760793.711958:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736944661311 time 736957576209 state 0x0 ref 1 00000100:00100000:1.0:1789760793.711970:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554878720:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760793.712018:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67563520, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20390 00010000:00010000:1.0:1789760793.712027:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d3f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.712043:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760793.712048:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.712054:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67563520->67567615] (req 67563520->67567615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.712067:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67563520->67567615] (req 67563520->67567615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.716170:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983804 is committed 00002000:00010000:1.0:1789760793.716314:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67563520 -> 67567616 00002000:00010000:1.0:1789760793.716320:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131984 -> 131992 00010000:00010000:1.0:1789760793.716332:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d3f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67563520->67567615] (req 67563520->67567615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.716344:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d3f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67563520->67567615] (req 67563520->67567615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.716353:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d3f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67563520->67567615] (req 67563520->67567615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.716361:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.716365:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d3f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67563520->67567615] (req 67563520->67567615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.716374:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d3f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67563520->67567615] (req 67563520->67567615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.716510:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d3f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67563520->67567615] (req 67563520->67567615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.716523:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d3f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67563520->67567615] (req 67563520->67567615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.716645:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554878720:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4672us (4902us total) trans 4294983804 rc 0/0 00000100:00100000:1.0:1789760793.716656:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16506 00000100:00100000:1.0:1789760793.716665:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736944661311 time 736957576209 state 0x2 ref 0 00000100:00100000:1.0:1789760793.720881:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760793.721045:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554878848 00000100:00100000:0.0:1789760793.721054:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554878848 00000100:00100000:0.0:1789760793.721107:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736944661311 time 736957576209 state 0x0 ref 1 00000100:00100000:0.0:1789760793.721123:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736957836625 time 736957576209 state 0x1 ref 1 00000100:00100000:0.0:1789760793.721204:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736957836625 time 736966821929 state 0x0 ref 1 00000100:00100000:0.0:1789760793.721216:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554878848:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760793.721265:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67567616, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20398 00010000:00010000:0.0:1789760793.721275:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d46 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.721291:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760793.721295:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.721301:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67567616->67571711] (req 67567616->67571711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.721315:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67567616->67571711] (req 67567616->67571711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.725140:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983805 is committed 00002000:00010000:0.0:1789760793.725659:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67567616 -> 67571712 00002000:00010000:0.0:1789760793.725665:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 131992 -> 132000 00010000:00010000:0.0:1789760793.725678:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d46 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67567616->67571711] (req 67567616->67571711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.725691:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d46 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67567616->67571711] (req 67567616->67571711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.725701:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d46 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67567616->67571711] (req 67567616->67571711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.725708:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760793.725713:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d46 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67567616->67571711] (req 67567616->67571711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.725722:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d46 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67567616->67571711] (req 67567616->67571711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.725741:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d46 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67567616->67571711] (req 67567616->67571711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.725749:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d46 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67567616->67571711] (req 67567616->67571711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.725843:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554878848:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4624us (4967us total) trans 4294983805 rc 0/0 00000100:00100000:0.0:1789760793.725853:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16507 00000100:00100000:0.0:1789760793.725863:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736957836625 time 736966821929 state 0x2 ref 0 00000100:00100000:2.0:1789760793.730201:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760793.731858:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554878976 00000100:00100000:1.0:1789760793.731869:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554878976 00000100:00100000:1.0:1789760793.731926:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736957836625 time 736966821929 state 0x0 ref 1 00000100:00100000:1.0:1789760793.731943:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736967082345 time 736966821929 state 0x1 ref 1 00000100:00100000:1.0:1789760793.732015:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736967082345 time 736977633299 state 0x0 ref 1 00000100:00100000:1.0:1789760793.732028:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554878976:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760793.732075:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67571712, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x203a0 00010000:00010000:1.0:1789760793.732084:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d4d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.732102:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:1.0:1789760793.732107:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.732113:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67571712->67575807] (req 67571712->67575807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.732127:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67571712->67575807] (req 67571712->67575807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.735085:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983806 is committed 00002000:00010000:1.0:1789760793.735289:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67571712 -> 67575808 00002000:00010000:1.0:1789760793.735294:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132000 -> 132008 00010000:00010000:1.0:1789760793.735306:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d4d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67571712->67575807] (req 67571712->67575807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.735318:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d4d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67571712->67575807] (req 67571712->67575807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.735328:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d4d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67571712->67575807] (req 67571712->67575807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.735335:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.735339:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d4d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67571712->67575807] (req 67571712->67575807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.735347:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d4d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67571712->67575807] (req 67571712->67575807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.735366:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d4d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67571712->67575807] (req 67571712->67575807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.735374:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d4d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67571712->67575807] (req 67571712->67575807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.736693:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554878976:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4658us (6492us total) trans 4294983806 rc 0/0 00000100:00100000:1.0:1789760793.736705:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16508 00000100:00100000:1.0:1789760793.736715:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736967082345 time 736977633299 state 0x2 ref 0 00000100:00100000:2.0:1789760793.747788:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760793.747893:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554879104 00000100:00100000:0.0:1789760793.747903:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554879104 00000100:00100000:0.0:1789760793.747954:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736967082345 time 736977633299 state 0x0 ref 1 00000100:00100000:0.0:1789760793.747969:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736977893715 time 736977633299 state 0x1 ref 1 00000100:00100000:0.0:1789760793.748001:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736977893715 time 736993619950 state 0x0 ref 1 00000100:00100000:0.0:1789760793.748014:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554879104:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760793.748059:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67575808, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x203a8 00010000:00010000:0.0:1789760793.748068:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d54 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.748083:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789760793.748088:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.748094:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67575808->67579903] (req 67575808->67579903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.748107:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67575808->67579903] (req 67575808->67579903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.754192:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983807 is committed 00002000:00010000:0.0:1789760793.754312:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67575808 -> 67579904 00002000:00010000:0.0:1789760793.754317:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132008 -> 132016 00010000:00010000:0.0:1789760793.754330:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d54 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67575808->67579903] (req 67575808->67579903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.754342:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d54 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67575808->67579903] (req 67575808->67579903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.754351:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d54 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67575808->67579903] (req 67575808->67579903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.754357:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760793.754362:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d54 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67575808->67579903] (req 67575808->67579903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.754370:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d54 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67575808->67579903] (req 67575808->67579903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.754457:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d54 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67575808->67579903] (req 67575808->67579903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.754465:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d54 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67575808->67579903] (req 67575808->67579903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.754555:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554879104:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6538us (6772us total) trans 4294983807 rc 0/0 00000100:00100000:0.0:1789760793.754565:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16509 00000100:00100000:0.0:1789760793.754573:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736977893715 time 736993619950 state 0x2 ref 0 00000100:00100000:2.0:1789760793.759355:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760793.764626:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554879232 00000100:00100000:0.0:1789760793.764641:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554879232 00000100:00100000:0.0:1789760793.764702:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736977893715 time 736993619950 state 0x0 ref 1 00000100:00100000:0.0:1789760793.764716:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736993880366 time 736993619950 state 0x1 ref 1 00000100:00100000:0.0:1789760793.764737:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736993880366 time 737010356456 state 0x0 ref 1 00000100:00100000:0.0:1789760793.764750:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554879232:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760793.764794:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67579904, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x203b0 00010000:00010000:0.0:1789760793.764803:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d5b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.764817:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760793.764823:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.764829:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67579904->67583999] (req 67579904->67583999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.764845:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67579904->67583999] (req 67579904->67583999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.768425:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983808 is committed 00002000:00010000:1.0:1789760793.768540:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67579904 -> 67584000 00002000:00010000:1.0:1789760793.768545:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132016 -> 132024 00010000:00010000:1.0:1789760793.768556:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d5b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67579904->67583999] (req 67579904->67583999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.768567:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d5b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67579904->67583999] (req 67579904->67583999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.768576:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d5b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67579904->67583999] (req 67579904->67583999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.768583:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.768587:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d5b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67579904->67583999] (req 67579904->67583999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.768607:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d5b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67579904->67583999] (req 67579904->67583999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.768628:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d5b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67579904->67583999] (req 67579904->67583999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.768705:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6d5b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67579904->67583999] (req 67579904->67583999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.768796:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554879232:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4044us (9445us total) trans 4294983808 rc 0/0 00000100:00100000:1.0:1789760793.768806:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16510 00000100:00100000:1.0:1789760793.768816:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736993880366 time 737010356456 state 0x2 ref 0 00000100:00100000:2.0:1789760793.771973:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760793.772202:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554879360 00000100:00100000:1.0:1789760793.772223:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554879360 00000100:00100000:1.0:1789760793.772278:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 736993880366 time 737010356456 state 0x0 ref 1 00000100:00100000:1.0:1789760793.772293:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737010616872 time 737010356456 state 0x1 ref 1 00000100:00100000:1.0:1789760793.772325:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737010616872 time 737017943702 state 0x0 ref 1 00000100:00100000:1.0:1789760793.772338:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554879360:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760793.772412:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67584000, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x203b8 00010000:00010000:1.0:1789760793.772422:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d62 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.772438:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760793.772442:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.772449:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67584000->67588095] (req 67584000->67588095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.772468:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67584000->67588095] (req 67584000->67588095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.777102:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983809 is committed 00002000:00010000:1.0:1789760793.777275:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67584000 -> 67588096 00002000:00010000:1.0:1789760793.777280:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132024 -> 132032 00010000:00010000:1.0:1789760793.777295:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d62 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67584000->67588095] (req 67584000->67588095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.777307:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d62 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67584000->67588095] (req 67584000->67588095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.777317:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67584000->67588095] (req 67584000->67588095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.777324:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.777329:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d62 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67584000->67588095] (req 67584000->67588095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.777338:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d62 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67584000->67588095] (req 67584000->67588095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.777356:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d62 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67584000->67588095] (req 67584000->67588095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.777364:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d62 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67584000->67588095] (req 67584000->67588095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.777516:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554879360:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5174us (5547us total) trans 4294983809 rc 0/0 00000100:00100000:1.0:1789760793.777528:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16511 00000100:00100000:1.0:1789760793.777537:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737010616872 time 737017943702 state 0x2 ref 0 00000100:00100000:2.0:1789760793.793591:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760793.793698:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554879488 00000100:00100000:1.0:1789760793.793709:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554879488 00000100:00100000:1.0:1789760793.793767:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737010616872 time 737017943702 state 0x0 ref 1 00000100:00100000:1.0:1789760793.793782:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737018204118 time 737017943702 state 0x1 ref 1 00000100:00100000:1.0:1789760793.793818:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737018204118 time 737039436431 state 0x0 ref 1 00000100:00100000:1.0:1789760793.793830:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554879488:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760793.793877:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67588096, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x203c0 00010000:00010000:1.0:1789760793.793887:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d69 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.793904:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760793.793908:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.793923:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67588096->67592191] (req 67588096->67592191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.793938:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67588096->67592191] (req 67588096->67592191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.797647:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983810 is committed 00002000:00010000:1.0:1789760793.797757:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67588096 -> 67592192 00002000:00010000:1.0:1789760793.797764:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132032 -> 132040 00010000:00010000:1.0:1789760793.797777:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d69 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67588096->67592191] (req 67588096->67592191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.797790:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d69 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67588096->67592191] (req 67588096->67592191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.797803:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67588096->67592191] (req 67588096->67592191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.797810:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.797818:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d69 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67588096->67592191] (req 67588096->67592191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.797828:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d69 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67588096->67592191] (req 67588096->67592191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.797851:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d69 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67588096->67592191] (req 67588096->67592191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.797859:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d69 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67588096->67592191] (req 67588096->67592191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.798059:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554879488:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4223us (4471us total) trans 4294983810 rc 0/0 00000100:00100000:1.0:1789760793.798073:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16512 00000100:00100000:1.0:1789760793.798082:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737018204118 time 737039436431 state 0x2 ref 0 00000100:00100000:2.0:1789760793.806185:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760793.806280:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554879616 00000100:00100000:0.0:1789760793.806289:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554879616 00000100:00100000:0.0:1789760793.806342:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737018204118 time 737039436431 state 0x0 ref 1 00000100:00100000:0.0:1789760793.806366:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737039696847 time 737039436431 state 0x1 ref 1 00000100:00100000:0.0:1789760793.806476:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737039696847 time 737052092914 state 0x0 ref 1 00000100:00100000:0.0:1789760793.806489:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554879616:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760793.806536:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67592192, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x203c8 00010000:00010000:0.0:1789760793.806545:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d70 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.806559:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760793.806563:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.806569:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67592192->67596287] (req 67592192->67596287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.806584:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67592192->67596287] (req 67592192->67596287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.811114:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983811 is committed 00002000:00010000:0.0:1789760793.812216:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67592192 -> 67596288 00002000:00010000:0.0:1789760793.812222:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132040 -> 132048 00010000:00010000:0.0:1789760793.812235:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d70 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67592192->67596287] (req 67592192->67596287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.812331:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d70 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67592192->67596287] (req 67592192->67596287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.812340:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d70 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67592192->67596287] (req 67592192->67596287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.812347:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760793.812529:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d70 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67592192->67596287] (req 67592192->67596287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.812540:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d70 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67592192->67596287] (req 67592192->67596287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.812561:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d70 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67592192->67596287] (req 67592192->67596287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.812569:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d70 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67592192->67596287] (req 67592192->67596287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.813149:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554879616:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6657us (6967us total) trans 4294983811 rc 0/0 00000100:00100000:0.0:1789760793.813160:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16513 00000100:00100000:0.0:1789760793.813170:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737039696847 time 737052092914 state 0x2 ref 0 00000100:00100000:2.0:1789760793.826643:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760793.827502:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554879744 00000100:00100000:0.0:1789760793.827512:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554879744 00000100:00100000:0.0:1789760793.827566:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737039696847 time 737052092914 state 0x0 ref 1 00000100:00100000:0.0:1789760793.828511:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737052353330 time 737052092914 state 0x1 ref 1 00000100:00100000:0.0:1789760793.828539:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737052353330 time 737074158751 state 0x0 ref 1 00000100:00100000:0.0:1789760793.828555:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554879744:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760793.828622:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67596288, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x203d0 00010000:00010000:0.0:1789760793.828632:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d77 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.828647:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760793.828652:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.828657:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67596288->67600383] (req 67596288->67600383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.828672:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67596288->67600383] (req 67596288->67600383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.832060:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983812 is committed 00002000:00010000:0.0:1789760793.832333:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67596288 -> 67600384 00002000:00010000:0.0:1789760793.832339:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132048 -> 132056 00010000:00010000:0.0:1789760793.832352:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d77 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67596288->67600383] (req 67596288->67600383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.832365:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d77 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67596288->67600383] (req 67596288->67600383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.832375:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67596288->67600383] (req 67596288->67600383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.832478:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760793.832483:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d77 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67596288->67600383] (req 67596288->67600383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.832493:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d77 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67596288->67600383] (req 67596288->67600383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.832513:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d77 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67596288->67600383] (req 67596288->67600383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.832521:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d77 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67596288->67600383] (req 67596288->67600383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.835244:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554879744:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6683us (8600us total) trans 4294983812 rc 0/0 00000100:00100000:1.0:1789760793.835260:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16514 00000100:00100000:1.0:1789760793.835269:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737052353330 time 737074158751 state 0x2 ref 0 00000100:00100000:2.0:1789760793.838500:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760793.838590:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554879872 00000100:00100000:1.0:1789760793.838610:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554879872 00000100:00100000:1.0:1789760793.838664:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737052353330 time 737074158751 state 0x0 ref 1 00000100:00100000:1.0:1789760793.838680:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737074419167 time 737074158751 state 0x1 ref 1 00000100:00100000:1.0:1789760793.838713:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737074419167 time 737084330892 state 0x0 ref 1 00000100:00100000:1.0:1789760793.838725:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554879872:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760793.838771:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67600384, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x203d8 00010000:00010000:1.0:1789760793.838780:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d7e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.838794:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760793.838798:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.838805:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67600384->67604479] (req 67600384->67604479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.838824:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67600384->67604479] (req 67600384->67604479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.846329:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983813 is committed 00002000:00010000:1.0:1789760793.846498:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67600384 -> 67604480 00002000:00010000:1.0:1789760793.846504:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132056 -> 132064 00010000:00010000:1.0:1789760793.846517:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d7e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67600384->67604479] (req 67600384->67604479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.846531:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d7e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67600384->67604479] (req 67600384->67604479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.846542:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67600384->67604479] (req 67600384->67604479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.846549:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.846555:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d7e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67600384->67604479] (req 67600384->67604479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.846564:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d7e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67600384->67604479] (req 67600384->67604479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.846605:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d7e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67600384->67604479] (req 67600384->67604479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.846613:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6d7e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67600384->67604479] (req 67600384->67604479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.846706:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554879872:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7978us (8223us total) trans 4294983813 rc 0/0 00000100:00100000:1.0:1789760793.846717:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16515 00000100:00100000:1.0:1789760793.846727:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737074419167 time 737084330892 state 0x2 ref 0 00000100:00100000:2.0:1789760793.854516:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760793.854557:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760793.854690:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760793.854723:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760793.854820:0:6514:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554880000 00000100:00100000:1.0:1789760793.854831:0:6514:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554880000 00000100:00100000:1.0:1789760793.854884:0:6514:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 476 00000100:00100000:1.0:1789760793.854894:0:6514:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc320b5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9765eb0b-55b9-40b6-bfdf-ffee156c3226+13:2414:x1876699554880000:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:1.0:1789760793.854974:0:6514:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc320b5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9765eb0b-55b9-40b6-bfdf-ffee156c3226+13:2414:x1876699554880000:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 82us (284us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760793.854982:0:6514:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 476 00000100:00100000:2.0:1789760793.857867:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760793.857904:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760793.862766:0:6522:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554880128 00000100:00100000:0.0:1789760793.862783:0:6522:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554880128 00000100:00100000:0.0:1789760793.862847:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554880640 00000100:00100000:0.0:1789760793.862852:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554880640 00000100:00100000:0.0:1789760793.862924:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737074419167 time 737084330892 state 0x0 ref 1 00000100:00100000:0.0:1789760793.862942:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737084591308 time 737084330892 state 0x1 ref 1 00000100:00100000:3.0:1789760793.863186:0:11089:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554880384 00000100:00100000:3.0:1789760793.863192:0:11089:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554880384 00000100:00100000:0.0:1789760793.863655:0:6521:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554880256 00000100:00100000:0.0:1789760793.863660:0:6521:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554880256 00000100:00100000:3.0:1789760793.863726:0:11089:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554880512 00000100:00100000:3.0:1789760793.863729:0:11089:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554880512 00000100:00100000:1.0:1789760793.864824:0:6520:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 905 00000100:00100000:1.0:1789760793.864844:0:6520:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2414:x1876699554880384:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:1.0:1789760793.864931:0:6520:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2414:x1876699554880384:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 96us (10374us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760793.864938:0:6520:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 905 00000100:00100000:0.0:1789760793.865492:0:6521:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 906 00000100:00100000:0.0:1789760793.865507:0:6521:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554880256:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:0.0:1789760793.865563:0:6521:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554880256:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 63us (10839us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760793.865570:0:6521:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 906 00000100:00100000:0.0:1789760793.865630:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737084591308 time 737111245199 state 0x0 ref 1 00000100:00100000:0.0:1789760793.865642:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554880640:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760793.865686:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67604480, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x203e0 00010000:00010000:0.0:1789760793.865696:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d85 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.865711:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760793.865716:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.865723:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67604480->67608575] (req 67604480->67608575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.865736:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67604480->67608575] (req 67604480->67608575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.866168:0:6522:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 907 00000100:00100000:0.0:1789760793.866176:0:6522:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+8:2414:x1876699554880128:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:0.0:1789760793.866209:0:6522:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+8:2414:x1876699554880128:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 34us (11702us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760793.866216:0:6522:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 907 00000100:00100000:3.0:1789760793.866834:0:11089:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.22@tcp, seq: 908 00000100:00100000:3.0:1789760793.866846:0:11089:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2414:x1876699554880512:12345-192.168.203.22@tcp:400:kworker.0 00000100:00100000:3.0:1789760793.866898:0:11089:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:92be3de5-27ce-42b1-87e1-12e8c11aec3e+5:2414:x1876699554880512:12345-192.168.203.22@tcp:400:kworker.0 Request processed in 56us (8993us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760793.866905:0:11089:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.22@tcp, seq: 908 00000100:00080000:3.0:1789760793.866934:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00080000:3.0:1789760793.866939:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:2.0:1789760793.870145:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983814 is committed 00002000:00010000:0.0:1789760793.871514:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67604480 -> 67608576 00002000:00010000:0.0:1789760793.871520:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132064 -> 132072 00010000:00010000:0.0:1789760793.871536:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d85 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67604480->67608575] (req 67604480->67608575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.871550:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d85 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67604480->67608575] (req 67604480->67608575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.871559:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d85 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67604480->67608575] (req 67604480->67608575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.871566:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760793.871571:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d85 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67604480->67608575] (req 67604480->67608575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.871580:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d85 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67604480->67608575] (req 67604480->67608575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.871610:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d85 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67604480->67608575] (req 67604480->67608575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.871619:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6d85 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67604480->67608575] (req 67604480->67608575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.871706:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554880640:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6060us (13845us total) trans 4294983814 rc 0/0 00000100:00100000:0.0:1789760793.871717:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16516 00000100:00100000:0.0:1789760793.871726:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737084591308 time 737111245199 state 0x2 ref 0 00000100:00100000:3.0:1789760793.876079:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760793.876168:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554880768 00000100:00100000:0.0:1789760793.876178:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554880768 00000100:00100000:0.0:1789760793.876245:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737084591308 time 737111245199 state 0x0 ref 1 00000100:00100000:0.0:1789760793.876262:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737111505615 time 737111245199 state 0x1 ref 1 00000100:00100000:0.0:1789760793.876294:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737111505615 time 737121912523 state 0x0 ref 1 00000100:00100000:0.0:1789760793.876305:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554880768:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760793.876353:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67608576, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x203e8 00010000:00010000:0.0:1789760793.876362:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6d8c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.876404:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760793.876410:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.876416:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6d8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67608576->67612671] (req 67608576->67612671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.876432:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6d8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67608576->67612671] (req 67608576->67612671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.879975:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983815 is committed 00002000:00010000:1.0:1789760793.880098:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67608576 -> 67612672 00002000:00010000:1.0:1789760793.880103:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132072 -> 132080 00010000:00010000:1.0:1789760793.880116:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6d8c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67608576->67612671] (req 67608576->67612671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.880127:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6d8c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67608576->67612671] (req 67608576->67612671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.880141:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6d8c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67608576->67612671] (req 67608576->67612671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.880148:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.880152:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6d8c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67608576->67612671] (req 67608576->67612671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.880161:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6d8c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67608576->67612671] (req 67608576->67612671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.880178:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6d8c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67608576->67612671] (req 67608576->67612671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.880185:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c6d8c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67608576->67612671] (req 67608576->67612671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.880265:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554880768:12345-192.168.203.22@tcp:4:dd.0 Request processed in 3956us (4191us total) trans 4294983815 rc 0/0 00000100:00100000:1.0:1789760793.880274:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16517 00000100:00100000:1.0:1789760793.880283:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737111505615 time 737121912523 state 0x2 ref 0 00000100:00100000:1.0:1789760793.884852:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760793.884941:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554880896 00000100:00100000:2.0:1789760793.884950:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554880896 00000100:00100000:2.0:1789760793.885052:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737111505615 time 737121912523 state 0x0 ref 1 00000100:00100000:2.0:1789760793.885067:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737122172939 time 737121912523 state 0x1 ref 1 00000100:00100000:2.0:1789760793.885153:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737122172939 time 737130771357 state 0x0 ref 1 00000100:00100000:2.0:1789760793.885166:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554880896:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760793.885214:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67612672, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x203f0 00010000:00010000:2.0:1789760793.885223:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6d93 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.885238:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760793.885241:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.885247:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6d93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67612672->67616767] (req 67612672->67616767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.885259:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6d93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67612672->67616767] (req 67612672->67616767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.891807:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983816 is committed 00002000:00010000:1.0:1789760793.891941:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67612672 -> 67616768 00002000:00010000:1.0:1789760793.891946:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132080 -> 132088 00010000:00010000:1.0:1789760793.891960:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6d93 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67612672->67616767] (req 67612672->67616767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.891985:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6d93 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67612672->67616767] (req 67612672->67616767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.891994:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6d93 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67612672->67616767] (req 67612672->67616767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.892002:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.892007:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6d93 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67612672->67616767] (req 67612672->67616767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.892016:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6d93 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67612672->67616767] (req 67612672->67616767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.892036:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6d93 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67612672->67616767] (req 67612672->67616767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.892044:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6d93 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67612672->67616767] (req 67612672->67616767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.892174:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554880896:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7006us (7496us total) trans 4294983816 rc 0/0 00000100:00100000:1.0:1789760793.892185:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16518 00000100:00100000:1.0:1789760793.892195:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737122172939 time 737130771357 state 0x2 ref 0 00000100:00100000:2.0:1789760793.896801:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760793.896866:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554881024 00000100:00100000:1.0:1789760793.896876:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554881024 00000100:00100000:1.0:1789760793.896923:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737122172939 time 737130771357 state 0x0 ref 1 00000100:00100000:1.0:1789760793.896937:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737131031773 time 737130771357 state 0x1 ref 1 00000100:00100000:1.0:1789760793.896963:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737131031773 time 737142582794 state 0x0 ref 1 00000100:00100000:1.0:1789760793.896973:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05b8f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554881024:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760793.897025:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67616768, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x203f8 00010000:00010000:1.0:1789760793.897035:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6d9a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.897051:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760793.897055:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.897061:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6d9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67616768->67620863] (req 67616768->67620863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.897075:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6d9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67616768->67620863] (req 67616768->67620863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.901284:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983817 is committed 00002000:00010000:1.0:1789760793.901548:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67616768 -> 67620864 00002000:00010000:1.0:1789760793.901554:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132088 -> 132096 00010000:00010000:1.0:1789760793.901567:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6d9a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67616768->67620863] (req 67616768->67620863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.901580:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6d9a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67616768->67620863] (req 67616768->67620863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.901616:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6d9a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67616768->67620863] (req 67616768->67620863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.901624:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.901630:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6d9a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67616768->67620863] (req 67616768->67620863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.901638:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6d9a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67616768->67620863] (req 67616768->67620863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.901657:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6d9a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67616768->67620863] (req 67616768->67620863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.901664:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6d9a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67616768->67620863] (req 67616768->67620863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.901876:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05b8f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554881024:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4898us (5078us total) trans 4294983817 rc 0/0 00000100:00100000:1.0:1789760793.901887:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16519 00000100:00100000:1.0:1789760793.901896:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737131031773 time 737142582794 state 0x2 ref 0 00000100:00100000:3.0:1789760793.906092:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760793.907611:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554881152 00000100:00100000:1.0:1789760793.907624:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554881152 00000100:00100000:1.0:1789760793.907680:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737131031773 time 737142582794 state 0x0 ref 1 00000100:00100000:1.0:1789760793.907753:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737142843210 time 737142582794 state 0x1 ref 1 00000100:00100000:1.0:1789760793.907779:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737142843210 time 737153397986 state 0x0 ref 1 00000100:00100000:1.0:1789760793.907792:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554881152:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760793.907840:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67620864, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20400 00010000:00010000:1.0:1789760793.907850:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6da1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.907866:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760793.907870:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.907877:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6da1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67620864->67624959] (req 67620864->67624959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.907892:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6da1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67620864->67624959] (req 67620864->67624959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.912457:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983818 is committed 00002000:00010000:1.0:1789760793.912549:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67620864 -> 67624960 00002000:00010000:1.0:1789760793.912554:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132096 -> 132104 00010000:00010000:1.0:1789760793.912566:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6da1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67620864->67624959] (req 67620864->67624959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.912578:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6da1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67620864->67624959] (req 67620864->67624959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.912587:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6da1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67620864->67624959] (req 67620864->67624959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.912610:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.912615:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6da1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67620864->67624959] (req 67620864->67624959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.912623:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6da1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67620864->67624959] (req 67620864->67624959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.912641:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6da1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67620864->67624959] (req 67620864->67624959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.912649:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6da1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67620864->67624959] (req 67620864->67624959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.912809:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554881152:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5013us (6724us total) trans 4294983818 rc 0/0 00000100:00100000:1.0:1789760793.912819:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16520 00000100:00100000:1.0:1789760793.912945:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737142843210 time 737153397986 state 0x2 ref 0 00000100:00100000:0.0:1789760793.917898:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760793.918040:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554881280 00000100:00100000:2.0:1789760793.918051:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554881280 00000100:00100000:2.0:1789760793.918105:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737142843210 time 737153397986 state 0x0 ref 1 00000100:00100000:2.0:1789760793.918120:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737153658402 time 737153397986 state 0x1 ref 1 00000100:00100000:2.0:1789760793.918146:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737153658402 time 737163764423 state 0x0 ref 1 00000100:00100000:2.0:1789760793.918157:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554881280:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760793.918203:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67624960, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20408 00010000:00010000:2.0:1789760793.918211:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6da8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.918227:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760793.918231:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.918237:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6da8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67624960->67629055] (req 67624960->67629055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.918251:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6da8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67624960->67629055] (req 67624960->67629055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760793.921330:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983819 is committed 00002000:00010000:2.0:1789760793.921616:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67624960 -> 67629056 00002000:00010000:2.0:1789760793.921621:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132104 -> 132112 00010000:00010000:2.0:1789760793.921632:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6da8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67624960->67629055] (req 67624960->67629055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.921644:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6da8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67624960->67629055] (req 67624960->67629055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.921653:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6da8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67624960->67629055] (req 67624960->67629055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.921660:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760793.921665:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6da8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67624960->67629055] (req 67624960->67629055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.921673:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6da8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67624960->67629055] (req 67624960->67629055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.921691:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6da8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67624960->67629055] (req 67624960->67629055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.921771:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6da8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67624960->67629055] (req 67624960->67629055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.921860:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554881280:12345-192.168.203.22@tcp:4:dd.0 Request processed in 3699us (3967us total) trans 4294983819 rc 0/0 00000100:00100000:2.0:1789760793.921870:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16521 00000100:00100000:2.0:1789760793.921879:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737153658402 time 737163764423 state 0x2 ref 0 00000100:00100000:2.0:1789760793.925899:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760793.925994:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554881408 00000100:00100000:0.0:1789760793.926005:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554881408 00000100:00100000:0.0:1789760793.926061:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737153658402 time 737163764423 state 0x0 ref 1 00000100:00100000:0.0:1789760793.926076:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737164024839 time 737163764423 state 0x1 ref 1 00000100:00100000:0.0:1789760793.926112:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737164024839 time 737171731036 state 0x0 ref 1 00000100:00100000:0.0:1789760793.926124:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05b8f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554881408:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760793.926497:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67629056, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20410 00010000:00010000:0.0:1789760793.926507:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6daf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.926520:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760793.926525:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.926531:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6daf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67629056->67633151] (req 67629056->67633151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.926543:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6daf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67629056->67633151] (req 67629056->67633151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.934350:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983820 is committed 00002000:00010000:0.0:1789760793.934603:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67629056 -> 67633152 00002000:00010000:0.0:1789760793.934608:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132112 -> 132120 00010000:00010000:0.0:1789760793.934622:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6daf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67629056->67633151] (req 67629056->67633151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.934635:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6daf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67629056->67633151] (req 67629056->67633151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.934644:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6daf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67629056->67633151] (req 67629056->67633151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.934652:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760793.934657:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6daf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67629056->67633151] (req 67629056->67633151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.934666:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6daf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67629056->67633151] (req 67629056->67633151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.934687:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6daf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67629056->67633151] (req 67629056->67633151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.934756:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6daf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67629056->67633151] (req 67629056->67633151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.934849:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05b8f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554881408:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8721us (8956us total) trans 4294983820 rc 0/0 00000100:00100000:0.0:1789760793.934859:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16522 00000100:00100000:0.0:1789760793.934869:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737164024839 time 737171731036 state 0x2 ref 0 00000100:00100000:3.0:1789760793.938827:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760793.939139:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554881536 00000100:00100000:1.0:1789760793.939148:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554881536 00000100:00100000:1.0:1789760793.939200:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737164024839 time 737171731036 state 0x0 ref 1 00000100:00100000:1.0:1789760793.939215:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737171991452 time 737171731036 state 0x1 ref 1 00000100:00100000:1.0:1789760793.939246:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737171991452 time 737184864312 state 0x0 ref 1 00000100:00100000:1.0:1789760793.939261:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554881536:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760793.939311:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67633152, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20418 00010000:00010000:1.0:1789760793.939319:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6db6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.939334:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760793.939338:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760793.939344:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6db6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67633152->67637247] (req 67633152->67637247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.939677:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6db6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67633152->67637247] (req 67633152->67637247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.944002:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983821 is committed 00002000:00010000:1.0:1789760793.944151:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67633152 -> 67637248 00002000:00010000:1.0:1789760793.944156:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132120 -> 132128 00010000:00010000:1.0:1789760793.944169:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6db6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67633152->67637247] (req 67633152->67637247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.944182:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6db6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67633152->67637247] (req 67633152->67637247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.944191:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6db6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67633152->67637247] (req 67633152->67637247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.944199:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.944205:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6db6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67633152->67637247] (req 67633152->67637247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.944215:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6db6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67633152->67637247] (req 67633152->67637247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.944235:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6db6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67633152->67637247] (req 67633152->67637247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.944243:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6db6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67633152->67637247] (req 67633152->67637247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.944335:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554881536:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5071us (5513us total) trans 4294983821 rc 0/0 00000100:00100000:1.0:1789760793.944345:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16523 00000100:00100000:1.0:1789760793.944354:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737171991452 time 737184864312 state 0x2 ref 0 00000100:00100000:1.0:1789760793.957122:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760793.957342:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554881664 00000100:00100000:2.0:1789760793.957361:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554881664 00000100:00100000:2.0:1789760793.957436:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737171991452 time 737184864312 state 0x0 ref 1 00000100:00100000:2.0:1789760793.957454:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737185124728 time 737184864312 state 0x1 ref 1 00000100:00100000:2.0:1789760793.957487:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737185124728 time 737203105182 state 0x0 ref 1 00000100:00100000:2.0:1789760793.957500:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554881664:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760793.957548:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67637248, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20420 00010000:00010000:2.0:1789760793.957557:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6dbd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.957573:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760793.957578:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.957585:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6dbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67637248->67641343] (req 67637248->67641343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.957617:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6dbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67637248->67641343] (req 67637248->67641343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760793.963190:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983822 is committed 00002000:00010000:2.0:1789760793.963321:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67637248 -> 67641344 00002000:00010000:2.0:1789760793.963327:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132128 -> 132136 00010000:00010000:2.0:1789760793.963339:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6dbd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67637248->67641343] (req 67637248->67641343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.963364:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6dbd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67637248->67641343] (req 67637248->67641343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.963373:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6dbd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67637248->67641343] (req 67637248->67641343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.963581:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760793.963586:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6dbd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67637248->67641343] (req 67637248->67641343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.963607:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6dbd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67637248->67641343] (req 67637248->67641343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.963628:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6dbd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67637248->67641343] (req 67637248->67641343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.963637:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6dbd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67637248->67641343] (req 67637248->67641343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760793.963731:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554881664:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6226us (6613us total) trans 4294983822 rc 0/0 00000100:00100000:2.0:1789760793.963742:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16524 00000100:00100000:2.0:1789760793.963751:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737185124728 time 737203105182 state 0x2 ref 0 00000100:00100000:3.0:1789760793.967172:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760793.967265:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554881792 00000100:00100000:2.0:1789760793.967275:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554881792 00000100:00100000:2.0:1789760793.967329:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737185124728 time 737203105182 state 0x0 ref 1 00000100:00100000:2.0:1789760793.967344:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737203365598 time 737203105182 state 0x1 ref 1 00000100:00100000:2.0:1789760793.967463:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737203365598 time 737212994061 state 0x0 ref 1 00000100:00100000:2.0:1789760793.967483:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554881792:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760793.967531:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67641344, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20428 00010000:00010000:2.0:1789760793.967540:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6dc4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.967555:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760793.967560:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760793.967566:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6dc4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67641344->67645439] (req 67641344->67645439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760793.967580:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6dc4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67641344->67645439] (req 67641344->67645439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.972422:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983823 is committed 00002000:00010000:1.0:1789760793.972614:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67641344 -> 67645440 00002000:00010000:1.0:1789760793.972620:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132136 -> 132144 00010000:00010000:1.0:1789760793.972633:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6dc4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67641344->67645439] (req 67641344->67645439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.972647:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6dc4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67641344->67645439] (req 67641344->67645439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.972656:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6dc4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67641344->67645439] (req 67641344->67645439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.972664:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760793.972668:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6dc4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67641344->67645439] (req 67641344->67645439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.972676:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6dc4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67641344->67645439] (req 67641344->67645439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.972693:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6dc4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67641344->67645439] (req 67641344->67645439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760793.972701:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6dc4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67641344->67645439] (req 67641344->67645439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760793.972845:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554881792:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5359us (5676us total) trans 4294983823 rc 0/0 00000100:00100000:1.0:1789760793.972855:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16525 00000100:00100000:1.0:1789760793.972865:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737203365598 time 737212994061 state 0x2 ref 0 00000100:00100000:3.0:1789760793.982157:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760793.982669:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554881920 00000100:00100000:0.0:1789760793.982687:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554881920 00000100:00100000:0.0:1789760793.982743:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737203365598 time 737212994061 state 0x0 ref 1 00000100:00100000:0.0:1789760793.982760:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737213254477 time 737212994061 state 0x1 ref 1 00000100:00100000:0.0:1789760793.982841:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737213254477 time 737228458382 state 0x0 ref 1 00000100:00100000:0.0:1789760793.982853:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554881920:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760793.982916:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67645440, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20430 00010000:00010000:0.0:1789760793.982926:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c6dcb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.982942:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760793.982948:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760793.982954:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c6dcb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67645440->67649535] (req 67645440->67649535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.982968:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c6dcb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67645440->67649535] (req 67645440->67649535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760793.988196:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983824 is committed 00002000:00010000:0.0:1789760793.997849:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67645440 -> 67649536 00002000:00010000:0.0:1789760793.997858:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132144 -> 132152 00010000:00010000:0.0:1789760793.997872:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c6dcb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67645440->67649535] (req 67645440->67649535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.997890:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c6dcb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67645440->67649535] (req 67645440->67649535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.997899:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c6dcb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67645440->67649535] (req 67645440->67649535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.997906:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760793.997911:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c6dcb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67645440->67649535] (req 67645440->67649535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.997920:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c6dcb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67645440->67649535] (req 67645440->67649535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.997938:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c6dcb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67645440->67649535] (req 67645440->67649535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760793.997945:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c6dcb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67645440->67649535] (req 67645440->67649535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760793.998610:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554881920:12345-192.168.203.22@tcp:4:dd.0 Request processed in 15743us (16448us total) trans 4294983824 rc 0/0 00000100:00100000:0.0:1789760793.998622:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16526 00000100:00100000:0.0:1789760793.998632:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737213254477 time 737228458382 state 0x2 ref 0 00000100:00100000:0.0:1789760794.006467:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760794.008532:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554882048 00000100:00100000:1.0:1789760794.008545:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554882048 00000100:00100000:1.0:1789760794.008612:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737213254477 time 737228458382 state 0x0 ref 1 00000100:00100000:1.0:1789760794.008634:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737228718798 time 737228458382 state 0x1 ref 1 00000100:00100000:1.0:1789760794.008671:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737228718798 time 737254289759 state 0x0 ref 1 00000100:00100000:1.0:1789760794.008687:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554882048:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760794.008738:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67649536, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20438 00010000:00010000:1.0:1789760794.008748:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6dd2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.008773:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760794.008778:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.008784:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6dd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67649536->67653631] (req 67649536->67653631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.008798:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6dd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67649536->67653631] (req 67649536->67653631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.017126:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983825 is committed 00002000:00010000:1.0:1789760794.017246:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67649536 -> 67653632 00002000:00010000:1.0:1789760794.017251:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132152 -> 132160 00010000:00010000:1.0:1789760794.017263:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6dd2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67649536->67653631] (req 67649536->67653631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.017276:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6dd2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67649536->67653631] (req 67649536->67653631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.017285:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6dd2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67649536->67653631] (req 67649536->67653631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.017292:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.017297:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6dd2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67649536->67653631] (req 67649536->67653631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.017306:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6dd2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67649536->67653631] (req 67649536->67653631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.017325:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6dd2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67649536->67653631] (req 67649536->67653631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.017333:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6dd2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67649536->67653631] (req 67649536->67653631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.017542:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554882048:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8853us (11080us total) trans 4294983825 rc 0/0 00000100:00100000:1.0:1789760794.017553:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16527 00000100:00100000:1.0:1789760794.017562:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737228718798 time 737254289759 state 0x2 ref 0 00000100:00100000:2.0:1789760794.023481:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760794.023585:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554882176 00000100:00100000:1.0:1789760794.023603:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554882176 00000100:00100000:1.0:1789760794.023653:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737228718798 time 737254289759 state 0x0 ref 1 00000100:00100000:1.0:1789760794.023667:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737254550175 time 737254289759 state 0x1 ref 1 00000100:00100000:1.0:1789760794.023750:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737254550175 time 737269368275 state 0x0 ref 1 00000100:00100000:1.0:1789760794.023764:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05b8df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554882176:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760794.023810:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67653632, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20440 00010000:00010000:1.0:1789760794.023820:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6dd9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.023845:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760794.023850:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.023856:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6dd9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67653632->67657727] (req 67653632->67657727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.023868:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6dd9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67653632->67657727] (req 67653632->67657727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.028075:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983826 is committed 00002000:00010000:1.0:1789760794.028175:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67653632 -> 67657728 00002000:00010000:1.0:1789760794.028180:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132160 -> 132168 00010000:00010000:1.0:1789760794.028192:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6dd9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67653632->67657727] (req 67653632->67657727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.028205:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6dd9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67653632->67657727] (req 67653632->67657727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.028214:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6dd9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67653632->67657727] (req 67653632->67657727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.028222:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.028227:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6dd9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67653632->67657727] (req 67653632->67657727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.028235:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6dd9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67653632->67657727] (req 67653632->67657727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.028254:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6dd9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67653632->67657727] (req 67653632->67657727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.028262:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6dd9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67653632->67657727] (req 67653632->67657727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.028457:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05b8df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554882176:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4581us (4872us total) trans 4294983826 rc 0/0 00000100:00100000:1.0:1789760794.028471:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16528 00000100:00100000:1.0:1789760794.028481:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737254550175 time 737269368275 state 0x2 ref 0 00000100:00100000:2.0:1789760794.032108:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760794.032164:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554882304 00000100:00100000:1.0:1789760794.032173:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554882304 00000100:00100000:1.0:1789760794.032228:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737254550175 time 737269368275 state 0x0 ref 1 00000100:00100000:1.0:1789760794.032243:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737269628691 time 737269368275 state 0x1 ref 1 00000100:00100000:1.0:1789760794.032320:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737269628691 time 737277937795 state 0x0 ref 1 00000100:00100000:1.0:1789760794.032333:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05b8d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554882304:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760794.032518:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67657728, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20448 00010000:00010000:1.0:1789760794.032526:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6de0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.032539:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760794.032543:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.032548:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6de0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67657728->67661823] (req 67657728->67661823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.032558:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6de0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67657728->67661823] (req 67657728->67661823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.036154:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983827 is committed 00002000:00010000:1.0:1789760794.036300:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67657728 -> 67661824 00002000:00010000:1.0:1789760794.036305:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132168 -> 132176 00010000:00010000:1.0:1789760794.036319:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6de0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67657728->67661823] (req 67657728->67661823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.036331:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6de0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67657728->67661823] (req 67657728->67661823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.036339:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6de0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67657728->67661823] (req 67657728->67661823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.036347:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.036488:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6de0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67657728->67661823] (req 67657728->67661823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.036498:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6de0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67657728->67661823] (req 67657728->67661823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.036518:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6de0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67657728->67661823] (req 67657728->67661823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.036532:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6de0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67657728->67661823] (req 67657728->67661823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.036634:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05b8d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554882304:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4298us (4539us total) trans 4294983827 rc 0/0 00000100:00100000:1.0:1789760794.036645:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16529 00000100:00100000:1.0:1789760794.036655:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737269628691 time 737277937795 state 0x2 ref 0 00000100:00100000:0.0:1789760794.040769:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760794.040876:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554882432 00000100:00100000:1.0:1789760794.040885:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554882432 00000100:00100000:1.0:1789760794.040937:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737269628691 time 737277937795 state 0x0 ref 1 00000100:00100000:1.0:1789760794.040952:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737278198211 time 737277937795 state 0x1 ref 1 00000100:00100000:1.0:1789760794.040990:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737278198211 time 737286608894 state 0x0 ref 1 00000100:00100000:1.0:1789760794.041004:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554882432:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760794.041049:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67661824, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20450 00010000:00010000:1.0:1789760794.041057:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6de7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.041077:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760794.041081:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.041087:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6de7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67661824->67665919] (req 67661824->67665919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.041105:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6de7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67661824->67665919] (req 67661824->67665919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.045050:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983828 is committed 00002000:00010000:1.0:1789760794.046737:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67661824 -> 67665920 00002000:00010000:1.0:1789760794.046745:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132176 -> 132184 00010000:00010000:1.0:1789760794.046758:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6de7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67661824->67665919] (req 67661824->67665919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.046772:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6de7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67661824->67665919] (req 67661824->67665919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.046787:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6de7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67661824->67665919] (req 67661824->67665919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.046794:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.046799:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6de7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67661824->67665919] (req 67661824->67665919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.046809:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6de7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67661824->67665919] (req 67661824->67665919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.046826:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6de7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67661824->67665919] (req 67661824->67665919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.046833:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c6de7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67661824->67665919] (req 67661824->67665919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.046923:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554882432:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5917us (6157us total) trans 4294983828 rc 0/0 00000100:00100000:1.0:1789760794.046933:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16530 00000100:00100000:1.0:1789760794.046942:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737278198211 time 737286608894 state 0x2 ref 0 00000100:00100000:2.0:1789760794.050812:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760794.050907:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554882560 00000100:00100000:0.0:1789760794.050917:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554882560 00000100:00100000:0.0:1789760794.050969:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737278198211 time 737286608894 state 0x0 ref 1 00000100:00100000:0.0:1789760794.050994:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737286869310 time 737286608894 state 0x1 ref 1 00000100:00100000:0.0:1789760794.051026:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737286869310 time 737296644648 state 0x0 ref 1 00000100:00100000:0.0:1789760794.051039:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05b8e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554882560:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760794.051083:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67665920, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20458 00010000:00010000:0.0:1789760794.051093:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c6dee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.051109:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760794.051113:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.051119:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c6dee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67665920->67670015] (req 67665920->67670015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.051132:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c6dee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67665920->67670015] (req 67665920->67670015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.054998:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983829 is committed 00002000:00010000:0.0:1789760794.055164:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67665920 -> 67670016 00002000:00010000:0.0:1789760794.055170:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132184 -> 132192 00010000:00010000:0.0:1789760794.055184:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c6dee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67665920->67670015] (req 67665920->67670015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.055201:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c6dee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67665920->67670015] (req 67665920->67670015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.055210:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c6dee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67665920->67670015] (req 67665920->67670015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.055217:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760794.055222:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c6dee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67665920->67670015] (req 67665920->67670015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.055232:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c6dee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67665920->67670015] (req 67665920->67670015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.055253:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c6dee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67665920->67670015] (req 67665920->67670015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.055261:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c6dee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67665920->67670015] (req 67665920->67670015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.055682:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05b8e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554882560:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4640us (4874us total) trans 4294983829 rc 0/0 00000100:00100000:0.0:1789760794.055693:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16531 00000100:00100000:0.0:1789760794.055773:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737286869310 time 737296644648 state 0x2 ref 0 00000100:00100000:2.0:1789760794.058322:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760794.058835:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554882688 00000100:00100000:1.0:1789760794.058846:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554882688 00000100:00100000:1.0:1789760794.058903:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737286869310 time 737296644648 state 0x0 ref 1 00000100:00100000:1.0:1789760794.058919:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737296905064 time 737296644648 state 0x1 ref 1 00000100:00100000:1.0:1789760794.058956:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737296905064 time 737304573972 state 0x0 ref 1 00000100:00100000:1.0:1789760794.058969:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05b8d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554882688:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760794.059018:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67670016, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20460 00010000:00010000:1.0:1789760794.059027:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6df5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.059043:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760794.059048:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.059055:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6df5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67670016->67674111] (req 67670016->67674111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.059069:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6df5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67670016->67674111] (req 67670016->67674111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.063332:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983830 is committed 00002000:00010000:1.0:1789760794.063671:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67670016 -> 67674112 00002000:00010000:1.0:1789760794.063677:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132192 -> 132200 00010000:00010000:1.0:1789760794.063690:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6df5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67670016->67674111] (req 67670016->67674111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.063703:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6df5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67670016->67674111] (req 67670016->67674111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.063712:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6df5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67670016->67674111] (req 67670016->67674111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.063719:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.063723:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6df5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67670016->67674111] (req 67670016->67674111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.063732:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6df5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67670016->67674111] (req 67670016->67674111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.063749:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6df5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67670016->67674111] (req 67670016->67674111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.063756:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6df5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67670016->67674111] (req 67670016->67674111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.063874:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05b8d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554882688:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4903us (5556us total) trans 4294983830 rc 0/0 00000100:00100000:1.0:1789760794.063886:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16532 00000100:00100000:1.0:1789760794.063895:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737296905064 time 737304573972 state 0x2 ref 0 00000100:00100000:1.0:1789760794.068263:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760794.068555:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554882816 00000100:00100000:2.0:1789760794.068565:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554882816 00000100:00100000:2.0:1789760794.068627:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737296905064 time 737304573972 state 0x0 ref 1 00000100:00100000:2.0:1789760794.068646:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737304834388 time 737304573972 state 0x1 ref 1 00000100:00100000:2.0:1789760794.068724:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737304834388 time 737314341376 state 0x0 ref 1 00000100:00100000:2.0:1789760794.068738:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554882816:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760794.068785:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67674112, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20468 00010000:00010000:2.0:1789760794.068794:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6dfc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.068810:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760794.068814:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.068820:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6dfc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67674112->67678207] (req 67674112->67678207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.068833:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6dfc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67674112->67678207] (req 67674112->67678207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.072909:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983831 is committed 00002000:00010000:2.0:1789760794.073323:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67674112 -> 67678208 00002000:00010000:2.0:1789760794.073328:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132200 -> 132208 00010000:00010000:2.0:1789760794.073340:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6dfc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67674112->67678207] (req 67674112->67678207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.073487:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6dfc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67674112->67678207] (req 67674112->67678207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.073498:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6dfc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67674112->67678207] (req 67674112->67678207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.073506:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760794.073512:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6dfc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67674112->67678207] (req 67674112->67678207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.073521:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6dfc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67674112->67678207] (req 67674112->67678207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.073540:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6dfc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67674112->67678207] (req 67674112->67678207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.073549:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c6dfc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67674112->67678207] (req 67674112->67678207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.073737:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554882816:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4996us (5479us total) trans 4294983831 rc 0/0 00000100:00100000:2.0:1789760794.073747:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16533 00000100:00100000:2.0:1789760794.073757:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737304834388 time 737314341376 state 0x2 ref 0 00000100:00100000:2.0:1789760794.089660:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760794.092086:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554882944 00000100:00100000:1.0:1789760794.092110:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554882944 00000100:00100000:1.0:1789760794.092173:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737304834388 time 737314341376 state 0x0 ref 1 00000100:00100000:1.0:1789760794.092189:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737314601792 time 737314341376 state 0x1 ref 1 00000100:00100000:1.0:1789760794.092224:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737314601792 time 737337842550 state 0x0 ref 1 00000100:00100000:1.0:1789760794.092237:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05b8c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554882944:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760794.092281:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67678208, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20470 00010000:00010000:1.0:1789760794.092290:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e03 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.092305:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760794.092310:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.092316:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67678208->67682303] (req 67678208->67682303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.092331:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67678208->67682303] (req 67678208->67682303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.096352:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983832 is committed 00002000:00010000:1.0:1789760794.096479:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67678208 -> 67682304 00002000:00010000:1.0:1789760794.096484:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132208 -> 132216 00010000:00010000:1.0:1789760794.096496:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e03 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67678208->67682303] (req 67678208->67682303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.096509:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e03 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67678208->67682303] (req 67678208->67682303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.096518:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e03 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67678208->67682303] (req 67678208->67682303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.096526:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.096531:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e03 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67678208->67682303] (req 67678208->67682303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.096540:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e03 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67678208->67682303] (req 67678208->67682303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.096560:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e03 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67678208->67682303] (req 67678208->67682303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.096569:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e03 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67678208->67682303] (req 67678208->67682303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.096670:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05b8c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554882944:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4430us (7015us total) trans 4294983832 rc 0/0 00000100:00100000:1.0:1789760794.096680:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16534 00000100:00100000:1.0:1789760794.096689:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737314601792 time 737337842550 state 0x2 ref 0 00000100:00100000:2.0:1789760794.102016:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760794.102529:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554883072 00000100:00100000:0.0:1789760794.102541:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554883072 00000100:00100000:0.0:1789760794.102613:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737314601792 time 737337842550 state 0x0 ref 1 00000100:00100000:0.0:1789760794.102630:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737338102966 time 737337842550 state 0x1 ref 1 00000100:00100000:0.0:1789760794.102702:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737338102966 time 737348318987 state 0x0 ref 1 00000100:00100000:0.0:1789760794.102716:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05b8ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554883072:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760794.102763:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67682304, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20478 00010000:00010000:0.0:1789760794.102772:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6e0a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.102788:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760794.102792:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.102799:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6e0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67682304->67686399] (req 67682304->67686399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.102813:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6e0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67682304->67686399] (req 67682304->67686399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.106562:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983833 is committed 00002000:00010000:0.0:1789760794.106832:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67682304 -> 67686400 00002000:00010000:0.0:1789760794.106837:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132216 -> 132224 00010000:00010000:0.0:1789760794.106849:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6e0a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67682304->67686399] (req 67682304->67686399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.106861:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6e0a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67682304->67686399] (req 67682304->67686399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.106876:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6e0a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67682304->67686399] (req 67682304->67686399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.106884:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760794.106889:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6e0a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67682304->67686399] (req 67682304->67686399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.106898:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6e0a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67682304->67686399] (req 67682304->67686399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.106916:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6e0a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67682304->67686399] (req 67682304->67686399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.106924:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c6e0a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67682304->67686399] (req 67682304->67686399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.107217:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05b8ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554883072:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4497us (5203us total) trans 4294983833 rc 0/0 00000100:00100000:0.0:1789760794.107228:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16535 00000100:00100000:0.0:1789760794.107237:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737338102966 time 737348318987 state 0x2 ref 0 00000100:00100000:0.0:1789760794.112297:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760794.112424:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554883200 00000100:00100000:2.0:1789760794.112434:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554883200 00000100:00100000:2.0:1789760794.112489:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737338102966 time 737348318987 state 0x0 ref 1 00000100:00100000:2.0:1789760794.112508:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737348579403 time 737348318987 state 0x1 ref 1 00000100:00100000:2.0:1789760794.112541:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737348579403 time 737358159314 state 0x0 ref 1 00000100:00100000:2.0:1789760794.112552:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554883200:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760794.112606:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67686400, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20480 00010000:00010000:2.0:1789760794.112615:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6e11 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.112632:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760794.112637:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.112644:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6e11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67686400->67690495] (req 67686400->67690495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.112658:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6e11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67686400->67690495] (req 67686400->67690495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.115901:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983834 is committed 00002000:00010000:1.0:1789760794.116114:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67686400 -> 67690496 00002000:00010000:1.0:1789760794.116119:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132224 -> 132232 00010000:00010000:1.0:1789760794.116132:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6e11 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67686400->67690495] (req 67686400->67690495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.116145:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6e11 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67686400->67690495] (req 67686400->67690495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.116154:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6e11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67686400->67690495] (req 67686400->67690495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.116161:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.116166:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6e11 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67686400->67690495] (req 67686400->67690495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.116176:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6e11 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67686400->67690495] (req 67686400->67690495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.116194:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6e11 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67686400->67690495] (req 67686400->67690495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.116202:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6e11 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67686400->67690495] (req 67686400->67690495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.116564:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554883200:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4009us (4271us total) trans 4294983834 rc 0/0 00000100:00100000:1.0:1789760794.116578:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16536 00000100:00100000:1.0:1789760794.116587:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737348579403 time 737358159314 state 0x2 ref 0 00000100:00100000:2.0:1789760794.120838:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760794.120900:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554883328 00000100:00100000:1.0:1789760794.120919:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554883328 00000100:00100000:1.0:1789760794.120973:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737348579403 time 737358159314 state 0x0 ref 1 00000100:00100000:1.0:1789760794.120988:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737358419730 time 737358159314 state 0x1 ref 1 00000100:00100000:1.0:1789760794.121073:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737358419730 time 737366684240 state 0x0 ref 1 00000100:00100000:1.0:1789760794.121082:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05b8fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554883328:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760794.121121:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67690496, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20488 00010000:00010000:1.0:1789760794.121128:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e18 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.121140:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760794.121144:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.121150:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67690496->67694591] (req 67690496->67694591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.121163:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67690496->67694591] (req 67690496->67694591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.125552:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983835 is committed 00002000:00010000:1.0:1789760794.126422:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67690496 -> 67694592 00002000:00010000:1.0:1789760794.126427:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132232 -> 132240 00010000:00010000:1.0:1789760794.126440:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e18 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67690496->67694591] (req 67690496->67694591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.126453:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e18 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67690496->67694591] (req 67690496->67694591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.126471:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e18 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67690496->67694591] (req 67690496->67694591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.126480:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.126484:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e18 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67690496->67694591] (req 67690496->67694591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.126493:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e18 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67690496->67694591] (req 67690496->67694591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.126512:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e18 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67690496->67694591] (req 67690496->67694591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.126520:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e18 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67690496->67694591] (req 67690496->67694591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.126614:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05b8fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554883328:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5527us (5780us total) trans 4294983835 rc 0/0 00000100:00100000:1.0:1789760794.126625:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16537 00000100:00100000:1.0:1789760794.126634:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737358419730 time 737366684240 state 0x2 ref 0 00000100:00100000:1.0:1789760794.134874:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760794.134936:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554883456 00000100:00100000:0.0:1789760794.134951:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554883456 00000100:00100000:0.0:1789760794.135004:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737358419730 time 737366684240 state 0x0 ref 1 00000100:00100000:0.0:1789760794.135023:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737366944656 time 737366684240 state 0x1 ref 1 00000100:00100000:0.0:1789760794.135055:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737366944656 time 737380673601 state 0x0 ref 1 00000100:00100000:0.0:1789760794.135201:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554883456:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760794.135248:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67694592, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20490 00010000:00010000:0.0:1789760794.135257:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6e1f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.135273:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760794.135278:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.135284:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6e1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67694592->67698687] (req 67694592->67698687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.135297:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6e1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67694592->67698687] (req 67694592->67698687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.139790:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983836 is committed 00002000:00010000:1.0:1789760794.139981:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67694592 -> 67698688 00002000:00010000:1.0:1789760794.139986:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132240 -> 132248 00010000:00010000:1.0:1789760794.139998:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6e1f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67694592->67698687] (req 67694592->67698687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.140011:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6e1f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67694592->67698687] (req 67694592->67698687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.140020:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6e1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67694592->67698687] (req 67694592->67698687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.140027:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.140031:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6e1f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67694592->67698687] (req 67694592->67698687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.140040:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6e1f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67694592->67698687] (req 67694592->67698687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.140056:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6e1f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67694592->67698687] (req 67694592->67698687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.140071:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c6e1f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67694592->67698687] (req 67694592->67698687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.140168:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554883456:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5097us (5298us total) trans 4294983836 rc 0/0 00000100:00100000:1.0:1789760794.140179:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16538 00000100:00100000:1.0:1789760794.140187:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737366944656 time 737380673601 state 0x2 ref 0 00000100:00100000:0.0:1789760794.146107:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760794.146323:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554883584 00000100:00100000:1.0:1789760794.146334:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554883584 00000100:00100000:1.0:1789760794.146422:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737366944656 time 737380673601 state 0x0 ref 1 00000100:00100000:1.0:1789760794.146437:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737380934017 time 737380673601 state 0x1 ref 1 00000100:00100000:1.0:1789760794.146468:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737380934017 time 737392086131 state 0x0 ref 1 00000100:00100000:1.0:1789760794.146480:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a2f0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554883584:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760794.146524:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67698688, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20498 00010000:00010000:1.0:1789760794.146542:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e26 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.146558:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760794.146562:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.146569:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67698688->67702783] (req 67698688->67702783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.146584:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67698688->67702783] (req 67698688->67702783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.152024:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983837 is committed 00002000:00010000:1.0:1789760794.152343:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67698688 -> 67702784 00002000:00010000:1.0:1789760794.152448:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132248 -> 132256 00010000:00010000:1.0:1789760794.152463:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e26 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67698688->67702783] (req 67698688->67702783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.152475:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e26 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67698688->67702783] (req 67698688->67702783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.152484:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67698688->67702783] (req 67698688->67702783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.152491:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.152495:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e26 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67698688->67702783] (req 67698688->67702783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.152503:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e26 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67698688->67702783] (req 67698688->67702783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.152521:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e26 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67698688->67702783] (req 67698688->67702783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.152882:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e26 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67698688->67702783] (req 67698688->67702783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.153028:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a2f0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554883584:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6545us (6926us total) trans 4294983837 rc 0/0 00000100:00100000:1.0:1789760794.153038:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16539 00000100:00100000:1.0:1789760794.153047:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737380934017 time 737392086131 state 0x2 ref 0 00000100:00100000:2.0:1789760794.159949:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760794.160253:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554883712 00000100:00100000:1.0:1789760794.160265:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554883712 00000100:00100000:1.0:1789760794.160334:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737380934017 time 737392086131 state 0x0 ref 1 00000100:00100000:1.0:1789760794.160358:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737392346547 time 737392086131 state 0x1 ref 1 00000100:00100000:1.0:1789760794.160419:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737392346547 time 737406037675 state 0x0 ref 1 00000100:00100000:1.0:1789760794.160432:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05b8f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554883712:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760794.160484:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67702784, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x204a0 00010000:00010000:1.0:1789760794.160493:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e2d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.160509:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760794.160513:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.160520:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67702784->67706879] (req 67702784->67706879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.160695:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67702784->67706879] (req 67702784->67706879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.164808:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983838 is committed 00002000:00010000:1.0:1789760794.164930:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67702784 -> 67706880 00002000:00010000:1.0:1789760794.164937:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132256 -> 132264 00010000:00010000:1.0:1789760794.164958:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e2d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67702784->67706879] (req 67702784->67706879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.164984:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e2d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67702784->67706879] (req 67702784->67706879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.164993:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67702784->67706879] (req 67702784->67706879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.165000:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.165004:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e2d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67702784->67706879] (req 67702784->67706879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.165014:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e2d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67702784->67706879] (req 67702784->67706879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.165034:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e2d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67702784->67706879] (req 67702784->67706879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.165041:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e2d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67702784->67706879] (req 67702784->67706879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.165137:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05b8f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554883712:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4701us (5191us total) trans 4294983838 rc 0/0 00000100:00100000:1.0:1789760794.165147:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16540 00000100:00100000:1.0:1789760794.165156:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737392346547 time 737406037675 state 0x2 ref 0 00000100:00100000:3.0:1789760794.171754:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760794.171858:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554883840 00000100:00100000:2.0:1789760794.171868:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554883840 00000100:00100000:2.0:1789760794.171929:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737392346547 time 737406037675 state 0x0 ref 1 00000100:00100000:2.0:1789760794.171954:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737406298091 time 737406037675 state 0x1 ref 1 00000100:00100000:2.0:1789760794.171991:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737406298091 time 737417609564 state 0x0 ref 1 00000100:00100000:2.0:1789760794.172003:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554883840:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760794.172051:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67706880, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x204a8 00010000:00010000:2.0:1789760794.172060:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e34 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.172076:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760794.172080:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.172086:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67706880->67710975] (req 67706880->67710975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.172099:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67706880->67710975] (req 67706880->67710975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.176675:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983839 is committed 00002000:00010000:2.0:1789760794.179584:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67706880 -> 67710976 00002000:00010000:2.0:1789760794.179605:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132264 -> 132272 00010000:00010000:2.0:1789760794.179619:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e34 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67706880->67710975] (req 67706880->67710975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.179633:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e34 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67706880->67710975] (req 67706880->67710975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.179643:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67706880->67710975] (req 67706880->67710975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.179655:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760794.179661:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e34 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67706880->67710975] (req 67706880->67710975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.179669:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e34 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67706880->67710975] (req 67706880->67710975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.179687:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e34 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67706880->67710975] (req 67706880->67710975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.179703:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e34 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67706880->67710975] (req 67706880->67710975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.179818:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554883840:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7812us (8069us total) trans 4294983839 rc 0/0 00000100:00100000:2.0:1789760794.179829:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16541 00000100:00100000:2.0:1789760794.179839:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737406298091 time 737417609564 state 0x2 ref 0 00000100:00100000:0.0:1789760794.183314:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760794.183663:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554883968 00000100:00100000:1.0:1789760794.183674:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554883968 00000100:00100000:1.0:1789760794.183727:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737406298091 time 737417609564 state 0x0 ref 1 00000100:00100000:1.0:1789760794.183741:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737417869980 time 737417609564 state 0x1 ref 1 00000100:00100000:1.0:1789760794.183803:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737417869980 time 737429420776 state 0x0 ref 1 00000100:00100000:1.0:1789760794.183816:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554883968:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760794.183864:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67710976, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x204b0 00010000:00010000:1.0:1789760794.183872:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e3b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.183889:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760794.183893:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.183898:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67710976->67715071] (req 67710976->67715071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.183910:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67710976->67715071] (req 67710976->67715071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.195010:0:6509:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789760794.202024:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983840 is committed 00002000:00010000:0.0:1789760794.202448:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67710976 -> 67715072 00002000:00010000:0.0:1789760794.202454:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132272 -> 132280 00010000:00010000:0.0:1789760794.202469:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e3b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67710976->67715071] (req 67710976->67715071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.202482:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e3b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67710976->67715071] (req 67710976->67715071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.202492:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e3b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67710976->67715071] (req 67710976->67715071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.202509:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760794.202515:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e3b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67710976->67715071] (req 67710976->67715071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.202524:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e3b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67710976->67715071] (req 67710976->67715071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.202543:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e3b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67710976->67715071] (req 67710976->67715071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.202551:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c6e3b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67710976->67715071] (req 67710976->67715071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.202653:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554883968:12345-192.168.203.22@tcp:4:dd.0 Request processed in 18834us (19345us total) trans 4294983840 rc 0/0 00000100:00100000:0.0:1789760794.202664:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16542 00000100:00100000:0.0:1789760794.202674:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737417869980 time 737429420776 state 0x2 ref 0 00000100:00100000:3.0:1789760794.214041:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760794.216350:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554884096 00000100:00100000:1.0:1789760794.216364:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554884096 00000100:00100000:1.0:1789760794.216439:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737417869980 time 737429420776 state 0x0 ref 1 00000100:00100000:1.0:1789760794.216453:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737429681192 time 737429420776 state 0x1 ref 1 00000100:00100000:1.0:1789760794.216490:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737429681192 time 737462108650 state 0x0 ref 1 00000100:00100000:1.0:1789760794.216506:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554884096:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760794.216555:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67715072, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x204b8 00010000:00010000:1.0:1789760794.216566:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6e42 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.216582:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760794.216586:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.216604:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6e42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67715072->67719167] (req 67715072->67719167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.216619:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6e42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67715072->67719167] (req 67715072->67719167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.224425:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983841 is committed 00002000:00010000:1.0:1789760794.224670:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67715072 -> 67719168 00002000:00010000:1.0:1789760794.224675:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132280 -> 132288 00010000:00010000:1.0:1789760794.224696:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6e42 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67715072->67719167] (req 67715072->67719167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.224709:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6e42 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67715072->67719167] (req 67715072->67719167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.224718:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6e42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67715072->67719167] (req 67715072->67719167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.224726:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.224735:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6e42 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67715072->67719167] (req 67715072->67719167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.224744:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6e42 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67715072->67719167] (req 67715072->67719167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.224764:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6e42 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67715072->67719167] (req 67715072->67719167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.224772:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6e42 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67715072->67719167] (req 67715072->67719167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.224860:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554884096:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8352us (10824us total) trans 4294983841 rc 0/0 00000100:00100000:1.0:1789760794.224871:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16543 00000100:00100000:1.0:1789760794.224880:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737429681192 time 737462108650 state 0x2 ref 0 00000100:00100000:3.0:1789760794.230723:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760794.230877:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554884224 00000100:00100000:2.0:1789760794.230886:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554884224 00000100:00100000:2.0:1789760794.230932:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737429681192 time 737462108650 state 0x0 ref 1 00000100:00100000:2.0:1789760794.230945:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737462369066 time 737462108650 state 0x1 ref 1 00000100:00100000:2.0:1789760794.230980:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737462369066 time 737476598301 state 0x0 ref 1 00000100:00100000:2.0:1789760794.230995:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554884224:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760794.231038:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67719168, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x204c0 00010000:00010000:2.0:1789760794.231046:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e49 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.231062:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760794.231066:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.231072:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67719168->67723263] (req 67719168->67723263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.231084:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67719168->67723263] (req 67719168->67723263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.241564:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983842 is committed 00002000:00010000:2.0:1789760794.241771:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67719168 -> 67723264 00002000:00010000:2.0:1789760794.241777:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132288 -> 132296 00010000:00010000:2.0:1789760794.241789:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e49 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67719168->67723263] (req 67719168->67723263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.241803:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e49 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67719168->67723263] (req 67719168->67723263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.241812:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67719168->67723263] (req 67719168->67723263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.241819:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760794.241825:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e49 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67719168->67723263] (req 67719168->67723263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.241835:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e49 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67719168->67723263] (req 67719168->67723263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.241854:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e49 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67719168->67723263] (req 67719168->67723263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.241862:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e49 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67719168->67723263] (req 67719168->67723263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.242125:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554884224:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11126us (11405us total) trans 4294983842 rc 0/0 00000100:00100000:2.0:1789760794.242137:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16544 00000100:00100000:2.0:1789760794.242145:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737462369066 time 737476598301 state 0x2 ref 0 00000100:00100000:0.0:1789760794.245875:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760794.246178:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554884352 00000100:00100000:2.0:1789760794.246188:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554884352 00000100:00100000:2.0:1789760794.246237:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737462369066 time 737476598301 state 0x0 ref 1 00000100:00100000:2.0:1789760794.246252:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737476858717 time 737476598301 state 0x1 ref 1 00000100:00100000:2.0:1789760794.246283:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737476858717 time 737491901848 state 0x0 ref 1 00000100:00100000:2.0:1789760794.246296:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554884352:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760794.246342:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67723264, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x204c8 00010000:00010000:2.0:1789760794.246352:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6e50 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.246367:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760794.246372:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.246412:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6e50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67723264->67727359] (req 67723264->67727359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.246427:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6e50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67723264->67727359] (req 67723264->67727359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.253070:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983843 is committed 00002000:00010000:2.0:1789760794.256764:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67723264 -> 67727360 00002000:00010000:2.0:1789760794.256774:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132296 -> 132304 00010000:00010000:2.0:1789760794.256788:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6e50 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67723264->67727359] (req 67723264->67727359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.256801:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6e50 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67723264->67727359] (req 67723264->67727359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.256812:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6e50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67723264->67727359] (req 67723264->67727359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.256819:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760794.256824:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6e50 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67723264->67727359] (req 67723264->67727359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.256833:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6e50 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67723264->67727359] (req 67723264->67727359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.256854:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6e50 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67723264->67727359] (req 67723264->67727359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.256862:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6e50 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67723264->67727359] (req 67723264->67727359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.256959:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554884352:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10661us (11088us total) trans 4294983843 rc 0/0 00000100:00100000:2.0:1789760794.256969:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16545 00000100:00100000:2.0:1789760794.256978:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737476858717 time 737491901848 state 0x2 ref 0 00000100:00100000:3.0:1789760794.265954:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760794.266269:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554884480 00000100:00100000:0.0:1789760794.266278:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554884480 00000100:00100000:0.0:1789760794.266330:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737476858717 time 737491901848 state 0x0 ref 1 00000100:00100000:0.0:1789760794.266345:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737492162264 time 737491901848 state 0x1 ref 1 00000100:00100000:0.0:1789760794.266503:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737492162264 time 737512120403 state 0x0 ref 1 00000100:00100000:0.0:1789760794.266517:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554884480:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760794.266563:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67727360, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x204d0 00010000:00010000:0.0:1789760794.266574:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6e57 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.266601:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760794.266606:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.266613:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6e57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67727360->67731455] (req 67727360->67731455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.266628:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6e57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67727360->67731455] (req 67727360->67731455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.274339:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983844 is committed 00002000:00010000:0.0:1789760794.276795:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67727360 -> 67731456 00002000:00010000:0.0:1789760794.276804:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132304 -> 132312 00010000:00010000:0.0:1789760794.276817:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6e57 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67727360->67731455] (req 67727360->67731455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.276836:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6e57 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67727360->67731455] (req 67727360->67731455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.276844:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6e57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67727360->67731455] (req 67727360->67731455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.276851:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760794.276855:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6e57 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67727360->67731455] (req 67727360->67731455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.276863:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6e57 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67727360->67731455] (req 67727360->67731455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.276883:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6e57 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67727360->67731455] (req 67727360->67731455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.276890:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6e57 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67727360->67731455] (req 67727360->67731455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.276995:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554884480:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10475us (11045us total) trans 4294983844 rc 0/0 00000100:00100000:0.0:1789760794.277006:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16546 00000100:00100000:0.0:1789760794.277015:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737492162264 time 737512120403 state 0x2 ref 0 00000100:00100000:3.0:1789760794.283761:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760794.283872:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554884608 00000100:00100000:1.0:1789760794.283881:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554884608 00000100:00100000:1.0:1789760794.283935:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737492162264 time 737512120403 state 0x0 ref 1 00000100:00100000:1.0:1789760794.283950:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737512380819 time 737512120403 state 0x1 ref 1 00000100:00100000:1.0:1789760794.283990:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737512380819 time 737529608565 state 0x0 ref 1 00000100:00100000:1.0:1789760794.284003:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554884608:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760794.284053:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67731456, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x204d8 00010000:00010000:1.0:1789760794.284063:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6e5e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.284080:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760794.284085:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.284091:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6e5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67731456->67735551] (req 67731456->67735551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.284104:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6e5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67731456->67735551] (req 67731456->67735551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.290767:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983845 is committed 00002000:00010000:1.0:1789760794.290877:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67731456 -> 67735552 00002000:00010000:1.0:1789760794.290882:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132312 -> 132320 00010000:00010000:1.0:1789760794.290896:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6e5e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67731456->67735551] (req 67731456->67735551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.290910:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6e5e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67731456->67735551] (req 67731456->67735551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.290921:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6e5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67731456->67735551] (req 67731456->67735551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.290929:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.290934:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6e5e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67731456->67735551] (req 67731456->67735551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.290944:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6e5e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67731456->67735551] (req 67731456->67735551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.291026:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6e5e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67731456->67735551] (req 67731456->67735551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.291036:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6e5e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67731456->67735551] (req 67731456->67735551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.291175:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554884608:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7168us (7417us total) trans 4294983845 rc 0/0 00000100:00100000:1.0:1789760794.291186:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16547 00000100:00100000:1.0:1789760794.291196:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737512380819 time 737529608565 state 0x2 ref 0 00000100:00100000:1.0:1789760794.295197:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760794.295910:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554884736 00000100:00100000:2.0:1789760794.295920:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554884736 00000100:00100000:2.0:1789760794.295973:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737512380819 time 737529608565 state 0x0 ref 1 00000100:00100000:2.0:1789760794.295989:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737529868981 time 737529608565 state 0x1 ref 1 00000100:00100000:2.0:1789760794.296017:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737529868981 time 737541635418 state 0x0 ref 1 00000100:00100000:2.0:1789760794.296030:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554884736:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760794.296075:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67735552, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x204e0 00010000:00010000:2.0:1789760794.296084:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6e65 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.296099:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760794.296104:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.296111:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6e65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67735552->67739647] (req 67735552->67739647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.296124:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6e65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67735552->67739647] (req 67735552->67739647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.303113:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983846 is committed 00002000:00010000:1.0:1789760794.303295:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67735552 -> 67739648 00002000:00010000:1.0:1789760794.303301:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132320 -> 132328 00010000:00010000:1.0:1789760794.303315:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6e65 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67735552->67739647] (req 67735552->67739647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.303328:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6e65 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67735552->67739647] (req 67735552->67739647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.303338:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6e65 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67735552->67739647] (req 67735552->67739647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.303346:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.303350:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6e65 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67735552->67739647] (req 67735552->67739647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.303359:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6e65 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67735552->67739647] (req 67735552->67739647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.303408:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6e65 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67735552->67739647] (req 67735552->67739647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.303417:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6e65 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67735552->67739647] (req 67735552->67739647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.303579:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554884736:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7547us (8386us total) trans 4294983846 rc 0/0 00000100:00100000:1.0:1789760794.303599:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16548 00000100:00100000:1.0:1789760794.303609:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737529868981 time 737541635418 state 0x2 ref 0 00000100:00100000:3.0:1789760794.308229:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760794.308512:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554884864 00000100:00100000:2.0:1789760794.308523:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554884864 00000100:00100000:2.0:1789760794.308573:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737529868981 time 737541635418 state 0x0 ref 1 00000100:00100000:2.0:1789760794.308588:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737541895834 time 737541635418 state 0x1 ref 1 00000100:00100000:2.0:1789760794.308621:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737541895834 time 737554239918 state 0x0 ref 1 00000100:00100000:2.0:1789760794.308693:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554884864:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760794.308752:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67739648, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x204e8 00010000:00010000:2.0:1789760794.308761:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e6c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.308777:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760794.308782:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.308788:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67739648->67743743] (req 67739648->67743743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.308801:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67739648->67743743] (req 67739648->67743743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.313709:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983847 is committed 00002000:00010000:2.0:1789760794.313819:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67739648 -> 67743744 00002000:00010000:2.0:1789760794.313825:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132328 -> 132336 00010000:00010000:2.0:1789760794.313841:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e6c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67739648->67743743] (req 67739648->67743743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.313853:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e6c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67739648->67743743] (req 67739648->67743743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.313862:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e6c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67739648->67743743] (req 67739648->67743743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.313869:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760794.313874:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e6c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67739648->67743743] (req 67739648->67743743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.313882:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e6c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67739648->67743743] (req 67739648->67743743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.313900:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e6c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67739648->67743743] (req 67739648->67743743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.313907:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e6c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67739648->67743743] (req 67739648->67743743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.314032:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554884864:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5396us (5806us total) trans 4294983847 rc 0/0 00000100:00100000:2.0:1789760794.314043:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16549 00000100:00100000:2.0:1789760794.314052:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737541895834 time 737554239918 state 0x2 ref 0 00000100:00100000:3.0:1789760794.317782:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760794.317996:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554884992 00000100:00100000:0.0:1789760794.318004:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554884992 00000100:00100000:0.0:1789760794.318054:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737541895834 time 737554239918 state 0x0 ref 1 00000100:00100000:0.0:1789760794.318069:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737554500334 time 737554239918 state 0x1 ref 1 00000100:00100000:0.0:1789760794.318141:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737554500334 time 737563759572 state 0x0 ref 1 00000100:00100000:0.0:1789760794.318153:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554884992:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760794.318199:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67743744, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x204f0 00010000:00010000:0.0:1789760794.318207:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6e73 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.318222:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760794.318226:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.318231:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6e73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67743744->67747839] (req 67743744->67747839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.318244:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6e73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67743744->67747839] (req 67743744->67747839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.324634:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983848 is committed 00002000:00010000:0.0:1789760794.324756:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67743744 -> 67747840 00002000:00010000:0.0:1789760794.324761:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132336 -> 132344 00010000:00010000:0.0:1789760794.324774:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6e73 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67743744->67747839] (req 67743744->67747839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.324794:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6e73 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67743744->67747839] (req 67743744->67747839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.324804:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6e73 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67743744->67747839] (req 67743744->67747839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.324811:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760794.324815:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6e73 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67743744->67747839] (req 67743744->67747839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.324824:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6e73 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67743744->67747839] (req 67743744->67747839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.324844:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6e73 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67743744->67747839] (req 67743744->67747839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.324852:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c6e73 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67743744->67747839] (req 67743744->67747839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.324944:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554884992:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6789us (7166us total) trans 4294983848 rc 0/0 00000100:00100000:0.0:1789760794.324955:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16550 00000100:00100000:0.0:1789760794.324972:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737554500334 time 737563759572 state 0x2 ref 0 00000100:00100000:1.0:1789760794.328583:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760794.328684:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554885120 00000100:00100000:0.0:1789760794.328694:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554885120 00000100:00100000:0.0:1789760794.328745:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737554500334 time 737563759572 state 0x0 ref 1 00000100:00100000:0.0:1789760794.328759:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737564019988 time 737563759572 state 0x1 ref 1 00000100:00100000:0.0:1789760794.328794:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737564019988 time 737574412051 state 0x0 ref 1 00000100:00100000:0.0:1789760794.328806:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554885120:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760794.328859:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67747840, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x204f8 00010000:00010000:0.0:1789760794.328868:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6e7a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.328882:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760794.328886:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.328892:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6e7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67747840->67751935] (req 67747840->67751935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.328905:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6e7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67747840->67751935] (req 67747840->67751935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.335791:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983849 is committed 00002000:00010000:1.0:1789760794.336104:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67747840 -> 67751936 00002000:00010000:1.0:1789760794.336110:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132344 -> 132352 00010000:00010000:1.0:1789760794.336123:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6e7a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67747840->67751935] (req 67747840->67751935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.336136:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6e7a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67747840->67751935] (req 67747840->67751935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.336145:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6e7a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67747840->67751935] (req 67747840->67751935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.336159:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.336164:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6e7a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67747840->67751935] (req 67747840->67751935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.336173:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6e7a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67747840->67751935] (req 67747840->67751935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.336191:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6e7a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67747840->67751935] (req 67747840->67751935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.336199:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c6e7a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67747840->67751935] (req 67747840->67751935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.336494:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554885120:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7683us (7913us total) trans 4294983849 rc 0/0 00000100:00100000:1.0:1789760794.336505:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16551 00000100:00100000:1.0:1789760794.336515:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737564019988 time 737574412051 state 0x2 ref 0 00000100:00100000:3.0:1789760794.343930:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760794.343995:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554885248 00000100:00100000:1.0:1789760794.344005:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554885248 00000100:00100000:1.0:1789760794.344067:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737564019988 time 737574412051 state 0x0 ref 1 00000100:00100000:1.0:1789760794.344082:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737574672467 time 737574412051 state 0x1 ref 1 00000100:00100000:1.0:1789760794.344342:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737574672467 time 737589959103 state 0x0 ref 1 00000100:00100000:1.0:1789760794.344360:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554885248:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760794.344443:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67751936, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20500 00010000:00010000:1.0:1789760794.344452:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6e81 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.344478:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760794.344483:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.344489:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6e81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67751936->67756031] (req 67751936->67756031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.344503:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6e81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67751936->67756031] (req 67751936->67756031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.352306:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983850 is committed 00002000:00010000:1.0:1789760794.353037:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67751936 -> 67756032 00002000:00010000:1.0:1789760794.353044:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132352 -> 132360 00010000:00010000:1.0:1789760794.353058:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6e81 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67751936->67756031] (req 67751936->67756031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.353071:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6e81 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67751936->67756031] (req 67751936->67756031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.353081:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6e81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67751936->67756031] (req 67751936->67756031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.353088:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.353094:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6e81 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67751936->67756031] (req 67751936->67756031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.353104:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6e81 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67751936->67756031] (req 67751936->67756031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.353126:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6e81 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67751936->67756031] (req 67751936->67756031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.353134:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c6e81 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67751936->67756031] (req 67751936->67756031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.353244:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554885248:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8882us (9318us total) trans 4294983850 rc 0/0 00000100:00100000:1.0:1789760794.353255:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16552 00000100:00100000:1.0:1789760794.353265:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737574672467 time 737589959103 state 0x2 ref 0 00000100:00100000:3.0:1789760794.357165:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760794.357268:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554885376 00000100:00100000:2.0:1789760794.357278:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554885376 00000100:00100000:2.0:1789760794.357333:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737574672467 time 737589959103 state 0x0 ref 1 00000100:00100000:2.0:1789760794.357349:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737590219519 time 737589959103 state 0x1 ref 1 00000100:00100000:2.0:1789760794.357405:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737590219519 time 737603022757 state 0x0 ref 1 00000100:00100000:2.0:1789760794.357418:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554885376:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760794.357465:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67756032, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20508 00010000:00010000:2.0:1789760794.357474:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e88 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.357491:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760794.357495:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.357502:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67756032->67760127] (req 67756032->67760127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.357515:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67756032->67760127] (req 67756032->67760127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.363465:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983851 is committed 00002000:00010000:2.0:1789760794.363563:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67756032 -> 67760128 00002000:00010000:2.0:1789760794.363568:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132360 -> 132368 00010000:00010000:2.0:1789760794.363580:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e88 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67756032->67760127] (req 67756032->67760127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.363603:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e88 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67756032->67760127] (req 67756032->67760127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.363612:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67756032->67760127] (req 67756032->67760127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.363620:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760794.363625:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e88 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67756032->67760127] (req 67756032->67760127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.363633:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e88 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67756032->67760127] (req 67756032->67760127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.363653:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e88 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67756032->67760127] (req 67756032->67760127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.363661:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6e88 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67756032->67760127] (req 67756032->67760127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.363745:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554885376:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6326us (6587us total) trans 4294983851 rc 0/0 00000100:00100000:2.0:1789760794.363755:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16553 00000100:00100000:2.0:1789760794.363765:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737590219519 time 737603022757 state 0x2 ref 0 00000100:00100000:0.0:1789760794.367166:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760794.367279:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554885504 00000100:00100000:1.0:1789760794.367289:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554885504 00000100:00100000:1.0:1789760794.367342:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737590219519 time 737603022757 state 0x0 ref 1 00000100:00100000:1.0:1789760794.367357:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737603283173 time 737603022757 state 0x1 ref 1 00000100:00100000:3.0:1789760794.367765:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737603283173 time 737613376943 state 0x0 ref 1 00000100:00100000:3.0:1789760794.367782:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554885504:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760794.367833:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67760128, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20510 00010000:00010000:3.0:1789760794.367842:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6e8f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.367859:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760794.367863:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.367869:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6e8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67760128->67764223] (req 67760128->67764223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.367883:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6e8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67760128->67764223] (req 67760128->67764223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.373542:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983852 is committed 00002000:00010000:3.0:1789760794.373658:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67760128 -> 67764224 00002000:00010000:3.0:1789760794.373664:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132368 -> 132376 00010000:00010000:3.0:1789760794.373677:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6e8f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67760128->67764223] (req 67760128->67764223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.373690:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6e8f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67760128->67764223] (req 67760128->67764223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.373700:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6e8f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67760128->67764223] (req 67760128->67764223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.373708:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760794.373713:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6e8f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67760128->67764223] (req 67760128->67764223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.373721:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6e8f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67760128->67764223] (req 67760128->67764223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.373745:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6e8f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67760128->67764223] (req 67760128->67764223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.373753:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6e8f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67760128->67764223] (req 67760128->67764223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.373843:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554885504:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6061us (6683us total) trans 4294983852 rc 0/0 00000100:00100000:3.0:1789760794.373854:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16554 00000100:00100000:3.0:1789760794.373864:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737603283173 time 737613376943 state 0x2 ref 0 00000100:00100000:3.0:1789760794.377570:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760794.377758:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554885632 00000100:00100000:1.0:1789760794.377767:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554885632 00000100:00100000:1.0:1789760794.377879:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737603283173 time 737613376943 state 0x0 ref 1 00000100:00100000:1.0:1789760794.377896:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737613637359 time 737613376943 state 0x1 ref 1 00000100:00100000:1.0:1789760794.377987:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737613637359 time 737623604759 state 0x0 ref 1 00000100:00100000:1.0:1789760794.378000:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ee7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554885632:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760794.378049:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67764224, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20518 00010000:00010000:1.0:1789760794.378058:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6e96 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.378075:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760794.378079:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.378086:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6e96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67764224->67768319] (req 67764224->67768319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.378100:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6e96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67764224->67768319] (req 67764224->67768319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.385133:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983853 is committed 00002000:00010000:1.0:1789760794.385542:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67764224 -> 67768320 00002000:00010000:1.0:1789760794.385548:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132376 -> 132384 00010000:00010000:1.0:1789760794.385561:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6e96 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67764224->67768319] (req 67764224->67768319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.385575:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6e96 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67764224->67768319] (req 67764224->67768319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.385584:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6e96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67764224->67768319] (req 67764224->67768319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.385606:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.385610:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6e96 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67764224->67768319] (req 67764224->67768319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.385619:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6e96 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67764224->67768319] (req 67764224->67768319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.385647:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6e96 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67764224->67768319] (req 67764224->67768319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.386232:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c6e96 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67764224->67768319] (req 67764224->67768319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.386337:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ee7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554885632:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8334us (8772us total) trans 4294983853 rc 0/0 00000100:00100000:1.0:1789760794.386358:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16555 00000100:00100000:1.0:1789760794.386367:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737613637359 time 737623604759 state 0x2 ref 0 00000100:00100000:1.0:1789760794.389724:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760794.389817:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554885760 00000100:00100000:0.0:1789760794.389826:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554885760 00000100:00100000:0.0:1789760794.389882:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737613637359 time 737623604759 state 0x0 ref 1 00000100:00100000:0.0:1789760794.389898:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737623865175 time 737623604759 state 0x1 ref 1 00000100:00100000:0.0:1789760794.389917:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737623865175 time 737635535768 state 0x0 ref 1 00000100:00100000:0.0:1789760794.389930:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554885760:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760794.389982:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67768320, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20520 00010000:00010000:0.0:1789760794.389991:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6e9d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.390007:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760794.390011:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.390017:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6e9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67768320->67772415] (req 67768320->67772415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.390031:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6e9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67768320->67772415] (req 67768320->67772415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.396034:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983854 is committed 00002000:00010000:1.0:1789760794.396349:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67768320 -> 67772416 00002000:00010000:1.0:1789760794.396354:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132384 -> 132392 00010000:00010000:1.0:1789760794.396369:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6e9d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67768320->67772415] (req 67768320->67772415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.396449:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6e9d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67768320->67772415] (req 67768320->67772415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.396459:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6e9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67768320->67772415] (req 67768320->67772415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.396467:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.396472:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6e9d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67768320->67772415] (req 67768320->67772415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.396482:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6e9d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67768320->67772415] (req 67768320->67772415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.396501:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6e9d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67768320->67772415] (req 67768320->67772415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.396509:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c6e9d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67768320->67772415] (req 67768320->67772415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.396647:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554885760:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6714us (6927us total) trans 4294983854 rc 0/0 00000100:00100000:1.0:1789760794.396659:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16556 00000100:00100000:1.0:1789760794.396668:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737623865175 time 737635535768 state 0x2 ref 0 00000100:00100000:0.0:1789760794.406113:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760794.406216:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554885888 00000100:00100000:2.0:1789760794.406226:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554885888 00000100:00100000:2.0:1789760794.406281:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737623865175 time 737635535768 state 0x0 ref 1 00000100:00100000:2.0:1789760794.406297:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737635796184 time 737635535768 state 0x1 ref 1 00000100:00100000:2.0:1789760794.406370:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737635796184 time 737651988149 state 0x0 ref 1 00000100:00100000:2.0:1789760794.406406:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554885888:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760794.406455:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67772416, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20528 00010000:00010000:2.0:1789760794.406464:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6ea4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.406481:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760794.406485:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.406492:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6ea4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67772416->67776511] (req 67772416->67776511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.406505:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6ea4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67772416->67776511] (req 67772416->67776511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.412462:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983855 is committed 00002000:00010000:2.0:1789760794.419424:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67772416 -> 67776512 00002000:00010000:2.0:1789760794.419434:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132392 -> 132400 00010000:00010000:2.0:1789760794.419448:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6ea4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67772416->67776511] (req 67772416->67776511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.419467:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6ea4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67772416->67776511] (req 67772416->67776511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.419475:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6ea4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67772416->67776511] (req 67772416->67776511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.419482:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760794.419487:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6ea4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67772416->67776511] (req 67772416->67776511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.419495:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6ea4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67772416->67776511] (req 67772416->67776511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.419577:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6ea4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67772416->67776511] (req 67772416->67776511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.419585:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6ea4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67772416->67776511] (req 67772416->67776511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.419802:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554885888:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13395us (13693us total) trans 4294983855 rc 0/0 00000100:00100000:2.0:1789760794.419814:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16557 00000100:00100000:2.0:1789760794.419824:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737635796184 time 737651988149 state 0x2 ref 0 00000100:00100000:2.0:1789760794.424900:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760794.425040:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554886016 00000100:00100000:3.0:1789760794.425049:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554886016 00000100:00100000:3.0:1789760794.425102:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737635796184 time 737651988149 state 0x0 ref 1 00000100:00100000:3.0:1789760794.425117:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737652248565 time 737651988149 state 0x1 ref 1 00000100:00100000:3.0:1789760794.425149:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737652248565 time 737670767709 state 0x0 ref 1 00000100:00100000:3.0:1789760794.425162:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05b8f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554886016:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760794.425205:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67776512, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20530 00010000:00010000:3.0:1789760794.425214:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6eab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.425237:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760794.425241:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.425248:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6eab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67776512->67780607] (req 67776512->67780607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.425264:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6eab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67776512->67780607] (req 67776512->67780607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.433429:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983856 is committed 00002000:00010000:3.0:1789760794.433683:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67776512 -> 67780608 00002000:00010000:3.0:1789760794.433688:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132400 -> 132408 00010000:00010000:3.0:1789760794.433702:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6eab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67776512->67780607] (req 67776512->67780607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.433716:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6eab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67776512->67780607] (req 67776512->67780607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.433726:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6eab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67776512->67780607] (req 67776512->67780607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.433735:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760794.433740:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6eab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67776512->67780607] (req 67776512->67780607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.433749:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6eab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67776512->67780607] (req 67776512->67780607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.433774:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6eab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67776512->67780607] (req 67776512->67780607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.433783:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6eab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67776512->67780607] (req 67776512->67780607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.433870:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05b8f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554886016:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8705us (8973us total) trans 4294983856 rc 0/0 00000100:00100000:3.0:1789760794.433881:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16558 00000100:00100000:3.0:1789760794.433890:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737652248565 time 737670767709 state 0x2 ref 0 00000100:00100000:1.0:1789760794.444200:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760794.444309:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554886144 00000100:00100000:0.0:1789760794.444319:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554886144 00000100:00100000:0.0:1789760794.444367:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737652248565 time 737670767709 state 0x0 ref 1 00000100:00100000:0.0:1789760794.444534:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737671028125 time 737670767709 state 0x1 ref 1 00000100:00100000:1.0:1789760794.446998:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737671028125 time 737692605514 state 0x0 ref 1 00000100:00100000:1.0:1789760794.447029:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554886144:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760794.447082:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67780608, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20538 00010000:00010000:1.0:1789760794.447091:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6eb2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.447107:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760794.447112:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.447118:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6eb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67780608->67784703] (req 67780608->67784703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.447131:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6eb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67780608->67784703] (req 67780608->67784703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.452772:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983857 is committed 00002000:00010000:2.0:1789760794.453223:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67780608 -> 67784704 00002000:00010000:2.0:1789760794.453229:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132408 -> 132416 00010000:00010000:2.0:1789760794.453241:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6eb2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67780608->67784703] (req 67780608->67784703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.453255:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6eb2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67780608->67784703] (req 67780608->67784703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.453265:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6eb2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67780608->67784703] (req 67780608->67784703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.453273:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760794.453278:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6eb2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67780608->67784703] (req 67780608->67784703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.453288:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6eb2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67780608->67784703] (req 67780608->67784703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.453309:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6eb2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67780608->67784703] (req 67780608->67784703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.453318:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6eb2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67780608->67784703] (req 67780608->67784703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.453820:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554886144:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6794us (9622us total) trans 4294983857 rc 0/0 00000100:00100000:2.0:1789760794.453832:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16559 00000100:00100000:2.0:1789760794.453843:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737671028125 time 737692605514 state 0x2 ref 0 00000100:00100000:0.0:1789760794.460204:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760794.460297:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554886272 00000100:00100000:1.0:1789760794.460307:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554886272 00000100:00100000:1.0:1789760794.460362:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737671028125 time 737692605514 state 0x0 ref 1 00000100:00100000:1.0:1789760794.460403:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737692865930 time 737692605514 state 0x1 ref 1 00000100:00100000:1.0:1789760794.460429:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737692865930 time 737706048302 state 0x0 ref 1 00000100:00100000:1.0:1789760794.460441:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554886272:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760794.460484:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67784704, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20540 00010000:00010000:1.0:1789760794.460492:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6eb9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.460509:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760794.460514:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.460521:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6eb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67784704->67788799] (req 67784704->67788799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.460534:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6eb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67784704->67788799] (req 67784704->67788799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.465225:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983858 is committed 00002000:00010000:1.0:1789760794.465342:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67784704 -> 67788800 00002000:00010000:1.0:1789760794.465448:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132416 -> 132424 00010000:00010000:1.0:1789760794.465463:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6eb9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67784704->67788799] (req 67784704->67788799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.465476:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6eb9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67784704->67788799] (req 67784704->67788799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.465487:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6eb9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67784704->67788799] (req 67784704->67788799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.465494:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655de00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.465500:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6eb9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67784704->67788799] (req 67784704->67788799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.465509:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6eb9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67784704->67788799] (req 67784704->67788799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.465534:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6eb9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67784704->67788799] (req 67784704->67788799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.465542:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6eb9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67784704->67788799] (req 67784704->67788799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.465648:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554886272:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5203us (5448us total) trans 4294983858 rc 0/0 00000100:00100000:1.0:1789760794.465658:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16560 00000100:00100000:1.0:1789760794.465668:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737692865930 time 737706048302 state 0x2 ref 0 00000100:00100000:3.0:1789760794.470790:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760794.470857:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554886400 00000100:00100000:2.0:1789760794.470867:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554886400 00000100:00100000:2.0:1789760794.470923:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737692865930 time 737706048302 state 0x0 ref 1 00000100:00100000:2.0:1789760794.470938:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737706308718 time 737706048302 state 0x1 ref 1 00000100:00100000:2.0:1789760794.470974:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737706308718 time 737716592331 state 0x0 ref 1 00000100:00100000:2.0:1789760794.470987:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554886400:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760794.471037:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67788800, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20548 00010000:00010000:2.0:1789760794.471046:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6ec0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.471074:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760794.471078:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.471085:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6ec0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67788800->67792895] (req 67788800->67792895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.471099:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6ec0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67788800->67792895] (req 67788800->67792895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.475822:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983859 is committed 00002000:00010000:2.0:1789760794.475937:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67788800 -> 67792896 00002000:00010000:2.0:1789760794.475943:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132424 -> 132432 00010000:00010000:2.0:1789760794.475954:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6ec0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67788800->67792895] (req 67788800->67792895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.475985:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6ec0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67788800->67792895] (req 67788800->67792895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.475995:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6ec0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67788800->67792895] (req 67788800->67792895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.476002:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760794.476007:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6ec0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67788800->67792895] (req 67788800->67792895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.476016:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6ec0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67788800->67792895] (req 67788800->67792895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.476035:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6ec0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67788800->67792895] (req 67788800->67792895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.476043:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6ec0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67788800->67792895] (req 67788800->67792895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.476147:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554886400:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5153us (5358us total) trans 4294983859 rc 0/0 00000100:00100000:2.0:1789760794.476158:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16561 00000100:00100000:2.0:1789760794.476168:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737706308718 time 737716592331 state 0x2 ref 0 00000100:00100000:3.0:1789760794.479867:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760794.481206:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554886528 00000100:00100000:1.0:1789760794.481220:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554886528 00000100:00100000:1.0:1789760794.481281:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737706308718 time 737716592331 state 0x0 ref 1 00000100:00100000:1.0:1789760794.481296:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737716852747 time 737716592331 state 0x1 ref 1 00000100:00100000:1.0:1789760794.481450:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737716852747 time 737727066825 state 0x0 ref 1 00000100:00100000:1.0:1789760794.481464:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554886528:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760794.481513:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67792896, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20550 00010000:00010000:1.0:1789760794.481523:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ec7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.481538:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760794.481543:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.481549:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ec7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67792896->67796991] (req 67792896->67796991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.481563:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ec7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67792896->67796991] (req 67792896->67796991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.487831:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983860 is committed 00002000:00010000:1.0:1789760794.487950:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67792896 -> 67796992 00002000:00010000:1.0:1789760794.487955:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132432 -> 132440 00010000:00010000:1.0:1789760794.487977:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ec7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67792896->67796991] (req 67792896->67796991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.487991:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ec7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67792896->67796991] (req 67792896->67796991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.488001:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ec7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67792896->67796991] (req 67792896->67796991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.488009:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655de00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.488014:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ec7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67792896->67796991] (req 67792896->67796991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.488023:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ec7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67792896->67796991] (req 67792896->67796991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.488042:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ec7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67792896->67796991] (req 67792896->67796991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.488056:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ec7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67792896->67796991] (req 67792896->67796991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.488148:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554886528:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6682us (8285us total) trans 4294983860 rc 0/0 00000100:00100000:1.0:1789760794.488158:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16562 00000100:00100000:1.0:1789760794.488167:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737716852747 time 737727066825 state 0x2 ref 0 00000100:00100000:0.0:1789760794.493210:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760794.493312:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554886656 00000100:00100000:1.0:1789760794.493321:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554886656 00000100:00100000:1.0:1789760794.493568:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737716852747 time 737727066825 state 0x0 ref 1 00000100:00100000:1.0:1789760794.493584:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737727327241 time 737727066825 state 0x1 ref 1 00000100:00100000:1.0:1789760794.494138:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737727327241 time 737739753448 state 0x0 ref 1 00000100:00100000:1.0:1789760794.494158:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554886656:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760794.494211:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67796992, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20558 00010000:00010000:1.0:1789760794.494221:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ece lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.494238:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760794.494242:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.494307:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ece lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67796992->67801087] (req 67796992->67801087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.494329:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ece lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67796992->67801087] (req 67796992->67801087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.502080:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983861 is committed 00002000:00010000:1.0:1789760794.502266:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67796992 -> 67801088 00002000:00010000:1.0:1789760794.502272:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132440 -> 132448 00010000:00010000:1.0:1789760794.502286:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ece lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67796992->67801087] (req 67796992->67801087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.502298:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ece lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67796992->67801087] (req 67796992->67801087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.502307:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ece lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67796992->67801087] (req 67796992->67801087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.502314:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655de00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.502320:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ece lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67796992->67801087] (req 67796992->67801087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.502329:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ece lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67796992->67801087] (req 67796992->67801087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.502347:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ece lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67796992->67801087] (req 67796992->67801087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.502354:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c6ece lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67796992->67801087] (req 67796992->67801087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.502500:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554886656:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8343us (9292us total) trans 4294983861 rc 0/0 00000100:00100000:1.0:1789760794.502511:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16563 00000100:00100000:1.0:1789760794.502520:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737727327241 time 737739753448 state 0x2 ref 0 00000100:00100000:3.0:1789760794.507078:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760794.507271:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554886784 00000100:00100000:0.0:1789760794.507281:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554886784 00000100:00100000:0.0:1789760794.507335:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737727327241 time 737739753448 state 0x0 ref 1 00000100:00100000:0.0:1789760794.507350:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737740013864 time 737739753448 state 0x1 ref 1 00000100:00100000:0.0:1789760794.507417:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737740013864 time 737753034423 state 0x0 ref 1 00000100:00100000:0.0:1789760794.507429:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554886784:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760794.507481:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67801088, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20560 00010000:00010000:0.0:1789760794.507490:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6ed5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.507506:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760794.507510:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.507516:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6ed5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67801088->67805183] (req 67801088->67805183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.507530:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6ed5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67801088->67805183] (req 67801088->67805183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.514316:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983862 is committed 00002000:00010000:0.0:1789760794.514494:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67801088 -> 67805184 00002000:00010000:0.0:1789760794.514499:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132448 -> 132456 00010000:00010000:0.0:1789760794.514513:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6ed5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67801088->67805183] (req 67801088->67805183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.514536:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6ed5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67801088->67805183] (req 67801088->67805183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.514545:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6ed5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67801088->67805183] (req 67801088->67805183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.514552:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760794.514557:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6ed5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67801088->67805183] (req 67801088->67805183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.514565:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6ed5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67801088->67805183] (req 67801088->67805183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.514584:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6ed5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67801088->67805183] (req 67801088->67805183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.514601:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c6ed5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67801088->67805183] (req 67801088->67805183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.514694:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554886784:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7261us (7619us total) trans 4294983862 rc 0/0 00000100:00100000:0.0:1789760794.514704:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16564 00000100:00100000:0.0:1789760794.514715:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737740013864 time 737753034423 state 0x2 ref 0 00000100:00100000:3.0:1789760794.525054:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760794.525175:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554886912 00000100:00100000:2.0:1789760794.525186:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554886912 00000100:00100000:2.0:1789760794.525237:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737740013864 time 737753034423 state 0x0 ref 1 00000100:00100000:2.0:1789760794.525253:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737753294839 time 737753034423 state 0x1 ref 1 00000100:00100000:2.0:1789760794.525288:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737753294839 time 737770906197 state 0x0 ref 1 00000100:00100000:2.0:1789760794.525300:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554886912:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760794.525345:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67805184, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20568 00010000:00010000:2.0:1789760794.525355:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6edc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.525371:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760794.525414:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.525421:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6edc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67805184->67809279] (req 67805184->67809279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.525435:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6edc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67805184->67809279] (req 67805184->67809279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.532524:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983863 is committed 00002000:00010000:2.0:1789760794.532700:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67805184 -> 67809280 00002000:00010000:2.0:1789760794.532705:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132456 -> 132464 00010000:00010000:2.0:1789760794.532718:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6edc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67805184->67809279] (req 67805184->67809279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.532731:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6edc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67805184->67809279] (req 67805184->67809279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.532747:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6edc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67805184->67809279] (req 67805184->67809279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.532754:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760794.532759:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6edc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67805184->67809279] (req 67805184->67809279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.532768:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6edc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67805184->67809279] (req 67805184->67809279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.532786:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6edc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67805184->67809279] (req 67805184->67809279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.532793:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6edc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67805184->67809279] (req 67805184->67809279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.532880:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554886912:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7578us (7831us total) trans 4294983863 rc 0/0 00000100:00100000:2.0:1789760794.532890:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16565 00000100:00100000:2.0:1789760794.532899:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737753294839 time 737770906197 state 0x2 ref 0 00000100:00100000:1.0:1789760794.535947:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760794.536652:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554887040 00000100:00100000:3.0:1789760794.536663:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554887040 00000100:00100000:3.0:1789760794.536722:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737753294839 time 737770906197 state 0x0 ref 1 00000100:00100000:3.0:1789760794.536737:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737771166613 time 737770906197 state 0x1 ref 1 00000100:00100000:3.0:1789760794.536770:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737771166613 time 737782388431 state 0x0 ref 1 00000100:00100000:3.0:1789760794.536783:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554887040:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760794.536827:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67809280, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20570 00010000:00010000:3.0:1789760794.536837:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6ee3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.536855:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760794.536859:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.536866:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6ee3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67809280->67813375] (req 67809280->67813375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.536881:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6ee3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67809280->67813375] (req 67809280->67813375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.546099:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983864 is committed 00002000:00010000:3.0:1789760794.546836:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67809280 -> 67813376 00002000:00010000:3.0:1789760794.546843:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132464 -> 132472 00010000:00010000:3.0:1789760794.546857:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6ee3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67809280->67813375] (req 67809280->67813375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.546870:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6ee3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67809280->67813375] (req 67809280->67813375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.546879:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6ee3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67809280->67813375] (req 67809280->67813375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.546893:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760794.546898:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6ee3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67809280->67813375] (req 67809280->67813375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.546907:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6ee3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67809280->67813375] (req 67809280->67813375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.546928:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6ee3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67809280->67813375] (req 67809280->67813375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.546936:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6ee3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67809280->67813375] (req 67809280->67813375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.547525:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554887040:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10738us (11588us total) trans 4294983864 rc 0/0 00000100:00100000:3.0:1789760794.547537:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16566 00000100:00100000:3.0:1789760794.547545:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737771166613 time 737782388431 state 0x2 ref 0 00000100:00100000:3.0:1789760794.551157:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760794.551248:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554887168 00000100:00100000:1.0:1789760794.551256:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554887168 00000100:00100000:1.0:1789760794.551302:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737771166613 time 737782388431 state 0x0 ref 1 00000100:00100000:1.0:1789760794.551313:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737782648847 time 737782388431 state 0x1 ref 1 00000100:00100000:1.0:1789760794.551336:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737782648847 time 737796955815 state 0x0 ref 1 00000100:00100000:1.0:1789760794.551344:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554887168:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760794.551424:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67813376, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20578 00010000:00010000:1.0:1789760794.551431:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6eea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.551443:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760794.551447:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.551451:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6eea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67813376->67817471] (req 67813376->67817471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.551461:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6eea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67813376->67817471] (req 67813376->67817471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.558284:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983865 is committed 00002000:00010000:1.0:1789760794.558440:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67813376 -> 67817472 00002000:00010000:1.0:1789760794.558446:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132472 -> 132480 00010000:00010000:1.0:1789760794.558459:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6eea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67813376->67817471] (req 67813376->67817471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.558472:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6eea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67813376->67817471] (req 67813376->67817471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.558481:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6eea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67813376->67817471] (req 67813376->67817471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.558489:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.558494:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6eea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67813376->67817471] (req 67813376->67817471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.558504:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6eea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67813376->67817471] (req 67813376->67817471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.558523:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6eea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67813376->67817471] (req 67813376->67817471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.558532:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6eea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67813376->67817471] (req 67813376->67817471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.559345:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554887168:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7992us (8188us total) trans 4294983865 rc 0/0 00000100:00100000:1.0:1789760794.559357:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16567 00000100:00100000:1.0:1789760794.559366:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737782648847 time 737796955815 state 0x2 ref 0 00000100:00100000:1.0:1789760794.565448:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760794.565606:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554887296 00000100:00100000:0.0:1789760794.565616:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554887296 00000100:00100000:0.0:1789760794.565867:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737782648847 time 737796955815 state 0x0 ref 1 00000100:00100000:0.0:1789760794.565892:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737797216231 time 737796955815 state 0x1 ref 1 00000100:00100000:0.0:1789760794.565917:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737797216231 time 737811536673 state 0x0 ref 1 00000100:00100000:0.0:1789760794.565927:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ebce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554887296:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760794.565971:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67817472, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20580 00010000:00010000:0.0:1789760794.565981:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6ef1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.565997:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760794.566001:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.566007:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6ef1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67817472->67821567] (req 67817472->67821567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.566020:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6ef1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67817472->67821567] (req 67817472->67821567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.571432:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983866 is committed 00002000:00010000:0.0:1789760794.571542:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67817472 -> 67821568 00002000:00010000:0.0:1789760794.571547:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132480 -> 132488 00010000:00010000:0.0:1789760794.571560:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6ef1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67817472->67821567] (req 67817472->67821567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.571576:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6ef1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67817472->67821567] (req 67817472->67821567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.571585:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6ef1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67817472->67821567] (req 67817472->67821567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.571602:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760794.571607:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6ef1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67817472->67821567] (req 67817472->67821567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.571616:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6ef1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67817472->67821567] (req 67817472->67821567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.571636:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6ef1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67817472->67821567] (req 67817472->67821567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.571644:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6ef1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67817472->67821567] (req 67817472->67821567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.571774:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ebce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554887296:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5843us (6329us total) trans 4294983866 rc 0/0 00000100:00100000:0.0:1789760794.571785:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16568 00000100:00100000:0.0:1789760794.571794:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737797216231 time 737811536673 state 0x2 ref 0 00000100:00100000:1.0:1789760794.579102:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760794.579211:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554887424 00000100:00100000:2.0:1789760794.579222:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554887424 00000100:00100000:2.0:1789760794.579280:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737797216231 time 737811536673 state 0x0 ref 1 00000100:00100000:2.0:1789760794.579296:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737811797089 time 737811536673 state 0x1 ref 1 00000100:00100000:2.0:1789760794.579329:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737811797089 time 737824947029 state 0x0 ref 1 00000100:00100000:2.0:1789760794.579341:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06448a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554887424:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760794.579414:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67821568, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20588 00010000:00010000:2.0:1789760794.579424:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6ef8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.579440:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760794.579445:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.579452:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6ef8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67821568->67825663] (req 67821568->67825663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.579466:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6ef8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67821568->67825663] (req 67821568->67825663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.586547:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983867 is committed 00002000:00010000:1.0:1789760794.586687:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67821568 -> 67825664 00002000:00010000:1.0:1789760794.586692:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132488 -> 132496 00010000:00010000:1.0:1789760794.586704:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6ef8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67821568->67825663] (req 67821568->67825663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.586716:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6ef8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67821568->67825663] (req 67821568->67825663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.586725:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6ef8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67821568->67825663] (req 67821568->67825663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.586732:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.586736:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6ef8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67821568->67825663] (req 67821568->67825663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.586744:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6ef8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67821568->67825663] (req 67821568->67825663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.586762:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6ef8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67821568->67825663] (req 67821568->67825663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.586773:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6ef8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67821568->67825663] (req 67821568->67825663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.586877:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06448a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554887424:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7531us (7778us total) trans 4294983867 rc 0/0 00000100:00100000:1.0:1789760794.586888:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16569 00000100:00100000:1.0:1789760794.586897:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737811797089 time 737824947029 state 0x2 ref 0 00000100:00100000:2.0:1789760794.594727:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760794.594911:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554887552 00000100:00100000:3.0:1789760794.594922:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554887552 00000100:00100000:3.0:1789760794.594985:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737811797089 time 737824947029 state 0x0 ref 1 00000100:00100000:3.0:1789760794.595003:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737825207445 time 737824947029 state 0x1 ref 1 00000100:00100000:3.0:1789760794.595042:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737825207445 time 737840660393 state 0x0 ref 1 00000100:00100000:3.0:1789760794.595056:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05b8df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554887552:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760794.595180:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67825664, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20590 00010000:00010000:0.0:1789760794.595192:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6eff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.595208:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760794.595213:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.595219:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6eff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67825664->67829759] (req 67825664->67829759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.595234:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6eff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67825664->67829759] (req 67825664->67829759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.601905:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983868 is committed 00002000:00010000:1.0:1789760794.602030:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67825664 -> 67829760 00002000:00010000:1.0:1789760794.602035:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132496 -> 132504 00010000:00010000:1.0:1789760794.602049:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6eff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67825664->67829759] (req 67825664->67829759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.602063:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6eff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67825664->67829759] (req 67825664->67829759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.602073:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6eff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67825664->67829759] (req 67825664->67829759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.602080:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.602086:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6eff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67825664->67829759] (req 67825664->67829759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.602095:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6eff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67825664->67829759] (req 67825664->67829759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.602120:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6eff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67825664->67829759] (req 67825664->67829759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.602128:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c6eff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67825664->67829759] (req 67825664->67829759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.602309:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05b8df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554887552:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7250us (7588us total) trans 4294983868 rc 0/0 00000100:00100000:1.0:1789760794.602319:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16570 00000100:00100000:1.0:1789760794.602329:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737825207445 time 737840660393 state 0x2 ref 0 00000100:00100000:2.0:1789760794.608909:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760794.609010:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554887680 00000100:00100000:3.0:1789760794.609021:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554887680 00000100:00100000:3.0:1789760794.609075:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737825207445 time 737840660393 state 0x0 ref 1 00000100:00100000:3.0:1789760794.609091:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737840920809 time 737840660393 state 0x1 ref 1 00000100:00100000:3.0:1789760794.609124:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737840920809 time 737854742226 state 0x0 ref 1 00000100:00100000:3.0:1789760794.609136:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05b8d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554887680:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760794.609195:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67829760, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x20598 00010000:00010000:3.0:1789760794.609204:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f06 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.609221:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760794.609226:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.609233:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67829760->67833855] (req 67829760->67833855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.609249:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67829760->67833855] (req 67829760->67833855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.616935:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983869 is committed 00002000:00010000:3.0:1789760794.617225:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67829760 -> 67833856 00002000:00010000:3.0:1789760794.617230:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132504 -> 132512 00010000:00010000:3.0:1789760794.617244:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f06 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67829760->67833855] (req 67829760->67833855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.617258:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f06 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67829760->67833855] (req 67829760->67833855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.617267:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f06 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67829760->67833855] (req 67829760->67833855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.617276:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760794.617281:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f06 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67829760->67833855] (req 67829760->67833855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.617291:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f06 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67829760->67833855] (req 67829760->67833855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.617311:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f06 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67829760->67833855] (req 67829760->67833855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.617320:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f06 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67829760->67833855] (req 67829760->67833855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.617544:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05b8d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554887680:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8404us (8639us total) trans 4294983869 rc 0/0 00000100:00100000:3.0:1789760794.617566:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16571 00000100:00100000:3.0:1789760794.617575:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737840920809 time 737854742226 state 0x2 ref 0 00000100:00100000:0.0:1789760794.626028:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760794.627727:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554887808 00000100:00100000:1.0:1789760794.627741:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554887808 00000100:00100000:1.0:1789760794.627994:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737840920809 time 737854742226 state 0x0 ref 1 00000100:00100000:1.0:1789760794.628011:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737855002642 time 737854742226 state 0x1 ref 1 00000100:00100000:1.0:1789760794.628045:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737855002642 time 737873663946 state 0x0 ref 1 00000100:00100000:1.0:1789760794.628061:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554887808:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760794.628113:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67833856, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x205a0 00010000:00010000:1.0:1789760794.628124:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f0d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.628140:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760794.628147:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.628153:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67833856->67837951] (req 67833856->67837951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.628166:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67833856->67837951] (req 67833856->67837951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.634303:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983870 is committed 00002000:00010000:1.0:1789760794.634488:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67833856 -> 67837952 00002000:00010000:1.0:1789760794.634493:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132512 -> 132520 00010000:00010000:1.0:1789760794.634506:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f0d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67833856->67837951] (req 67833856->67837951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.634518:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f0d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67833856->67837951] (req 67833856->67837951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.634526:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f0d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67833856->67837951] (req 67833856->67837951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.634533:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.634538:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f0d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67833856->67837951] (req 67833856->67837951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.634547:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f0d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67833856->67837951] (req 67833856->67837951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.634568:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f0d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67833856->67837951] (req 67833856->67837951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.634598:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f0d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67833856->67837951] (req 67833856->67837951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.634690:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554887808:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6630us (8672us total) trans 4294983870 rc 0/0 00000100:00100000:1.0:1789760794.634701:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16572 00000100:00100000:1.0:1789760794.634710:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737855002642 time 737873663946 state 0x2 ref 0 00000100:00100000:2.0:1789760794.640460:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760794.640546:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554887936 00000100:00100000:1.0:1789760794.640556:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554887936 00000100:00100000:1.0:1789760794.640633:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737855002642 time 737873663946 state 0x0 ref 1 00000100:00100000:1.0:1789760794.640648:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737873924362 time 737873663946 state 0x1 ref 1 00000100:00100000:1.0:1789760794.640680:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737873924362 time 737886298347 state 0x0 ref 1 00000100:00100000:1.0:1789760794.640691:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05b8e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554887936:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760794.640735:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67837952, mtime: 0x6aad9519, atime: 0x0, ctime: 0x6aad9519, blocks: 0x205a8 00010000:00010000:1.0:1789760794.640743:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f14 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.640758:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760794.640763:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.640770:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67837952->67842047] (req 67837952->67842047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.640800:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67837952->67842047] (req 67837952->67842047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.647077:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983871 is committed 00002000:00010000:1.0:1789760794.647199:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67837952 -> 67842048 00002000:00010000:1.0:1789760794.647204:0:15101:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760793 -> 1789760794 00002000:00010000:1.0:1789760794.647207:0:15101:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760793 -> 1789760794 00002000:00010000:1.0:1789760794.647210:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132520 -> 132528 00010000:00010000:1.0:1789760794.647223:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f14 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67837952->67842047] (req 67837952->67842047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.647236:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f14 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67837952->67842047] (req 67837952->67842047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.647245:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f14 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67837952->67842047] (req 67837952->67842047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.647252:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.647257:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f14 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67837952->67842047] (req 67837952->67842047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.647267:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f14 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67837952->67842047] (req 67837952->67842047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.647286:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f14 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67837952->67842047] (req 67837952->67842047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.647298:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f14 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67837952->67842047] (req 67837952->67842047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.647481:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05b8e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554887936:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6785us (7024us total) trans 4294983871 rc 0/0 00000100:00100000:1.0:1789760794.647491:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16573 00000100:00100000:1.0:1789760794.647500:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737873924362 time 737886298347 state 0x2 ref 0 00000100:00100000:2.0:1789760794.658116:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760794.660977:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554888064 00000100:00100000:1.0:1789760794.660994:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554888064 00000100:00100000:1.0:1789760794.661058:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737873924362 time 737886298347 state 0x0 ref 1 00000100:00100000:1.0:1789760794.661076:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737886558763 time 737886298347 state 0x1 ref 1 00000100:00100000:1.0:1789760794.661181:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737886558763 time 737906797420 state 0x0 ref 1 00000100:00100000:1.0:1789760794.661195:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05b8d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554888064:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760794.661240:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67842048, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x205b0 00010000:00010000:1.0:1789760794.661249:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f1b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.661265:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760794.661270:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.661277:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67842048->67846143] (req 67842048->67846143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.661297:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67842048->67846143] (req 67842048->67846143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.671721:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983872 is committed 00002000:00010000:0.0:1789760794.673079:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67842048 -> 67846144 00002000:00010000:0.0:1789760794.673087:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132528 -> 132536 00010000:00010000:0.0:1789760794.673100:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f1b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67842048->67846143] (req 67842048->67846143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.673114:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f1b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67842048->67846143] (req 67842048->67846143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.673123:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f1b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67842048->67846143] (req 67842048->67846143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.673130:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760794.673135:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f1b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67842048->67846143] (req 67842048->67846143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.673217:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f1b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67842048->67846143] (req 67842048->67846143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.673238:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f1b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67842048->67846143] (req 67842048->67846143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.673246:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f1b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67842048->67846143] (req 67842048->67846143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.675031:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05b8d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554888064:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13827us (16914us total) trans 4294983872 rc 0/0 00000100:00100000:0.0:1789760794.675046:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16574 00000100:00100000:0.0:1789760794.675057:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737886558763 time 737906797420 state 0x2 ref 0 00000100:00100000:0.0:1789760794.685516:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760794.685598:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554888192 00000100:00100000:3.0:1789760794.685608:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554888192 00000100:00100000:3.0:1789760794.685676:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737886558763 time 737906797420 state 0x0 ref 1 00000100:00100000:3.0:1789760794.685692:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737907057836 time 737906797420 state 0x1 ref 1 00000100:00100000:3.0:1789760794.685723:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737907057836 time 737931341780 state 0x0 ref 1 00000100:00100000:3.0:1789760794.685737:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03de4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554888192:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760794.685788:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67846144, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x205b8 00010000:00010000:3.0:1789760794.685799:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f22 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.685817:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:3.0:1789760794.685821:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.685828:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67846144->67850239] (req 67846144->67850239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.685843:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67846144->67850239] (req 67846144->67850239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.691528:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983873 is committed 00002000:00010000:3.0:1789760794.691657:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67846144 -> 67850240 00002000:00010000:3.0:1789760794.691663:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132536 -> 132544 00010000:00010000:3.0:1789760794.691675:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f22 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67846144->67850239] (req 67846144->67850239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.691693:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f22 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67846144->67850239] (req 67846144->67850239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.691703:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f22 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67846144->67850239] (req 67846144->67850239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.691711:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760794.691716:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f22 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67846144->67850239] (req 67846144->67850239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.691725:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f22 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67846144->67850239] (req 67846144->67850239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.691743:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f22 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67846144->67850239] (req 67846144->67850239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.691750:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f22 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67846144->67850239] (req 67846144->67850239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.691849:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03de4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554888192:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6109us (6336us total) trans 4294983873 rc 0/0 00000100:00100000:3.0:1789760794.691861:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16575 00000100:00100000:3.0:1789760794.691871:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737907057836 time 737931341780 state 0x2 ref 0 00000100:00100000:2.0:1789760794.696830:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760794.697020:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554888320 00000100:00100000:0.0:1789760794.697030:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554888320 00000100:00100000:0.0:1789760794.697087:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737907057836 time 737931341780 state 0x0 ref 1 00000100:00100000:0.0:1789760794.697103:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737931602196 time 737931341780 state 0x1 ref 1 00000100:00100000:0.0:1789760794.697138:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737931602196 time 737942756407 state 0x0 ref 1 00000100:00100000:0.0:1789760794.697151:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05b8c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554888320:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760794.697197:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67850240, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x205c0 00010000:00010000:0.0:1789760794.697207:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f29 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.697222:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760794.697227:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.697234:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67850240->67854335] (req 67850240->67854335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.697302:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67850240->67854335] (req 67850240->67854335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.703326:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983874 is committed 00002000:00010000:0.0:1789760794.706109:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67850240 -> 67854336 00002000:00010000:0.0:1789760794.706119:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132544 -> 132552 00010000:00010000:0.0:1789760794.706133:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f29 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67850240->67854335] (req 67850240->67854335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.706146:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f29 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67850240->67854335] (req 67850240->67854335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.706155:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f29 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67850240->67854335] (req 67850240->67854335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.706163:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760794.706177:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f29 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67850240->67854335] (req 67850240->67854335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.706187:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f29 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67850240->67854335] (req 67850240->67854335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.706215:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f29 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67850240->67854335] (req 67850240->67854335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.706225:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f29 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67850240->67854335] (req 67850240->67854335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.706408:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05b8c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554888320:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9222us (9550us total) trans 4294983874 rc 0/0 00000100:00100000:0.0:1789760794.706420:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16576 00000100:00100000:0.0:1789760794.706430:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737931602196 time 737942756407 state 0x2 ref 0 00000100:00100000:2.0:1789760794.709712:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760794.711495:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554888448 00000100:00100000:1.0:1789760794.711511:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554888448 00000100:00100000:1.0:1789760794.711578:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737931602196 time 737942756407 state 0x0 ref 1 00000100:00100000:1.0:1789760794.711602:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737943016823 time 737942756407 state 0x1 ref 1 00000100:00100000:1.0:1789760794.711639:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737943016823 time 737957257898 state 0x0 ref 1 00000100:00100000:1.0:1789760794.711652:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05b8ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554888448:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760794.711700:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67854336, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x205c8 00010000:00010000:1.0:1789760794.711709:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6f30 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.711724:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:1.0:1789760794.711728:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.711735:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6f30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67854336->67858431] (req 67854336->67858431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.711749:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6f30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67854336->67858431] (req 67854336->67858431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.719791:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983875 is committed 00002000:00010000:1.0:1789760794.720149:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67854336 -> 67858432 00002000:00010000:1.0:1789760794.720155:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132552 -> 132560 00010000:00010000:1.0:1789760794.720167:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6f30 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67854336->67858431] (req 67854336->67858431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.720180:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6f30 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67854336->67858431] (req 67854336->67858431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.720188:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6f30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67854336->67858431] (req 67854336->67858431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.720195:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.720200:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6f30 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67854336->67858431] (req 67854336->67858431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.720208:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6f30 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67854336->67858431] (req 67854336->67858431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.720226:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6f30 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67854336->67858431] (req 67854336->67858431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.720233:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c6f30 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67854336->67858431] (req 67854336->67858431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.720331:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05b8ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554888448:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8676us (10622us total) trans 4294983875 rc 0/0 00000100:00100000:1.0:1789760794.720342:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16577 00000100:00100000:1.0:1789760794.720352:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737943016823 time 737957257898 state 0x2 ref 0 00000100:00100000:1.0:1789760794.725679:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760794.725767:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554888576 00000100:00100000:2.0:1789760794.725778:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554888576 00000100:00100000:2.0:1789760794.726064:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737943016823 time 737957257898 state 0x0 ref 1 00000100:00100000:2.0:1789760794.726081:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737957518314 time 737957257898 state 0x1 ref 1 00000100:00100000:2.0:1789760794.726151:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737957518314 time 737971769027 state 0x0 ref 1 00000100:00100000:2.0:1789760794.726165:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06449f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554888576:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760794.726212:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67858432, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x205d0 00010000:00010000:2.0:1789760794.726222:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6f37 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.726238:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:2.0:1789760794.726242:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.726249:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6f37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67858432->67862527] (req 67858432->67862527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.726264:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6f37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67858432->67862527] (req 67858432->67862527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.736505:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983876 is committed 00002000:00010000:2.0:1789760794.736687:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67858432 -> 67862528 00002000:00010000:2.0:1789760794.736693:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132560 -> 132568 00010000:00010000:2.0:1789760794.736705:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6f37 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67858432->67862527] (req 67858432->67862527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.736767:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6f37 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67858432->67862527] (req 67858432->67862527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.736781:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6f37 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67858432->67862527] (req 67858432->67862527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.736787:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760794.736792:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6f37 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67858432->67862527] (req 67858432->67862527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.736802:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6f37 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67858432->67862527] (req 67858432->67862527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.736821:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6f37 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67858432->67862527] (req 67858432->67862527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.736875:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6f37 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67858432->67862527] (req 67858432->67862527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.737121:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06449f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554888576:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10953us (11446us total) trans 4294983876 rc 0/0 00000100:00100000:2.0:1789760794.737132:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16578 00000100:00100000:2.0:1789760794.737141:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737957518314 time 737971769027 state 0x2 ref 0 00000100:00100000:2.0:1789760794.743576:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760794.743691:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554888704 00000100:00100000:0.0:1789760794.743701:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554888704 00000100:00100000:0.0:1789760794.743753:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737957518314 time 737971769027 state 0x0 ref 1 00000100:00100000:0.0:1789760794.743767:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737972029443 time 737971769027 state 0x1 ref 1 00000100:00100000:0.0:1789760794.743801:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737972029443 time 737989419437 state 0x0 ref 1 00000100:00100000:0.0:1789760794.743813:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb05b8fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554888704:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760794.743864:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67862528, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x205d8 00010000:00010000:0.0:1789760794.743874:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f3e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.743890:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=737s v=(0 0) 00010000:00010000:0.0:1789760794.743896:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.743902:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67862528->67866623] (req 67862528->67866623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.743915:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67862528->67866623] (req 67862528->67866623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.753737:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983877 is committed 00002000:00010000:0.0:1789760794.753881:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67862528 -> 67866624 00002000:00010000:0.0:1789760794.753886:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132568 -> 132576 00010000:00010000:0.0:1789760794.753899:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f3e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67862528->67866623] (req 67862528->67866623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.753912:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f3e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67862528->67866623] (req 67862528->67866623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.753921:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67862528->67866623] (req 67862528->67866623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.753928:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760794.753933:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f3e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67862528->67866623] (req 67862528->67866623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.753941:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f3e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67862528->67866623] (req 67862528->67866623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.753961:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f3e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67862528->67866623] (req 67862528->67866623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.754097:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f3e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67862528->67866623] (req 67862528->67866623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.754281:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb05b8fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554888704:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10464us (10709us total) trans 4294983877 rc 0/0 00000100:00100000:0.0:1789760794.754292:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16579 00000100:00100000:0.0:1789760794.754301:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737972029443 time 737989419437 state 0x2 ref 0 00000100:00100000:1.0:1789760794.759129:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760794.759211:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554888832 00000100:00100000:0.0:1789760794.759221:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554888832 00000100:00100000:0.0:1789760794.759273:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737972029443 time 737989419437 state 0x0 ref 1 00000100:00100000:0.0:1789760794.759288:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737989679853 time 737989419437 state 0x1 ref 1 00000100:00100000:0.0:1789760794.759324:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737989679853 time 738004942636 state 0x0 ref 1 00000100:00100000:0.0:1789760794.759338:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0644b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554888832:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760794.759435:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67866624, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x205e0 00010000:00010000:0.0:1789760794.759444:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f45 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.759460:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760794.759466:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.759474:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67866624->67870719] (req 67866624->67870719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.759495:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67866624->67870719] (req 67866624->67870719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.766491:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983878 is committed 00002000:00010000:0.0:1789760794.766776:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67866624 -> 67870720 00002000:00010000:0.0:1789760794.766782:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132576 -> 132584 00010000:00010000:0.0:1789760794.766801:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f45 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67866624->67870719] (req 67866624->67870719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.766815:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f45 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67866624->67870719] (req 67866624->67870719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.766826:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f45 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67866624->67870719] (req 67866624->67870719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.766834:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760794.766839:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f45 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67866624->67870719] (req 67866624->67870719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.766849:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f45 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67866624->67870719] (req 67866624->67870719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.766873:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f45 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67866624->67870719] (req 67866624->67870719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.766882:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f45 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67866624->67870719] (req 67866624->67870719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.767012:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0644b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554888832:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7670us (7885us total) trans 4294983878 rc 0/0 00000100:00100000:0.0:1789760794.767024:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16580 00000100:00100000:0.0:1789760794.767036:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737989679853 time 738004942636 state 0x2 ref 0 00000100:00100000:1.0:1789760794.772078:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760794.773720:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554888960 00000100:00100000:2.0:1789760794.773736:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554888960 00000100:00100000:2.0:1789760794.773796:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 737989679853 time 738004942636 state 0x0 ref 1 00000100:00100000:2.0:1789760794.773815:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738005203052 time 738004942636 state 0x1 ref 1 00000100:00100000:2.0:1789760794.773848:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738005203052 time 738019466418 state 0x0 ref 1 00000100:00100000:2.0:1789760794.773863:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06448380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554888960:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760794.773909:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67870720, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x205e8 00010000:00010000:2.0:1789760794.773918:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6f4c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.773935:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760794.773940:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.773945:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6f4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67870720->67874815] (req 67870720->67874815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.773958:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6f4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67870720->67874815] (req 67870720->67874815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.782981:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983879 is committed 00002000:00010000:2.0:1789760794.783874:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67870720 -> 67874816 00002000:00010000:2.0:1789760794.783880:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132584 -> 132592 00010000:00010000:2.0:1789760794.783897:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6f4c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67870720->67874815] (req 67870720->67874815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.783913:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6f4c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67870720->67874815] (req 67870720->67874815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.783923:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6f4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67870720->67874815] (req 67870720->67874815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.783932:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760794.783937:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6f4c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67870720->67874815] (req 67870720->67874815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.783948:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6f4c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67870720->67874815] (req 67870720->67874815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.783972:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6f4c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67870720->67874815] (req 67870720->67874815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.783981:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6f4c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67870720->67874815] (req 67870720->67874815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.784122:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06448380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554888960:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10254us (12049us total) trans 4294983879 rc 0/0 00000100:00100000:2.0:1789760794.784135:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16581 00000100:00100000:2.0:1789760794.784147:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738005203052 time 738019466418 state 0x2 ref 0 00000100:00100000:2.0:1789760794.789447:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760794.789631:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554889088 00000100:00100000:0.0:1789760794.789645:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554889088 00000100:00100000:0.0:1789760794.789721:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738005203052 time 738019466418 state 0x0 ref 1 00000100:00100000:0.0:1789760794.789741:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738019726834 time 738019466418 state 0x1 ref 1 00000100:00100000:0.0:1789760794.789910:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738019726834 time 738035526244 state 0x0 ref 1 00000100:00100000:0.0:1789760794.789927:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09310000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554889088:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760794.789997:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67874816, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x205f0 00010000:00010000:0.0:1789760794.790009:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f53 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.790032:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760794.790037:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.790045:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67874816->67878911] (req 67874816->67878911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.790062:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67874816->67878911] (req 67874816->67878911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.798462:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983880 is committed 00002000:00010000:0.0:1789760794.799625:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67874816 -> 67878912 00002000:00010000:0.0:1789760794.799634:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132592 -> 132600 00010000:00010000:0.0:1789760794.799652:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f53 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67874816->67878911] (req 67874816->67878911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.799669:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f53 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67874816->67878911] (req 67874816->67878911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.799680:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f53 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67874816->67878911] (req 67874816->67878911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.799689:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760794.799694:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f53 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67874816->67878911] (req 67874816->67878911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.799786:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f53 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67874816->67878911] (req 67874816->67878911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.799814:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f53 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67874816->67878911] (req 67874816->67878911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.799823:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f53 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67874816->67878911] (req 67874816->67878911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.800374:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09310000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554889088:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10442us (10934us total) trans 4294983880 rc 0/0 00000100:00100000:0.0:1789760794.800541:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16582 00000100:00100000:0.0:1789760794.800553:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738019726834 time 738035526244 state 0x2 ref 0 00000100:00100000:1.0:1789760794.807024:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760794.807175:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554889216 00000100:00100000:0.0:1789760794.807302:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554889216 00000100:00100000:0.0:1789760794.808154:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738019726834 time 738035526244 state 0x0 ref 1 00000100:00100000:0.0:1789760794.808182:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738035786660 time 738035526244 state 0x1 ref 1 00000100:00100000:0.0:1789760794.808364:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738035786660 time 738053981429 state 0x0 ref 1 00000100:00100000:0.0:1789760794.808470:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0644a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554889216:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760794.809207:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67878912, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x205f8 00010000:00010000:0.0:1789760794.809220:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f5a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.809467:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760794.809473:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.809481:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67878912->67883007] (req 67878912->67883007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.809499:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67878912->67883007] (req 67878912->67883007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.818634:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983881 is committed 00002000:00010000:0.0:1789760794.818828:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67878912 -> 67883008 00002000:00010000:0.0:1789760794.818834:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132600 -> 132608 00010000:00010000:0.0:1789760794.818853:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f5a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67878912->67883007] (req 67878912->67883007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.818867:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f5a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67878912->67883007] (req 67878912->67883007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.818876:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f5a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67878912->67883007] (req 67878912->67883007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.818884:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760794.818890:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f5a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67878912->67883007] (req 67878912->67883007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.818900:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f5a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67878912->67883007] (req 67878912->67883007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.818923:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f5a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67878912->67883007] (req 67878912->67883007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.818931:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c6f5a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67878912->67883007] (req 67878912->67883007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.819052:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0644a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554889216:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10668us (12104us total) trans 4294983881 rc 0/0 00000100:00100000:0.0:1789760794.819064:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16583 00000100:00100000:0.0:1789760794.819076:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738035786660 time 738053981429 state 0x2 ref 0 00000100:00100000:1.0:1789760794.823472:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760794.823568:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554889344 00000100:00100000:0.0:1789760794.823582:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554889344 00000100:00100000:0.0:1789760794.823669:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738035786660 time 738053981429 state 0x0 ref 1 00000100:00100000:0.0:1789760794.823689:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738054241845 time 738053981429 state 0x1 ref 1 00000100:00100000:0.0:1789760794.823726:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738054241845 time 738069344353 state 0x0 ref 1 00000100:00100000:0.0:1789760794.823740:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06449c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554889344:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760794.823803:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67883008, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20600 00010000:00010000:0.0:1789760794.823815:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f61 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.823834:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760794.823839:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.823846:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67883008->67887103] (req 67883008->67887103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.823863:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67883008->67887103] (req 67883008->67887103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.829126:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983882 is committed 00002000:00010000:0.0:1789760794.829494:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67883008 -> 67887104 00002000:00010000:0.0:1789760794.829501:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132608 -> 132616 00010000:00010000:0.0:1789760794.829651:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f61 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67883008->67887103] (req 67883008->67887103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.829674:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f61 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67883008->67887103] (req 67883008->67887103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.829685:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67883008->67887103] (req 67883008->67887103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.829694:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760794.829699:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f61 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67883008->67887103] (req 67883008->67887103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.829710:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f61 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67883008->67887103] (req 67883008->67887103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.829736:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f61 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67883008->67887103] (req 67883008->67887103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.829745:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c6f61 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67883008->67887103] (req 67883008->67887103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.829916:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06449c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554889344:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6169us (6451us total) trans 4294983882 rc 0/0 00000100:00100000:0.0:1789760794.829930:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16584 00000100:00100000:0.0:1789760794.829944:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738054241845 time 738069344353 state 0x2 ref 0 00000100:00100000:2.0:1789760794.833898:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760794.834155:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554889472 00000100:00100000:1.0:1789760794.834168:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554889472 00000100:00100000:1.0:1789760794.834254:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738054241845 time 738069344353 state 0x0 ref 1 00000100:00100000:1.0:1789760794.834275:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738069604769 time 738069344353 state 0x1 ref 1 00000100:00100000:1.0:1789760794.834349:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738069604769 time 738079966227 state 0x0 ref 1 00000100:00100000:1.0:1789760794.834365:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09310a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554889472:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760794.834464:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67887104, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20608 00010000:00010000:1.0:1789760794.834477:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6f68 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.834498:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760794.834504:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.834521:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6f68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67887104->67891199] (req 67887104->67891199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.834538:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6f68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67887104->67891199] (req 67887104->67891199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.840668:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983883 is committed 00002000:00010000:1.0:1789760794.840825:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67887104 -> 67891200 00002000:00010000:1.0:1789760794.840832:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132616 -> 132624 00010000:00010000:1.0:1789760794.840851:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6f68 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67887104->67891199] (req 67887104->67891199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.840869:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6f68 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67887104->67891199] (req 67887104->67891199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.840880:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6f68 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67887104->67891199] (req 67887104->67891199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.840888:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.840895:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6f68 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67887104->67891199] (req 67887104->67891199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.840905:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6f68 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67887104->67891199] (req 67887104->67891199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.840932:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6f68 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67887104->67891199] (req 67887104->67891199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.840945:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c6f68 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67887104->67891199] (req 67887104->67891199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.841065:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09310a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554889472:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6697us (7175us total) trans 4294983883 rc 0/0 00000100:00100000:1.0:1789760794.841075:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16585 00000100:00100000:1.0:1789760794.841085:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738069604769 time 738079966227 state 0x2 ref 0 00000100:00100000:1.0:1789760794.845086:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760794.845213:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554889600 00000100:00100000:2.0:1789760794.845225:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554889600 00000100:00100000:2.0:1789760794.845595:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738069604769 time 738079966227 state 0x0 ref 1 00000100:00100000:2.0:1789760794.845614:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738080226643 time 738079966227 state 0x1 ref 1 00000100:00100000:2.0:1789760794.845681:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738080226643 time 738091297904 state 0x0 ref 1 00000100:00100000:2.0:1789760794.845696:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06449880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554889600:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760794.845808:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67891200, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20610 00010000:00010000:2.0:1789760794.845823:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6f6f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.845844:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760794.845849:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.845860:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6f6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67891200->67895295] (req 67891200->67895295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.845876:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6f6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67891200->67895295] (req 67891200->67895295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.850872:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983884 is committed 00002000:00010000:2.0:1789760794.851111:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67891200 -> 67895296 00002000:00010000:2.0:1789760794.851117:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132624 -> 132632 00010000:00010000:2.0:1789760794.851135:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6f6f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67891200->67895295] (req 67891200->67895295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.851151:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6f6f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67891200->67895295] (req 67891200->67895295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.851167:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6f6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67891200->67895295] (req 67891200->67895295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.851174:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760794.851180:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6f6f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67891200->67895295] (req 67891200->67895295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.851191:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6f6f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67891200->67895295] (req 67891200->67895295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.851215:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6f6f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67891200->67895295] (req 67891200->67895295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.851224:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6f6f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67891200->67895295] (req 67891200->67895295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.851469:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06449880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554889600:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5769us (6391us total) trans 4294983884 rc 0/0 00000100:00100000:2.0:1789760794.851491:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16586 00000100:00100000:2.0:1789760794.851504:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738080226643 time 738091297904 state 0x2 ref 0 00000100:00100000:0.0:1789760794.859578:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760794.859695:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554889728 00000100:00100000:3.0:1789760794.859709:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554889728 00000100:00100000:3.0:1789760794.859785:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738080226643 time 738091297904 state 0x0 ref 1 00000100:00100000:3.0:1789760794.859805:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738091558320 time 738091297904 state 0x1 ref 1 00000100:00100000:3.0:1789760794.859835:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738091558320 time 738105452595 state 0x0 ref 1 00000100:00100000:3.0:1789760794.859849:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e57b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554889728:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760794.859922:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67895296, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20618 00010000:00010000:3.0:1789760794.859934:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f76 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.859953:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760794.859957:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.860544:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67895296->67899391] (req 67895296->67899391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.860566:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67895296->67899391] (req 67895296->67899391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.867235:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983885 is committed 00002000:00010000:3.0:1789760794.867437:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67895296 -> 67899392 00002000:00010000:3.0:1789760794.867445:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132632 -> 132640 00010000:00010000:3.0:1789760794.867463:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f76 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67895296->67899391] (req 67895296->67899391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.867482:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f76 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67895296->67899391] (req 67895296->67899391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.867492:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f76 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67895296->67899391] (req 67895296->67899391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.867499:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760794.867505:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f76 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67895296->67899391] (req 67895296->67899391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.867516:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f76 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67895296->67899391] (req 67895296->67899391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.867543:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f76 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67895296->67899391] (req 67895296->67899391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.867550:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f76 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67895296->67899391] (req 67895296->67899391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760794.867744:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e57b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554889728:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7887us (8173us total) trans 4294983885 rc 0/0 00000100:00100000:3.0:1789760794.867759:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16587 00000100:00100000:3.0:1789760794.867770:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738091558320 time 738105452595 state 0x2 ref 0 00000100:00100000:0.0:1789760794.872372:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760794.872466:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554889856 00000100:00100000:1.0:1789760794.872482:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554889856 00000100:00100000:1.0:1789760794.872566:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738091558320 time 738105452595 state 0x0 ref 1 00000100:00100000:1.0:1789760794.872594:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738105713011 time 738105452595 state 0x1 ref 1 00000100:00100000:1.0:1789760794.872637:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738105713011 time 738118255285 state 0x0 ref 1 00000100:00100000:1.0:1789760794.872652:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e57480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554889856:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760794.872714:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67899392, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20620 00010000:00010000:1.0:1789760794.872726:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f7d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.872745:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760794.872752:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.872759:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67899392->67903487] (req 67899392->67903487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.872774:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67899392->67903487] (req 67899392->67903487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.877820:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983886 is committed 00002000:00010000:1.0:1789760794.878210:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67899392 -> 67903488 00002000:00010000:1.0:1789760794.878217:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132640 -> 132648 00010000:00010000:1.0:1789760794.878237:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f7d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67899392->67903487] (req 67899392->67903487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.878254:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f7d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67899392->67903487] (req 67899392->67903487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.878263:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f7d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67899392->67903487] (req 67899392->67903487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.878272:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760794.878277:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f7d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67899392->67903487] (req 67899392->67903487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.878289:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f7d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67899392->67903487] (req 67899392->67903487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.878316:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f7d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67899392->67903487] (req 67899392->67903487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.878330:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c6f7d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67899392->67903487] (req 67899392->67903487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760794.878537:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e57480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554889856:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5879us (6171us total) trans 4294983886 rc 0/0 00000100:00100000:1.0:1789760794.878551:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16588 00000100:00100000:1.0:1789760794.878571:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738105713011 time 738118255285 state 0x2 ref 0 00000100:00100000:1.0:1789760794.885116:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760794.885203:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554889984 00000100:00100000:2.0:1789760794.885219:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554889984 00000100:00100000:2.0:1789760794.885296:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738105713011 time 738118255285 state 0x0 ref 1 00000100:00100000:2.0:1789760794.885319:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738118515701 time 738118255285 state 0x1 ref 1 00000100:00100000:2.0:1789760794.885690:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738118515701 time 738131305338 state 0x0 ref 1 00000100:00100000:2.0:1789760794.885707:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06448e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554889984:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760794.885911:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67903488, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20628 00010000:00010000:2.0:1789760794.885923:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6f84 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.885944:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760794.885949:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.885957:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6f84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67903488->67907583] (req 67903488->67907583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.885974:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6f84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67903488->67907583] (req 67903488->67907583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.893042:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983887 is committed 00002000:00010000:2.0:1789760794.893727:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67903488 -> 67907584 00002000:00010000:2.0:1789760794.893736:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132648 -> 132656 00010000:00010000:2.0:1789760794.893753:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6f84 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67903488->67907583] (req 67903488->67907583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.893772:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6f84 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67903488->67907583] (req 67903488->67907583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.893782:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6f84 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67903488->67907583] (req 67903488->67907583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.893791:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760794.893798:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6f84 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67903488->67907583] (req 67903488->67907583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.893808:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6f84 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67903488->67907583] (req 67903488->67907583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.893833:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6f84 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67903488->67907583] (req 67903488->67907583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.893842:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6f84 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67903488->67907583] (req 67903488->67907583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.893979:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06448e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554889984:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8267us (8876us total) trans 4294983887 rc 0/0 00000100:00100000:2.0:1789760794.893994:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16589 00000100:00100000:2.0:1789760794.894010:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738118515701 time 738131305338 state 0x2 ref 0 00000100:00100000:0.0:1789760794.898167:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760794.898303:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554890112 00000100:00100000:2.0:1789760794.898316:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554890112 00000100:00100000:2.0:1789760794.898430:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738118515701 time 738131305338 state 0x0 ref 1 00000100:00100000:2.0:1789760794.898450:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738131565754 time 738131305338 state 0x1 ref 1 00000100:00100000:2.0:1789760794.898492:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738131565754 time 738144109646 state 0x0 ref 1 00000100:00100000:2.0:1789760794.898511:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e54700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554890112:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760794.898575:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67907584, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20630 00010000:00010000:2.0:1789760794.898595:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6f8b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.898617:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760794.898622:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.898629:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6f8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67907584->67911679] (req 67907584->67911679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.898645:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6f8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67907584->67911679] (req 67907584->67911679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760794.906262:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983888 is committed 00002000:00010000:2.0:1789760794.910231:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67907584 -> 67911680 00002000:00010000:2.0:1789760794.910246:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132656 -> 132664 00010000:00010000:2.0:1789760794.910330:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6f8b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67907584->67911679] (req 67907584->67911679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.910347:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6f8b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67907584->67911679] (req 67907584->67911679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.910358:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6f8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67907584->67911679] (req 67907584->67911679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.910365:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760794.910371:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6f8b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67907584->67911679] (req 67907584->67911679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.910413:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6f8b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67907584->67911679] (req 67907584->67911679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.910440:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6f8b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67907584->67911679] (req 67907584->67911679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.910448:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c6f8b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67907584->67911679] (req 67907584->67911679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.910756:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554890112:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12067us (12429us total) trans 4294983888 rc 0/0 00000100:00100000:2.0:1789760794.910771:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16590 00000100:00100000:2.0:1789760794.910784:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738131565754 time 738144109646 state 0x2 ref 0 00000100:00100000:0.0:1789760794.918859:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760794.920939:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554890240 00000100:00100000:3.0:1789760794.920960:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554890240 00000100:00100000:3.0:1789760794.921293:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738131565754 time 738144109646 state 0x0 ref 1 00000100:00100000:3.0:1789760794.921312:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738144370062 time 738144109646 state 0x1 ref 1 00000100:00100000:3.0:1789760794.921430:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738144370062 time 738167046733 state 0x0 ref 1 00000100:00100000:3.0:1789760794.921445:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e57800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554890240:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760794.921502:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67911680, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20638 00010000:00010000:3.0:1789760794.921523:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f92 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.921543:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760794.921547:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760794.921554:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67911680->67915775] (req 67911680->67915775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760794.921570:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67911680->67915775] (req 67911680->67915775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.932279:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983889 is committed 00002000:00010000:2.0:1789760794.932573:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67911680 -> 67915776 00002000:00010000:2.0:1789760794.932579:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132664 -> 132672 00010000:00010000:2.0:1789760794.932606:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f92 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67911680->67915775] (req 67911680->67915775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.932623:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f92 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67911680->67915775] (req 67911680->67915775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.932634:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67911680->67915775] (req 67911680->67915775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.932646:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760794.932652:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f92 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67911680->67915775] (req 67911680->67915775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.932663:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f92 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67911680->67915775] (req 67911680->67915775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.932687:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f92 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67911680->67915775] (req 67911680->67915775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.932695:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c6f92 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67911680->67915775] (req 67911680->67915775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.932822:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e57800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554890240:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11373us (13970us total) trans 4294983889 rc 0/0 00000100:00100000:2.0:1789760794.932836:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16591 00000100:00100000:2.0:1789760794.932848:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738144370062 time 738167046733 state 0x2 ref 0 00000100:00100000:1.0:1789760794.938715:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760794.938895:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554890368 00000100:00100000:0.0:1789760794.938907:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554890368 00000100:00100000:0.0:1789760794.938993:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738144370062 time 738167046733 state 0x0 ref 1 00000100:00100000:0.0:1789760794.939017:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738167307149 time 738167046733 state 0x1 ref 1 00000100:00100000:0.0:1789760794.939054:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738167307149 time 738184672106 state 0x0 ref 1 00000100:00100000:0.0:1789760794.939068:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06449500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554890368:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760794.939129:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67915776, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20640 00010000:00010000:0.0:1789760794.939140:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6f99 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.939168:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760794.939174:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.939181:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6f99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67915776->67919871] (req 67915776->67919871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.939199:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6f99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67915776->67919871] (req 67915776->67919871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760794.947683:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983890 is committed 00002000:00010000:0.0:1789760794.947819:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67915776 -> 67919872 00002000:00010000:0.0:1789760794.947825:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132672 -> 132680 00010000:00010000:0.0:1789760794.947843:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6f99 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67915776->67919871] (req 67915776->67919871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.947857:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6f99 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67915776->67919871] (req 67915776->67919871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.947867:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6f99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67915776->67919871] (req 67915776->67919871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.947875:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760794.947881:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6f99 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67915776->67919871] (req 67915776->67919871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.947892:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6f99 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67915776->67919871] (req 67915776->67919871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.947916:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6f99 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67915776->67919871] (req 67915776->67919871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.947925:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6f99 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67915776->67919871] (req 67915776->67919871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.948164:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06449500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554890368:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9090us (9457us total) trans 4294983890 rc 0/0 00000100:00100000:0.0:1789760794.948177:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16592 00000100:00100000:0.0:1789760794.948188:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738167307149 time 738184672106 state 0x2 ref 0 00000100:00100000:0.0:1789760794.959814:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760794.959940:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554890496 00000100:00100000:2.0:1789760794.959954:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554890496 00000100:00100000:2.0:1789760794.960036:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738167307149 time 738184672106 state 0x0 ref 1 00000100:00100000:2.0:1789760794.960057:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738184932522 time 738184672106 state 0x1 ref 1 00000100:00100000:2.0:1789760794.960200:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738184932522 time 738205817058 state 0x0 ref 1 00000100:00100000:2.0:1789760794.960218:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554890496:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760794.960286:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67919872, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20648 00010000:00010000:2.0:1789760794.960299:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6fa0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.960321:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760794.960326:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760794.960334:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6fa0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67919872->67923967] (req 67919872->67923967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.960360:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6fa0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67919872->67923967] (req 67919872->67923967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760794.970886:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983891 is committed 00002000:00010000:2.0:1789760794.972441:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67919872 -> 67923968 00002000:00010000:2.0:1789760794.972452:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132680 -> 132688 00010000:00010000:2.0:1789760794.972730:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6fa0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67919872->67923967] (req 67919872->67923967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.972746:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6fa0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67919872->67923967] (req 67919872->67923967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.972757:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6fa0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67919872->67923967] (req 67919872->67923967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.972765:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760794.972773:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6fa0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67919872->67923967] (req 67919872->67923967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.972783:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6fa0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67919872->67923967] (req 67919872->67923967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.972812:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6fa0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67919872->67923967] (req 67919872->67923967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760794.972820:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c6fa0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67919872->67923967] (req 67919872->67923967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760794.972976:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554890496:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12754us (13172us total) trans 4294983891 rc 0/0 00000100:00100000:2.0:1789760794.972989:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16593 00000100:00100000:2.0:1789760794.973000:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738184932522 time 738205817058 state 0x2 ref 0 00000100:00100000:2.0:1789760794.978246:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760794.978565:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554890624 00000100:00100000:0.0:1789760794.978578:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554890624 00000100:00100000:0.0:1789760794.978668:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738184932522 time 738205817058 state 0x0 ref 1 00000100:00100000:0.0:1789760794.978688:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738206077474 time 738205817058 state 0x1 ref 1 00000100:00100000:0.0:1789760794.978723:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738206077474 time 738224341632 state 0x0 ref 1 00000100:00100000:0.0:1789760794.978736:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09312a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554890624:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760794.978800:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67923968, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20650 00010000:00010000:0.0:1789760794.978811:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6fa7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.978831:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760794.978835:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760794.978843:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6fa7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67923968->67928063] (req 67923968->67928063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.978859:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6fa7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67923968->67928063] (req 67923968->67928063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760794.985317:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983892 is committed 00002000:00010000:0.0:1789760794.985603:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67923968 -> 67928064 00002000:00010000:0.0:1789760794.985610:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132688 -> 132696 00010000:00010000:0.0:1789760794.985627:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6fa7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67923968->67928063] (req 67923968->67928063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.985643:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6fa7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67923968->67928063] (req 67923968->67928063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.985653:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6fa7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67923968->67928063] (req 67923968->67928063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.985661:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760794.985667:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6fa7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67923968->67928063] (req 67923968->67928063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.985677:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6fa7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67923968->67928063] (req 67923968->67928063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.985703:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6fa7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67923968->67928063] (req 67923968->67928063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760794.985711:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6fa7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67923968->67928063] (req 67923968->67928063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760794.985876:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09312a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554890624:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7135us (7638us total) trans 4294983892 rc 0/0 00000100:00100000:0.0:1789760794.985887:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16594 00000100:00100000:0.0:1789760794.985899:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738206077474 time 738224341632 state 0x2 ref 0 00000100:00100000:1.0:1789760794.997168:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760794.997274:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554890752 00000100:00100000:0.0:1789760794.997291:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554890752 00000100:00100000:0.0:1789760794.997428:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738206077474 time 738224341632 state 0x0 ref 1 00000100:00100000:0.0:1789760794.997447:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738224602048 time 738224341632 state 0x1 ref 1 00000100:00100000:1.0:1789760794.997627:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738224602048 time 738243242367 state 0x0 ref 1 00000100:00100000:1.0:1789760794.997647:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0644b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554890752:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760794.997711:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67928064, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20658 00010000:00010000:1.0:1789760794.997722:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6fae lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.997741:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760794.997746:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760794.997752:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6fae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67928064->67932159] (req 67928064->67932159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760794.997773:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6fae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67928064->67932159] (req 67928064->67932159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.006613:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983893 is committed 00002000:00010000:1.0:1789760795.006925:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67928064 -> 67932160 00002000:00010000:1.0:1789760795.006932:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132696 -> 132704 00010000:00010000:1.0:1789760795.006950:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6fae lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67928064->67932159] (req 67928064->67932159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.006965:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6fae lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67928064->67932159] (req 67928064->67932159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.006975:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6fae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67928064->67932159] (req 67928064->67932159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.006983:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.006989:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6fae lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67928064->67932159] (req 67928064->67932159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.007000:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6fae lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67928064->67932159] (req 67928064->67932159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.007087:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6fae lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67928064->67932159] (req 67928064->67932159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.007096:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c6fae lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67928064->67932159] (req 67928064->67932159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.007284:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0644b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554890752:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9635us (10125us total) trans 4294983893 rc 0/0 00000100:00100000:1.0:1789760795.007298:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16595 00000100:00100000:1.0:1789760795.007309:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738224602048 time 738243242367 state 0x2 ref 0 00000100:00100000:1.0:1789760795.011263:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760795.011657:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554890880 00000100:00100000:0.0:1789760795.011670:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554890880 00000100:00080000:3.0:1789760795.011729:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760795.011762:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738224602048 time 738243242367 state 0x0 ref 1 00000100:00100000:0.0:1789760795.011783:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738243502783 time 738243242367 state 0x1 ref 1 00000100:00100000:0.0:1789760795.011817:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738243502783 time 738257435178 state 0x0 ref 1 00000100:00100000:0.0:1789760795.011838:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a2f2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554890880:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760795.011900:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67932160, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20660 00010000:00010000:0.0:1789760795.011912:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6fb5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.011931:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760795.011936:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.011944:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6fb5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67932160->67936255] (req 67932160->67936255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.011960:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6fb5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67932160->67936255] (req 67932160->67936255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.019196:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983894 is committed 00002000:00010000:0.0:1789760795.019331:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67932160 -> 67936256 00002000:00010000:0.0:1789760795.019338:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132704 -> 132712 00010000:00010000:0.0:1789760795.019356:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6fb5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67932160->67936255] (req 67932160->67936255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.019372:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6fb5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67932160->67936255] (req 67932160->67936255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.019414:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6fb5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67932160->67936255] (req 67932160->67936255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.019423:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760795.019429:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6fb5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67932160->67936255] (req 67932160->67936255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.019438:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6fb5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67932160->67936255] (req 67932160->67936255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.019464:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6fb5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67932160->67936255] (req 67932160->67936255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.019473:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c6fb5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67932160->67936255] (req 67932160->67936255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.019595:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a2f2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554890880:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7751us (8340us total) trans 4294983894 rc 0/0 00000100:00100000:0.0:1789760795.019608:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16596 00000100:00100000:0.0:1789760795.019620:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738243502783 time 738257435178 state 0x2 ref 0 00000100:00100000:2.0:1789760795.026023:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789760795.026053:0:3649:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03e57b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3649:1876699573923200:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789760795.026322:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789760795.027231:0:8439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573923200 00000100:00100000:1.0:1789760795.027243:0:8439:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573923200 00000100:00100000:1.0:1789760795.027398:0:8439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 372 00000100:00100000:1.0:1789760795.027411:0:8439:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09312680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3649:x1876699573923200:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789760795.027443:0:8439:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484916928 pending 0 free 3855937536 avail 3621920768 00000100:00100000:1.0:1789760795.027517:0:8439:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09312680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3649:x1876699573923200:12345-0@lo:13:osp-pre-0-0.0 Request processed in 109us (1197us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760795.027525:0:8439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 372 00000100:00080000:2.0:1789760795.027582:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760795.028742:0:3649:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03e57b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3649:1876699573923200:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789760795.028867:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760795.028959:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554891008 00000100:00100000:2.0:1789760795.029192:0:3648:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789760795.029207:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb09311c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3648:1876699573923328:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1789760795.029242:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554891008 00000100:00100000:2.0:1789760795.029257:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760795.029299:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738243502783 time 738257435178 state 0x0 ref 1 00000100:00100000:3.0:1789760795.029318:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738257695594 time 738257435178 state 0x1 ref 1 00000100:00100000:3.0:1789760795.029358:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738257695594 time 738274976267 state 0x0 ref 1 00000100:00100000:3.0:1789760795.029371:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e57b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554891008:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760795.029451:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67936256, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20668 00010000:00010000:3.0:1789760795.029463:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6fbc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.029483:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760795.029488:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.029496:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6fbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67936256->67940351] (req 67936256->67940351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.029513:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6fbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67936256->67940351] (req 67936256->67940351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.030358:0:8439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573923328 00000100:00100000:1.0:1789760795.030369:0:8439:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573923328 00000100:00100000:1.0:1789760795.030435:0:8439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 373 00000100:00100000:1.0:1789760795.030444:0:8439:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09310380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3648:x1876699573923328:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1789760795.030461:0:8439:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 484945600 pending 28672 free 3855937536 avail 3621920768 00000100:00080000:0.0:1789760795.030481:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789760795.030546:0:8439:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09310380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3648:x1876699573923328:12345-0@lo:13:osp-pre-0-1.0 Request processed in 102us (1287us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760795.030553:0:8439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 373 00000100:00100000:0.0:1789760795.030665:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb09311c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3648:1876699573923328:0@lo:13:osp-pre-0-1.0 00000001:00080000:0.0:1789760795.044357:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983895 is committed 00002000:00010000:2.0:1789760795.045738:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67936256 -> 67940352 00002000:00010000:2.0:1789760795.045748:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132712 -> 132720 00010000:00010000:2.0:1789760795.045767:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6fbc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67936256->67940351] (req 67936256->67940351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.045783:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6fbc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67936256->67940351] (req 67936256->67940351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.045792:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6fbc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67936256->67940351] (req 67936256->67940351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.045801:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760795.045806:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6fbc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67936256->67940351] (req 67936256->67940351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.045816:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6fbc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67936256->67940351] (req 67936256->67940351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.045842:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6fbc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67936256->67940351] (req 67936256->67940351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.045850:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c6fbc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67936256->67940351] (req 67936256->67940351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.046187:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e57b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554891008:12345-192.168.203.22@tcp:4:dd.0 Request processed in 16810us (17315us total) trans 4294983895 rc 0/0 00000100:00100000:2.0:1789760795.046200:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16597 00000100:00100000:2.0:1789760795.046212:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738257695594 time 738274976267 state 0x2 ref 0 00000100:00100000:1.0:1789760795.056868:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760795.056993:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554891136 00000100:00100000:2.0:1789760795.057005:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554891136 00000100:00100000:2.0:1789760795.057079:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738257695594 time 738274976267 state 0x0 ref 1 00000100:00100000:2.0:1789760795.057099:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738275236683 time 738274976267 state 0x1 ref 1 00000100:00100000:2.0:1789760795.057136:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738275236683 time 738302754392 state 0x0 ref 1 00000100:00100000:2.0:1789760795.057155:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06448a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554891136:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760795.057219:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67940352, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20670 00010000:00010000:2.0:1789760795.057230:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6fc3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.057250:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760795.057255:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.057263:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6fc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67940352->67944447] (req 67940352->67944447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.057280:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6fc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67940352->67944447] (req 67940352->67944447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.065148:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983896 is committed 00002000:00010000:2.0:1789760795.066221:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67940352 -> 67944448 00002000:00010000:2.0:1789760795.066230:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132720 -> 132728 00010000:00010000:2.0:1789760795.066246:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6fc3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67940352->67944447] (req 67940352->67944447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.066267:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6fc3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67940352->67944447] (req 67940352->67944447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.066276:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6fc3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67940352->67944447] (req 67940352->67944447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.066284:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760795.066290:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6fc3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67940352->67944447] (req 67940352->67944447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.066300:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6fc3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67940352->67944447] (req 67940352->67944447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.066324:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6fc3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67940352->67944447] (req 67940352->67944447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.066332:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c6fc3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67940352->67944447] (req 67940352->67944447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.066616:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06448a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554891136:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9460us (9757us total) trans 4294983896 rc 0/0 00000100:00100000:2.0:1789760795.066636:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16598 00000100:00100000:2.0:1789760795.066650:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738275236683 time 738302754392 state 0x2 ref 0 00000100:00100000:1.0:1789760795.073421:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760795.073543:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554891264 00000100:00100000:2.0:1789760795.073558:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554891264 00000100:00100000:2.0:1789760795.073764:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738275236683 time 738302754392 state 0x0 ref 1 00000100:00100000:2.0:1789760795.073786:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738303014808 time 738302754392 state 0x1 ref 1 00000100:00100000:2.0:1789760795.073868:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738303014808 time 738319486138 state 0x0 ref 1 00000100:00100000:2.0:1789760795.073883:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06449f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554891264:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760795.073960:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67944448, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20678 00010000:00010000:2.0:1789760795.073972:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6fca lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.073992:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760795.073997:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.074005:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6fca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67944448->67948543] (req 67944448->67948543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.074022:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6fca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67944448->67948543] (req 67944448->67948543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.081574:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983897 is committed 00002000:00010000:2.0:1789760795.082287:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67944448 -> 67948544 00002000:00010000:2.0:1789760795.082295:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132728 -> 132736 00010000:00010000:2.0:1789760795.082312:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6fca lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67944448->67948543] (req 67944448->67948543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.082326:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6fca lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67944448->67948543] (req 67944448->67948543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.082352:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6fca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67944448->67948543] (req 67944448->67948543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.082361:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760795.082366:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6fca lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67944448->67948543] (req 67944448->67948543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.082399:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6fca lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67944448->67948543] (req 67944448->67948543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.082430:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6fca lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67944448->67948543] (req 67944448->67948543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.082438:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c6fca lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67944448->67948543] (req 67944448->67948543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.082677:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06449f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554891264:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8789us (9263us total) trans 4294983897 rc 0/0 00000100:00100000:2.0:1789760795.082690:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16599 00000100:00100000:2.0:1789760795.082704:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738303014808 time 738319486138 state 0x2 ref 0 00000100:00100000:0.0:1789760795.087904:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760795.088359:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554891392 00000100:00100000:1.0:1789760795.088373:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554891392 00000100:00100000:1.0:1789760795.088469:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738303014808 time 738319486138 state 0x0 ref 1 00000100:00100000:1.0:1789760795.088489:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738319746554 time 738319486138 state 0x1 ref 1 00000100:00100000:1.0:1789760795.088528:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738319746554 time 738334146185 state 0x0 ref 1 00000100:00100000:1.0:1789760795.088543:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e57480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554891392:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760795.088616:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67948544, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20680 00010000:00010000:1.0:1789760795.088828:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6fd1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.088850:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760795.088856:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.088864:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6fd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67948544->67952639] (req 67948544->67952639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.088881:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6fd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67948544->67952639] (req 67948544->67952639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.097774:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983898 is committed 00002000:00010000:1.0:1789760795.097944:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67948544 -> 67952640 00002000:00010000:1.0:1789760795.097951:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132736 -> 132744 00010000:00010000:1.0:1789760795.097969:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6fd1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67948544->67952639] (req 67948544->67952639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.097985:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6fd1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67948544->67952639] (req 67948544->67952639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.098003:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6fd1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67948544->67952639] (req 67948544->67952639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.098012:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.098018:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6fd1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67948544->67952639] (req 67948544->67952639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.098029:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6fd1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67948544->67952639] (req 67948544->67952639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.098054:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6fd1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67948544->67952639] (req 67948544->67952639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.098062:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c6fd1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67948544->67952639] (req 67948544->67952639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.098792:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e57480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554891392:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10240us (10902us total) trans 4294983898 rc 0/0 00000100:00100000:1.0:1789760795.098809:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16600 00000100:00100000:1.0:1789760795.098821:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738319746554 time 738334146185 state 0x2 ref 0 00000100:00100000:1.0:1789760795.103063:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760795.103484:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554891520 00000100:00100000:0.0:1789760795.103498:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554891520 00000100:00100000:0.0:1789760795.103568:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738319746554 time 738334146185 state 0x0 ref 1 00000100:00100000:0.0:1789760795.103601:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738334406601 time 738334146185 state 0x1 ref 1 00000100:00100000:0.0:1789760795.103639:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738334406601 time 738349256749 state 0x0 ref 1 00000100:00100000:0.0:1789760795.103701:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0644b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554891520:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760795.103764:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67952640, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20688 00010000:00010000:0.0:1789760795.103775:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fd8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.103794:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760795.103799:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.103806:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67952640->67956735] (req 67952640->67956735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.103822:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67952640->67956735] (req 67952640->67956735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.112362:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983899 is committed 00002000:00010000:0.0:1789760795.112850:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67952640 -> 67956736 00002000:00010000:0.0:1789760795.112856:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132744 -> 132752 00010000:00010000:0.0:1789760795.112873:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fd8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67952640->67956735] (req 67952640->67956735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.112889:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fd8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67952640->67956735] (req 67952640->67956735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.112898:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fd8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67952640->67956735] (req 67952640->67956735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.112906:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760795.112912:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fd8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67952640->67956735] (req 67952640->67956735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.112921:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fd8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67952640->67956735] (req 67952640->67956735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.112955:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fd8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67952640->67956735] (req 67952640->67956735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.112970:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fd8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67952640->67956735] (req 67952640->67956735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.113091:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0644b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554891520:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9434us (10038us total) trans 4294983899 rc 0/0 00000100:00100000:0.0:1789760795.113105:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16601 00000100:00100000:0.0:1789760795.113117:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738334406601 time 738349256749 state 0x2 ref 0 00000100:00100000:2.0:1789760795.121833:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760795.121988:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554891648 00000100:00100000:0.0:1789760795.122000:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554891648 00000100:00100000:0.0:1789760795.122073:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738334406601 time 738349256749 state 0x0 ref 1 00000100:00100000:0.0:1789760795.122091:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738349517165 time 738349256749 state 0x1 ref 1 00000100:00100000:0.0:1789760795.122127:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738349517165 time 738367746197 state 0x0 ref 1 00000100:00100000:0.0:1789760795.122141:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09310700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554891648:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760795.122200:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67956736, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20690 00010000:00010000:0.0:1789760795.122211:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fdf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.122230:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760795.122235:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.122242:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fdf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67956736->67960831] (req 67956736->67960831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.122265:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fdf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67956736->67960831] (req 67956736->67960831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.134224:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983900 is committed 00002000:00010000:0.0:1789760795.135621:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67956736 -> 67960832 00002000:00010000:0.0:1789760795.135639:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132752 -> 132760 00010000:00010000:0.0:1789760795.135658:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fdf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67956736->67960831] (req 67956736->67960831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.135673:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fdf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67956736->67960831] (req 67956736->67960831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.135683:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fdf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67956736->67960831] (req 67956736->67960831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.135692:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760795.135697:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fdf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67956736->67960831] (req 67956736->67960831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.135707:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fdf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67956736->67960831] (req 67956736->67960831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.135731:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fdf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67956736->67960831] (req 67956736->67960831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.135739:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fdf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67956736->67960831] (req 67956736->67960831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.135858:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09310700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554891648:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13712us (14085us total) trans 4294983900 rc 0/0 00000100:00100000:0.0:1789760795.135872:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16602 00000100:00100000:0.0:1789760795.135885:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738349517165 time 738367746197 state 0x2 ref 0 00000100:00100000:2.0:1789760795.144947:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760795.145080:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554891776 00000100:00100000:1.0:1789760795.145093:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554891776 00000100:00100000:1.0:1789760795.145171:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738349517165 time 738367746197 state 0x0 ref 1 00000100:00100000:1.0:1789760795.145194:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738368006613 time 738367746197 state 0x1 ref 1 00000100:00100000:1.0:1789760795.145231:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738368006613 time 738390849625 state 0x0 ref 1 00000100:00100000:1.0:1789760795.145246:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09313480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554891776:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760795.145304:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67960832, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20698 00010000:00010000:1.0:1789760795.145315:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6fe6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.145345:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760795.145351:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.145358:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6fe6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67960832->67964927] (req 67960832->67964927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.145375:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6fe6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67960832->67964927] (req 67960832->67964927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.154171:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983901 is committed 00002000:00010000:1.0:1789760795.154331:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67960832 -> 67964928 00002000:00010000:1.0:1789760795.154337:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132760 -> 132768 00010000:00010000:1.0:1789760795.154353:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6fe6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67960832->67964927] (req 67960832->67964927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.154369:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6fe6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67960832->67964927] (req 67960832->67964927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.154423:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6fe6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67960832->67964927] (req 67960832->67964927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.154433:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.154439:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6fe6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67960832->67964927] (req 67960832->67964927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.154449:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6fe6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67960832->67964927] (req 67960832->67964927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.154473:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6fe6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67960832->67964927] (req 67960832->67964927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.154482:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c6fe6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67960832->67964927] (req 67960832->67964927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.154786:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09313480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554891776:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9535us (9846us total) trans 4294983901 rc 0/0 00000100:00100000:1.0:1789760795.154799:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16603 00000100:00100000:1.0:1789760795.154810:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738368006613 time 738390849625 state 0x2 ref 0 00000100:00100000:1.0:1789760795.160148:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760795.160247:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554891904 00000100:00100000:0.0:1789760795.160260:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554891904 00000100:00100000:0.0:1789760795.160334:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738368006613 time 738390849625 state 0x0 ref 1 00000100:00100000:0.0:1789760795.160355:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738391110041 time 738390849625 state 0x1 ref 1 00000100:00100000:0.0:1789760795.160424:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738391110041 time 738406042111 state 0x0 ref 1 00000100:00100000:0.0:1789760795.160438:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06448380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554891904:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760795.160503:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67964928, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x206a0 00010000:00010000:0.0:1789760795.160513:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fed lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.160532:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760795.160538:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.160546:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67964928->67969023] (req 67964928->67969023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.160562:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67964928->67969023] (req 67964928->67969023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.171678:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983902 is committed 00002000:00010000:1.0:1789760795.171944:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67964928 -> 67969024 00002000:00010000:1.0:1789760795.171951:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132768 -> 132776 00010000:00010000:1.0:1789760795.171969:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fed lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67964928->67969023] (req 67964928->67969023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.171986:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fed lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67964928->67969023] (req 67964928->67969023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.171997:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67964928->67969023] (req 67964928->67969023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.172005:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.172013:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fed lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67964928->67969023] (req 67964928->67969023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.172024:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fed lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67964928->67969023] (req 67964928->67969023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.172061:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fed lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67964928->67969023] (req 67964928->67969023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.172074:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c6fed lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67964928->67969023] (req 67964928->67969023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.172217:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06448380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554891904:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11773us (12076us total) trans 4294983902 rc 0/0 00000100:00100000:1.0:1789760795.172231:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16604 00000100:00100000:1.0:1789760795.172244:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738391110041 time 738406042111 state 0x2 ref 0 00000100:00100000:2.0:1789760795.177415:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760795.177577:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554892032 00000100:00100000:0.0:1789760795.177600:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554892032 00000100:00100000:0.0:1789760795.177673:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738391110041 time 738406042111 state 0x0 ref 1 00000100:00100000:0.0:1789760795.177695:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738406302527 time 738406042111 state 0x1 ref 1 00000100:00100000:0.0:1789760795.177731:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738406302527 time 738423348541 state 0x0 ref 1 00000100:00100000:0.0:1789760795.177746:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09311f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554892032:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760795.177802:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67969024, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x206a8 00010000:00010000:0.0:1789760795.177813:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6ff4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.177833:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760795.177838:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.177846:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6ff4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67969024->67973119] (req 67969024->67973119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.177863:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6ff4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67969024->67973119] (req 67969024->67973119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.184041:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983903 is committed 00002000:00010000:0.0:1789760795.184182:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67969024 -> 67973120 00002000:00010000:0.0:1789760795.184189:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132776 -> 132784 00010000:00010000:0.0:1789760795.184206:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6ff4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67969024->67973119] (req 67969024->67973119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.184222:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6ff4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67969024->67973119] (req 67969024->67973119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.184233:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6ff4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67969024->67973119] (req 67969024->67973119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.184243:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760795.184248:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6ff4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67969024->67973119] (req 67969024->67973119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.184259:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6ff4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67969024->67973119] (req 67969024->67973119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.184283:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6ff4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67969024->67973119] (req 67969024->67973119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.184291:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6ff4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67969024->67973119] (req 67969024->67973119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.184656:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09311f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554892032:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6905us (7279us total) trans 4294983903 rc 0/0 00000100:00100000:0.0:1789760795.184671:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16605 00000100:00100000:0.0:1789760795.184683:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738406302527 time 738423348541 state 0x2 ref 0 00000100:00100000:2.0:1789760795.187571:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760795.189631:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554892160 00000100:00100000:0.0:1789760795.189651:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554892160 00000100:00100000:0.0:1789760795.189731:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738406302527 time 738423348541 state 0x0 ref 1 00000100:00100000:0.0:1789760795.189750:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738423608957 time 738423348541 state 0x1 ref 1 00000100:00100000:0.0:1789760795.189839:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738423608957 time 738435456163 state 0x0 ref 1 00000100:00100000:0.0:1789760795.189858:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09310e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554892160:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760795.189919:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67973120, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x206b0 00010000:00010000:0.0:1789760795.189929:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6ffb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.189951:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760795.189956:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.189964:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6ffb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67973120->67977215] (req 67973120->67977215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.189981:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6ffb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67973120->67977215] (req 67973120->67977215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.195940:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983904 is committed 00002000:00010000:0.0:1789760795.196136:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67973120 -> 67977216 00002000:00010000:0.0:1789760795.196143:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132784 -> 132792 00010000:00010000:0.0:1789760795.196159:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6ffb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67973120->67977215] (req 67973120->67977215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.196175:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6ffb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67973120->67977215] (req 67973120->67977215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.196186:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6ffb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67973120->67977215] (req 67973120->67977215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.196195:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760795.196202:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6ffb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67973120->67977215] (req 67973120->67977215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.196213:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6ffb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67973120->67977215] (req 67973120->67977215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.196239:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6ffb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67973120->67977215] (req 67973120->67977215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.196248:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c6ffb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67973120->67977215] (req 67973120->67977215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.197714:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09310e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554892160:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7842us (10140us total) trans 4294983904 rc 0/0 00000100:00100000:0.0:1789760795.197733:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16606 00000100:00100000:0.0:1789760795.197747:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738423608957 time 738435456163 state 0x2 ref 0 00000100:00100000:1.0:1789760795.205542:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760795.205976:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554892288 00000100:00100000:2.0:1789760795.205990:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554892288 00000100:00100000:2.0:1789760795.206623:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738423608957 time 738435456163 state 0x0 ref 1 00000100:00100000:2.0:1789760795.206644:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738435716579 time 738435456163 state 0x1 ref 1 00000100:00100000:2.0:1789760795.206686:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738435716579 time 738452304652 state 0x0 ref 1 00000100:00100000:2.0:1789760795.206703:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0644a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554892288:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760795.206766:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67977216, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x206b8 00010000:00010000:2.0:1789760795.206778:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7002 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.206802:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760795.206807:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.206815:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7002 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67977216->67981311] (req 67977216->67981311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.206832:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7002 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67977216->67981311] (req 67977216->67981311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760795.218542:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983905 is committed 00010000:00010000:3.0:1789760795.218706:0:6508:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760795.218820:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67977216 -> 67981312 00002000:00010000:2.0:1789760795.218826:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132792 -> 132800 00010000:00010000:2.0:1789760795.218842:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7002 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67977216->67981311] (req 67977216->67981311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.218864:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7002 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67977216->67981311] (req 67977216->67981311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.218875:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7002 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67977216->67981311] (req 67977216->67981311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.218884:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760795.218890:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7002 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67977216->67981311] (req 67977216->67981311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.218902:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7002 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67977216->67981311] (req 67977216->67981311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.218927:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7002 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67977216->67981311] (req 67977216->67981311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.218936:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7002 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67977216->67981311] (req 67977216->67981311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.220224:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0644a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554892288:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13512us (14684us total) trans 4294983905 rc 0/0 00000100:00100000:2.0:1789760795.220243:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16607 00000100:00100000:2.0:1789760795.220256:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738435716579 time 738452304652 state 0x2 ref 0 00000100:00100000:2.0:1789760795.224176:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760795.224404:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554892416 00000100:00100000:1.0:1789760795.224417:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554892416 00000100:00100000:1.0:1789760795.224488:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738435716579 time 738452304652 state 0x0 ref 1 00000100:00100000:1.0:1789760795.224506:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738452565068 time 738452304652 state 0x1 ref 1 00000100:00100000:1.0:1789760795.224529:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738452565068 time 738470148455 state 0x0 ref 1 00000100:00100000:1.0:1789760795.224542:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09311880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554892416:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760795.224609:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67981312, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x206c0 00010000:00010000:1.0:1789760795.224621:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7009 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.224639:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760795.224644:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.224651:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7009 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67981312->67985407] (req 67981312->67985407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.224666:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7009 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67981312->67985407] (req 67981312->67985407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.232299:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983906 is committed 00002000:00010000:1.0:1789760795.232566:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67981312 -> 67985408 00002000:00010000:1.0:1789760795.232573:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132800 -> 132808 00010000:00010000:1.0:1789760795.232600:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7009 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67981312->67985407] (req 67981312->67985407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.232615:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7009 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67981312->67985407] (req 67981312->67985407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.232630:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7009 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67981312->67985407] (req 67981312->67985407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.232639:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.232645:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7009 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67981312->67985407] (req 67981312->67985407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.232656:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7009 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67981312->67985407] (req 67981312->67985407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.232681:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7009 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67981312->67985407] (req 67981312->67985407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.232690:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7009 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67981312->67985407] (req 67981312->67985407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.232818:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09311880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554892416:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8270us (8648us total) trans 4294983906 rc 0/0 00000100:00100000:1.0:1789760795.232831:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16608 00000100:00100000:1.0:1789760795.232843:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738452565068 time 738470148455 state 0x2 ref 0 00000100:00100000:2.0:1789760795.237721:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760795.237952:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554892544 00000100:00100000:0.0:1789760795.238080:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554892544 00000100:00100000:0.0:1789760795.238159:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738452565068 time 738470148455 state 0x0 ref 1 00000100:00100000:0.0:1789760795.238179:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738470408871 time 738470148455 state 0x1 ref 1 00000100:00100000:0.0:1789760795.238254:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738470408871 time 738483871576 state 0x0 ref 1 00000100:00100000:0.0:1789760795.238268:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09313800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554892544:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760795.238339:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67985408, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x206c8 00010000:00010000:0.0:1789760795.238367:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7010 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.238524:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760795.238530:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.238537:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7010 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67985408->67989503] (req 67985408->67989503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.238555:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7010 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67985408->67989503] (req 67985408->67989503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.245791:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983907 is committed 00002000:00010000:1.0:1789760795.245929:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67985408 -> 67989504 00002000:00010000:1.0:1789760795.245935:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132808 -> 132816 00010000:00010000:1.0:1789760795.245954:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7010 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67985408->67989503] (req 67985408->67989503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.245979:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7010 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67985408->67989503] (req 67985408->67989503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.245992:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7010 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67985408->67989503] (req 67985408->67989503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.246001:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.246008:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7010 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67985408->67989503] (req 67985408->67989503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.246019:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7010 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67985408->67989503] (req 67985408->67989503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.246162:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7010 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67985408->67989503] (req 67985408->67989503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.246170:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7010 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67985408->67989503] (req 67985408->67989503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.246610:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09313800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554892544:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8336us (8897us total) trans 4294983907 rc 0/0 00000100:00100000:1.0:1789760795.246623:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16609 00000100:00100000:1.0:1789760795.246635:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738470408871 time 738483871576 state 0x2 ref 0 00000100:00100000:2.0:1789760795.257045:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760795.257363:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554892672 00000100:00100000:0.0:1789760795.257533:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554892672 00000100:00100000:1.0:1789760795.257781:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738470408871 time 738483871576 state 0x0 ref 1 00000100:00100000:1.0:1789760795.257803:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738484131992 time 738483871576 state 0x1 ref 1 00000100:00100000:1.0:1789760795.257844:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738484131992 time 738503462169 state 0x0 ref 1 00000100:00100000:1.0:1789760795.257858:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09311c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554892672:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760795.257919:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67989504, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x206d0 00010000:00010000:1.0:1789760795.257930:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7017 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.257960:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760795.257965:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.257973:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7017 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67989504->67993599] (req 67989504->67993599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.257990:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7017 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67989504->67993599] (req 67989504->67993599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.266485:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983908 is committed 00002000:00010000:1.0:1789760795.266767:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67989504 -> 67993600 00002000:00010000:1.0:1789760795.266774:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132816 -> 132824 00010000:00010000:1.0:1789760795.266793:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7017 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67989504->67993599] (req 67989504->67993599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.266809:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7017 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67989504->67993599] (req 67989504->67993599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.266822:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7017 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67989504->67993599] (req 67989504->67993599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.266831:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.266837:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7017 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67989504->67993599] (req 67989504->67993599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.266847:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7017 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67989504->67993599] (req 67989504->67993599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.266870:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7017 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67989504->67993599] (req 67989504->67993599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.266880:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7017 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67989504->67993599] (req 67989504->67993599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.267011:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09311c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554892672:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9147us (9971us total) trans 4294983908 rc 0/0 00000100:00100000:1.0:1789760795.267023:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16610 00000100:00100000:1.0:1789760795.267035:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738484131992 time 738503462169 state 0x2 ref 0 00000100:00100000:2.0:1789760795.272508:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760795.272753:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554892800 00000100:00100000:1.0:1789760795.272767:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554892800 00000100:00100000:1.0:1789760795.272845:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738484131992 time 738503462169 state 0x0 ref 1 00000100:00100000:1.0:1789760795.272865:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738503722585 time 738503462169 state 0x1 ref 1 00000100:00100000:1.0:1789760795.272950:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738503722585 time 738518566865 state 0x0 ref 1 00000100:00100000:1.0:1789760795.272980:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09310000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554892800:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760795.273046:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67993600, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x206d8 00010000:00010000:1.0:1789760795.273058:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c701e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.273079:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760795.273084:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.273092:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c701e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67993600->67997695] (req 67993600->67997695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.273109:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c701e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67993600->67997695] (req 67993600->67997695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.278851:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983909 is committed 00002000:00010000:1.0:1789760795.279105:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67993600 -> 67997696 00002000:00010000:1.0:1789760795.279112:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132824 -> 132832 00010000:00010000:1.0:1789760795.279130:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c701e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67993600->67997695] (req 67993600->67997695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.279146:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c701e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67993600->67997695] (req 67993600->67997695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.279156:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c701e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67993600->67997695] (req 67993600->67997695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.279164:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.279172:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c701e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67993600->67997695] (req 67993600->67997695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.279182:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c701e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67993600->67997695] (req 67993600->67997695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.279207:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c701e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67993600->67997695] (req 67993600->67997695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.279215:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c701e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67993600->67997695] (req 67993600->67997695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.279633:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09310000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554892800:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6642us (7125us total) trans 4294983909 rc 0/0 00000100:00100000:1.0:1789760795.279648:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16611 00000100:00100000:1.0:1789760795.279661:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738503722585 time 738518566865 state 0x2 ref 0 00000100:00100000:2.0:1789760795.282941:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760795.283044:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554892928 00000100:00100000:1.0:1789760795.283057:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554892928 00000100:00100000:1.0:1789760795.283130:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738503722585 time 738518566865 state 0x0 ref 1 00000100:00100000:1.0:1789760795.283150:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738518827281 time 738518566865 state 0x1 ref 1 00000100:00100000:1.0:1789760795.283187:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738518827281 time 738528805292 state 0x0 ref 1 00000100:00100000:1.0:1789760795.283201:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09310a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554892928:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760795.283273:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 67997696, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x206e0 00010000:00010000:1.0:1789760795.283286:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7025 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.283308:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760795.283314:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.283322:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7025 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67997696->68001791] (req 67997696->68001791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.283347:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7025 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67997696->68001791] (req 67997696->68001791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.291926:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983910 is committed 00002000:00010000:1.0:1789760795.292748:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 67997696 -> 68001792 00002000:00010000:1.0:1789760795.292756:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132832 -> 132840 00010000:00010000:1.0:1789760795.292775:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7025 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67997696->68001791] (req 67997696->68001791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.292791:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7025 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67997696->68001791] (req 67997696->68001791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.292801:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7025 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67997696->68001791] (req 67997696->68001791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.292809:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.292814:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7025 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67997696->68001791] (req 67997696->68001791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.292824:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7025 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67997696->68001791] (req 67997696->68001791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.292850:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7025 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67997696->68001791] (req 67997696->68001791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.292857:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7025 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [67997696->68001791] (req 67997696->68001791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.293823:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09310a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554892928:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10609us (10881us total) trans 4294983910 rc 0/0 00000100:00100000:1.0:1789760795.293840:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16612 00000100:00100000:1.0:1789760795.293853:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738518827281 time 738528805292 state 0x2 ref 0 00000100:00100000:2.0:1789760795.303174:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760795.306175:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554893056 00000100:00100000:1.0:1789760795.306195:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554893056 00000100:00100000:1.0:1789760795.306350:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738518827281 time 738528805292 state 0x0 ref 1 00000100:00100000:1.0:1789760795.306371:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738529065708 time 738528805292 state 0x1 ref 1 00000100:00100000:1.0:1789760795.306428:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738529065708 time 738552044946 state 0x0 ref 1 00000100:00100000:1.0:1789760795.306445:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb09312a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554893056:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760795.306511:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68001792, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x206e8 00010000:00010000:1.0:1789760795.306525:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c702c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.306545:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760795.306552:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.306560:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c702c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68001792->68005887] (req 68001792->68005887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.306579:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c702c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68001792->68005887] (req 68001792->68005887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.314945:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983911 is committed 00002000:00010000:1.0:1789760795.315491:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68001792 -> 68005888 00002000:00010000:1.0:1789760795.315497:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132840 -> 132848 00010000:00010000:1.0:1789760795.315512:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c702c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68001792->68005887] (req 68001792->68005887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.315527:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c702c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68001792->68005887] (req 68001792->68005887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.315536:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c702c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68001792->68005887] (req 68001792->68005887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.315543:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.315548:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c702c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68001792->68005887] (req 68001792->68005887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.315558:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c702c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68001792->68005887] (req 68001792->68005887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.315598:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c702c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68001792->68005887] (req 68001792->68005887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.315613:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c702c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68001792->68005887] (req 68001792->68005887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.315739:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb09312a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554893056:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9290us (12572us total) trans 4294983911 rc 0/0 00000100:00100000:1.0:1789760795.315752:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16613 00000100:00100000:1.0:1789760795.315764:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738529065708 time 738552044946 state 0x2 ref 0 00000100:00100000:2.0:1789760795.320213:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760795.320317:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554893184 00000100:00100000:1.0:1789760795.320332:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554893184 00000100:00100000:1.0:1789760795.321541:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738529065708 time 738552044946 state 0x0 ref 1 00000100:00100000:1.0:1789760795.321564:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738552305362 time 738552044946 state 0x1 ref 1 00000100:00100000:1.0:1789760795.321754:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738552305362 time 738567370000 state 0x0 ref 1 00000100:00100000:1.0:1789760795.321770:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03660380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554893184:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760795.321834:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68005888, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x206f0 00010000:00010000:1.0:1789760795.321846:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7033 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.321866:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760795.321871:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.321878:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7033 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68005888->68009983] (req 68005888->68009983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.321900:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7033 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68005888->68009983] (req 68005888->68009983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.328718:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983912 is committed 00002000:00010000:1.0:1789760795.328848:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68005888 -> 68009984 00002000:00010000:1.0:1789760795.328854:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132848 -> 132856 00010000:00010000:1.0:1789760795.328871:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7033 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68005888->68009983] (req 68005888->68009983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.328887:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7033 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68005888->68009983] (req 68005888->68009983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.328896:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7033 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68005888->68009983] (req 68005888->68009983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.328904:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.328910:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7033 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68005888->68009983] (req 68005888->68009983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.328920:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7033 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68005888->68009983] (req 68005888->68009983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.328942:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7033 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68005888->68009983] (req 68005888->68009983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.328950:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7033 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68005888->68009983] (req 68005888->68009983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.329068:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03660380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554893184:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7295us (8864us total) trans 4294983912 rc 0/0 00000100:00100000:1.0:1789760795.329080:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16614 00000100:00100000:1.0:1789760795.329091:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738552305362 time 738567370000 state 0x2 ref 0 00000100:00100000:2.0:1789760795.333748:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760795.333799:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554893312 00000100:00100000:1.0:1789760795.333812:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554893312 00000100:00100000:1.0:1789760795.333889:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738552305362 time 738567370000 state 0x0 ref 1 00000100:00100000:1.0:1789760795.333908:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738567630416 time 738567370000 state 0x1 ref 1 00000100:00100000:2.0:1789760795.334052:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738567630416 time 738579667051 state 0x0 ref 1 00000100:00100000:2.0:1789760795.334070:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03663800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554893312:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760795.334144:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68009984, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x206f8 00010000:00010000:2.0:1789760795.334155:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c703a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.334175:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760795.334180:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.334188:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c703a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68009984->68014079] (req 68009984->68014079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.334205:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c703a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68009984->68014079] (req 68009984->68014079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.341858:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983913 is committed 00002000:00010000:2.0:1789760795.342118:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68009984 -> 68014080 00002000:00010000:2.0:1789760795.342123:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132856 -> 132864 00010000:00010000:2.0:1789760795.342149:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c703a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68009984->68014079] (req 68009984->68014079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.342161:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c703a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68009984->68014079] (req 68009984->68014079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.342170:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c703a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68009984->68014079] (req 68009984->68014079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.342177:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760795.342182:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c703a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68009984->68014079] (req 68009984->68014079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.342191:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c703a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68009984->68014079] (req 68009984->68014079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.342209:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c703a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68009984->68014079] (req 68009984->68014079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.342217:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c703a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68009984->68014079] (req 68009984->68014079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.342312:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03663800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554893312:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8242us (8571us total) trans 4294983913 rc 0/0 00000100:00100000:2.0:1789760795.342323:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16615 00000100:00100000:2.0:1789760795.342332:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738567630416 time 738579667051 state 0x2 ref 0 00000100:00100000:2.0:1789760795.348467:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760795.348707:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554893440 00000100:00100000:1.0:1789760795.348716:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554893440 00000100:00100000:1.0:1789760795.348771:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738567630416 time 738579667051 state 0x0 ref 1 00000100:00100000:1.0:1789760795.348786:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738579927467 time 738579667051 state 0x1 ref 1 00000100:00100000:1.0:1789760795.348817:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738579927467 time 738594435142 state 0x0 ref 1 00000100:00100000:1.0:1789760795.348830:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03662680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554893440:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760795.348877:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68014080, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20700 00010000:00010000:1.0:1789760795.348887:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c7041 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.348903:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760795.348908:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.348913:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c7041 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68014080->68018175] (req 68014080->68018175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.348927:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c7041 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68014080->68018175] (req 68014080->68018175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.354547:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983914 is committed 00002000:00010000:1.0:1789760795.354783:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68014080 -> 68018176 00002000:00010000:1.0:1789760795.354788:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132864 -> 132872 00010000:00010000:1.0:1789760795.354801:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c7041 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68014080->68018175] (req 68014080->68018175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.354813:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c7041 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68014080->68018175] (req 68014080->68018175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.354821:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c7041 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68014080->68018175] (req 68014080->68018175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.354828:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ce00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.354832:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c7041 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68014080->68018175] (req 68014080->68018175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.354841:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c7041 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68014080->68018175] (req 68014080->68018175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.354860:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c7041 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68014080->68018175] (req 68014080->68018175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.354873:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3495890/0x1f99687d854c7041 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68014080->68018175] (req 68014080->68018175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.355051:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03662680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554893440:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6218us (6588us total) trans 4294983914 rc 0/0 00000100:00100000:1.0:1789760795.355061:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16616 00000100:00100000:1.0:1789760795.355070:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738579927467 time 738594435142 state 0x2 ref 0 00000100:00100000:1.0:1789760795.359831:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760795.359920:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554893568 00000100:00100000:2.0:1789760795.359929:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554893568 00000100:00100000:2.0:1789760795.360036:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738579927467 time 738594435142 state 0x0 ref 1 00000100:00100000:2.0:1789760795.360050:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738594695558 time 738594435142 state 0x1 ref 1 00000100:00100000:2.0:1789760795.360079:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738594695558 time 738605697707 state 0x0 ref 1 00000100:00100000:2.0:1789760795.360090:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06449c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554893568:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760795.360131:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68018176, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20708 00010000:00010000:2.0:1789760795.360140:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7048 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.360155:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760795.360160:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.360166:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7048 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68018176->68022271] (req 68018176->68022271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.360180:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7048 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68018176->68022271] (req 68018176->68022271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.365216:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983915 is committed 00002000:00010000:2.0:1789760795.365754:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68018176 -> 68022272 00002000:00010000:2.0:1789760795.365760:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132872 -> 132880 00010000:00010000:2.0:1789760795.365773:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7048 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68018176->68022271] (req 68018176->68022271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.365787:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7048 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68018176->68022271] (req 68018176->68022271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.365796:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7048 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68018176->68022271] (req 68018176->68022271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.365803:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760795.365809:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7048 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68018176->68022271] (req 68018176->68022271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.365818:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7048 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68018176->68022271] (req 68018176->68022271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.365839:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7048 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68018176->68022271] (req 68018176->68022271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.365847:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7048 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68018176->68022271] (req 68018176->68022271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.366048:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06449c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554893568:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5952us (6220us total) trans 4294983915 rc 0/0 00000100:00100000:2.0:1789760795.366059:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16617 00000100:00100000:2.0:1789760795.366069:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738594695558 time 738605697707 state 0x2 ref 0 00000100:00100000:2.0:1789760795.370855:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760795.371010:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554893696 00000100:00100000:1.0:1789760795.371020:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554893696 00000100:00100000:1.0:1789760795.371068:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738594695558 time 738605697707 state 0x0 ref 1 00000100:00100000:1.0:1789760795.371082:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738605958123 time 738605697707 state 0x1 ref 1 00000100:00100000:1.0:1789760795.371136:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738605958123 time 738616754571 state 0x0 ref 1 00000100:00100000:1.0:1789760795.371148:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03662300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554893696:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760795.371192:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68022272, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20710 00010000:00010000:1.0:1789760795.371201:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c704f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.371216:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760795.371220:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.371226:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c704f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68022272->68026367] (req 68022272->68026367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.371241:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c704f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68022272->68026367] (req 68022272->68026367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.383181:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983916 is committed 00002000:00010000:1.0:1789760795.383310:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68022272 -> 68026368 00002000:00010000:1.0:1789760795.383315:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132880 -> 132888 00010000:00010000:1.0:1789760795.383329:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c704f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68022272->68026367] (req 68022272->68026367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.383352:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c704f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68022272->68026367] (req 68022272->68026367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.383360:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c704f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68022272->68026367] (req 68022272->68026367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.383367:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.383373:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c704f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68022272->68026367] (req 68022272->68026367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.383417:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c704f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68022272->68026367] (req 68022272->68026367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.383437:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c704f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68022272->68026367] (req 68022272->68026367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.383443:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c704f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68022272->68026367] (req 68022272->68026367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.383543:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03662300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554893696:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12391us (12694us total) trans 4294983916 rc 0/0 00000100:00100000:1.0:1789760795.383553:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16618 00000100:00100000:1.0:1789760795.383562:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738605958123 time 738616754571 state 0x2 ref 0 00000100:00100000:1.0:1789760795.387996:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760795.388145:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554893824 00000100:00100000:2.0:1789760795.388154:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554893824 00000100:00100000:2.0:1789760795.388207:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738605958123 time 738616754571 state 0x0 ref 1 00000100:00100000:2.0:1789760795.388222:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738617014987 time 738616754571 state 0x1 ref 1 00000100:00100000:2.0:1789760795.388277:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738617014987 time 738633895333 state 0x0 ref 1 00000100:00100000:2.0:1789760795.388291:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06449880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554893824:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760795.388348:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68026368, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20718 00010000:00010000:2.0:1789760795.388358:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7056 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.388374:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760795.388413:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.388420:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7056 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68026368->68030463] (req 68026368->68030463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.388435:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7056 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68026368->68030463] (req 68026368->68030463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.395674:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983917 is committed 00002000:00010000:2.0:1789760795.396091:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68026368 -> 68030464 00002000:00010000:2.0:1789760795.396166:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132888 -> 132896 00010000:00010000:2.0:1789760795.396179:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7056 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68026368->68030463] (req 68026368->68030463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.396192:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7056 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68026368->68030463] (req 68026368->68030463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.396200:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7056 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68026368->68030463] (req 68026368->68030463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.396206:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760795.396209:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7056 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68026368->68030463] (req 68026368->68030463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.396216:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7056 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68026368->68030463] (req 68026368->68030463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.396234:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7056 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68026368->68030463] (req 68026368->68030463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.396240:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7056 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68026368->68030463] (req 68026368->68030463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.396915:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06449880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554893824:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8618us (8920us total) trans 4294983917 rc 0/0 00000100:00100000:2.0:1789760795.396926:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16619 00000100:00100000:2.0:1789760795.396935:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738617014987 time 738633895333 state 0x2 ref 0 00000100:00100000:2.0:1789760795.402949:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760795.403147:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554893952 00000100:00100000:1.0:1789760795.403157:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554893952 00000100:00100000:1.0:1789760795.403212:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738617014987 time 738633895333 state 0x0 ref 1 00000100:00100000:1.0:1789760795.403227:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738634155749 time 738633895333 state 0x1 ref 1 00000100:00100000:1.0:1789760795.403297:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738634155749 time 738648910716 state 0x0 ref 1 00000100:00100000:1.0:1789760795.403310:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03661c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554893952:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760795.403375:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68030464, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20720 00010000:00010000:1.0:1789760795.403409:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c705d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.403424:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760795.403429:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.403435:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c705d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68030464->68034559] (req 68030464->68034559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.403448:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c705d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68030464->68034559] (req 68030464->68034559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.410431:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983918 is committed 00002000:00010000:1.0:1789760795.410662:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68030464 -> 68034560 00002000:00010000:1.0:1789760795.410667:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132896 -> 132904 00010000:00010000:1.0:1789760795.410680:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c705d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68030464->68034559] (req 68030464->68034559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.410693:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c705d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68030464->68034559] (req 68030464->68034559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.410703:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c705d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68030464->68034559] (req 68030464->68034559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.410712:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.410729:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c705d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68030464->68034559] (req 68030464->68034559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.410739:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c705d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68030464->68034559] (req 68030464->68034559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.410759:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c705d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68030464->68034559] (req 68030464->68034559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.410766:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c705d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68030464->68034559] (req 68030464->68034559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.410862:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03661c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554893952:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7549us (7919us total) trans 4294983918 rc 0/0 00000100:00100000:1.0:1789760795.410873:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16620 00000100:00100000:1.0:1789760795.410882:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738634155749 time 738648910716 state 0x2 ref 0 00000100:00100000:0.0:1789760795.418231:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760795.418314:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554894080 00000100:00100000:1.0:1789760795.418323:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554894080 00000100:00100000:1.0:1789760795.418406:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738634155749 time 738648910716 state 0x0 ref 1 00000100:00100000:1.0:1789760795.418422:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738649171132 time 738648910716 state 0x1 ref 1 00000100:00100000:1.0:1789760795.418455:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738649171132 time 738664073978 state 0x0 ref 1 00000100:00100000:1.0:1789760795.418467:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e54700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554894080:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760795.418518:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68034560, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20728 00010000:00010000:1.0:1789760795.418526:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7064 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.418542:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760795.418546:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.418553:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7064 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68034560->68038655] (req 68034560->68038655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.418567:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7064 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68034560->68038655] (req 68034560->68038655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.425006:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983919 is committed 00002000:00010000:1.0:1789760795.425212:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68034560 -> 68038656 00002000:00010000:1.0:1789760795.425217:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132904 -> 132912 00010000:00010000:1.0:1789760795.425231:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7064 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68034560->68038655] (req 68034560->68038655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.425244:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7064 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68034560->68038655] (req 68034560->68038655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.425252:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7064 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68034560->68038655] (req 68034560->68038655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.425260:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.425264:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7064 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68034560->68038655] (req 68034560->68038655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.425274:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7064 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68034560->68038655] (req 68034560->68038655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.425293:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7064 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68034560->68038655] (req 68034560->68038655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.425301:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7064 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68034560->68038655] (req 68034560->68038655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.425569:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554894080:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7099us (7343us total) trans 4294983919 rc 0/0 00000100:00100000:1.0:1789760795.425580:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16621 00000100:00100000:1.0:1789760795.425599:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738649171132 time 738664073978 state 0x2 ref 0 00000100:00100000:2.0:1789760795.429814:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760795.429894:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554894208 00000100:00100000:1.0:1789760795.429904:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554894208 00000100:00100000:1.0:1789760795.429975:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738649171132 time 738664073978 state 0x0 ref 1 00000100:00100000:1.0:1789760795.429990:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738664334394 time 738664073978 state 0x1 ref 1 00000100:00100000:1.0:1789760795.430019:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738664334394 time 738675637906 state 0x0 ref 1 00000100:00100000:1.0:1789760795.430033:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03660e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554894208:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760795.430076:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68038656, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20730 00010000:00010000:1.0:1789760795.430085:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c706b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.430110:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760795.430115:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.430120:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c706b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68038656->68042751] (req 68038656->68042751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.430134:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c706b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68038656->68042751] (req 68038656->68042751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.435809:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983920 is committed 00002000:00010000:1.0:1789760795.435995:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68038656 -> 68042752 00002000:00010000:1.0:1789760795.436000:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132912 -> 132920 00010000:00010000:1.0:1789760795.436014:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c706b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68038656->68042751] (req 68038656->68042751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.436033:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c706b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68038656->68042751] (req 68038656->68042751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.436042:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c706b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68038656->68042751] (req 68038656->68042751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.436049:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.436054:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c706b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68038656->68042751] (req 68038656->68042751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.436063:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c706b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68038656->68042751] (req 68038656->68042751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.436082:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c706b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68038656->68042751] (req 68038656->68042751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.436089:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c706b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68038656->68042751] (req 68038656->68042751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.436568:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03660e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554894208:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6533us (6759us total) trans 4294983920 rc 0/0 00000100:00100000:1.0:1789760795.436592:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16622 00000100:00100000:1.0:1789760795.436666:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738664334394 time 738675637906 state 0x2 ref 0 00000100:00100000:2.0:1789760795.443811:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760795.443945:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554894336 00000100:00100000:1.0:1789760795.443956:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554894336 00000100:00100000:1.0:1789760795.444009:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738664334394 time 738675637906 state 0x0 ref 1 00000100:00100000:1.0:1789760795.444023:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738675898322 time 738675637906 state 0x1 ref 1 00000100:00100000:1.0:1789760795.444091:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738675898322 time 738689709507 state 0x0 ref 1 00000100:00100000:1.0:1789760795.444104:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3f9e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554894336:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760795.444150:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68042752, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20738 00010000:00010000:1.0:1789760795.444159:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7072 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.444185:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760795.444190:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.444197:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7072 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68042752->68046847] (req 68042752->68046847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.444214:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7072 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68042752->68046847] (req 68042752->68046847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.451133:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983921 is committed 00002000:00010000:1.0:1789760795.451302:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68042752 -> 68046848 00002000:00010000:1.0:1789760795.451308:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132920 -> 132928 00010000:00010000:1.0:1789760795.451321:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7072 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68042752->68046847] (req 68042752->68046847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.451334:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7072 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68042752->68046847] (req 68042752->68046847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.451348:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7072 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68042752->68046847] (req 68042752->68046847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.451356:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.451360:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7072 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68042752->68046847] (req 68042752->68046847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.451369:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7072 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68042752->68046847] (req 68042752->68046847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.451460:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7072 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68042752->68046847] (req 68042752->68046847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.451470:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7072 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68042752->68046847] (req 68042752->68046847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.451844:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3f9e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554894336:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7607us (7908us total) trans 4294983921 rc 0/0 00000100:00100000:1.0:1789760795.451856:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16623 00000100:00100000:1.0:1789760795.451867:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738675898322 time 738689709507 state 0x2 ref 0 00000100:00100000:2.0:1789760795.457067:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760795.457208:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554894464 00000100:00100000:1.0:1789760795.457217:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554894464 00000100:00100000:1.0:1789760795.457270:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738675898322 time 738689709507 state 0x0 ref 1 00000100:00100000:1.0:1789760795.457285:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738689969923 time 738689709507 state 0x1 ref 1 00000100:00100000:1.0:1789760795.457448:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738689969923 time 738703065112 state 0x0 ref 1 00000100:00100000:1.0:1789760795.457461:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3f9e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554894464:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760795.457509:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68046848, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20740 00010000:00010000:1.0:1789760795.457517:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7079 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.457533:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760795.457537:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.457544:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7079 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68046848->68050943] (req 68046848->68050943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.457559:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7079 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68046848->68050943] (req 68046848->68050943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.462838:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983922 is committed 00002000:00010000:1.0:1789760795.462997:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68046848 -> 68050944 00002000:00010000:1.0:1789760795.463002:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132928 -> 132936 00010000:00010000:1.0:1789760795.463014:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7079 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68046848->68050943] (req 68046848->68050943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.463027:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7079 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68046848->68050943] (req 68046848->68050943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.463036:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7079 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68046848->68050943] (req 68046848->68050943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.463049:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.463054:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7079 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68046848->68050943] (req 68046848->68050943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.463063:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7079 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68046848->68050943] (req 68046848->68050943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.463084:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7079 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68046848->68050943] (req 68046848->68050943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.463091:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7079 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68046848->68050943] (req 68046848->68050943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.463219:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3f9e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554894464:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5756us (6159us total) trans 4294983922 rc 0/0 00000100:00100000:1.0:1789760795.463229:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16624 00000100:00100000:1.0:1789760795.463239:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738689969923 time 738703065112 state 0x2 ref 0 00000100:00100000:2.0:1789760795.468080:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760795.468275:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554894592 00000100:00100000:1.0:1789760795.468285:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554894592 00000100:00100000:1.0:1789760795.468340:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738689969923 time 738703065112 state 0x0 ref 1 00000100:00100000:1.0:1789760795.468355:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738703325528 time 738703065112 state 0x1 ref 1 00000100:00100000:1.0:1789760795.468893:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738703325528 time 738714508306 state 0x0 ref 1 00000100:00100000:1.0:1789760795.468912:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3f9ead80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554894592:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760795.468961:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68050944, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20748 00010000:00010000:1.0:1789760795.468970:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7080 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.468987:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760795.468992:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.468998:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7080 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68050944->68055039] (req 68050944->68055039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.469012:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7080 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68050944->68055039] (req 68050944->68055039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.477833:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983923 is committed 00002000:00010000:1.0:1789760795.478074:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68050944 -> 68055040 00002000:00010000:1.0:1789760795.478080:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132936 -> 132944 00010000:00010000:1.0:1789760795.478093:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7080 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68050944->68055039] (req 68050944->68055039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.478105:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7080 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68050944->68055039] (req 68050944->68055039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.478114:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7080 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68050944->68055039] (req 68050944->68055039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.478121:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.478126:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7080 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68050944->68055039] (req 68050944->68055039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.478135:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7080 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68050944->68055039] (req 68050944->68055039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.478155:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7080 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68050944->68055039] (req 68050944->68055039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.478163:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7080 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68050944->68055039] (req 68050944->68055039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.478290:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3f9ead80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554894592:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9377us (10214us total) trans 4294983923 rc 0/0 00000100:00100000:1.0:1789760795.478301:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16625 00000100:00100000:1.0:1789760795.478310:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738703325528 time 738714508306 state 0x2 ref 0 00000100:00100000:2.0:1789760795.484850:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760795.487388:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554894720 00000100:00100000:1.0:1789760795.487404:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554894720 00000100:00100000:1.0:1789760795.487469:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738703325528 time 738714508306 state 0x0 ref 1 00000100:00100000:1.0:1789760795.487484:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738714768722 time 738714508306 state 0x1 ref 1 00000100:00100000:1.0:1789760795.487599:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738714768722 time 738733216614 state 0x0 ref 1 00000100:00100000:1.0:1789760795.487614:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3f9eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554894720:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760795.487666:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68055040, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20750 00010000:00010000:1.0:1789760795.487676:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7087 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.487693:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760795.487698:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.487704:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7087 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68055040->68059135] (req 68055040->68059135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.489512:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7087 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68055040->68059135] (req 68055040->68059135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.494152:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983924 is committed 00002000:00010000:1.0:1789760795.494273:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68055040 -> 68059136 00002000:00010000:1.0:1789760795.494278:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132944 -> 132952 00010000:00010000:1.0:1789760795.494291:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7087 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68055040->68059135] (req 68055040->68059135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.494304:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7087 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68055040->68059135] (req 68055040->68059135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.494313:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7087 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68055040->68059135] (req 68055040->68059135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.494321:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.494326:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7087 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68055040->68059135] (req 68055040->68059135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.494335:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7087 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68055040->68059135] (req 68055040->68059135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.494355:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7087 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68055040->68059135] (req 68055040->68059135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.494362:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c7087 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68055040->68059135] (req 68055040->68059135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.494488:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3f9eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554894720:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6874us (9642us total) trans 4294983924 rc 0/0 00000100:00100000:1.0:1789760795.494499:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16626 00000100:00100000:1.0:1789760795.494508:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738714768722 time 738733216614 state 0x2 ref 0 00000100:00100000:2.0:1789760795.500867:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760795.500952:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554894848 00000100:00100000:1.0:1789760795.500962:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554894848 00000100:00100000:1.0:1789760795.501012:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738714768722 time 738733216614 state 0x0 ref 1 00000100:00100000:1.0:1789760795.501026:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738733477030 time 738733216614 state 0x1 ref 1 00000100:00100000:2.0:1789760795.501615:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738733477030 time 738747221571 state 0x0 ref 1 00000100:00100000:2.0:1789760795.501782:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3f9ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554894848:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760795.501832:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68059136, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20758 00010000:00010000:2.0:1789760795.501843:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c708e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.501858:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760795.501863:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.501870:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c708e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68059136->68063231] (req 68059136->68063231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.501884:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c708e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68059136->68063231] (req 68059136->68063231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.508310:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983925 is committed 00002000:00010000:2.0:1789760795.508496:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68059136 -> 68063232 00002000:00010000:2.0:1789760795.508502:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132952 -> 132960 00010000:00010000:2.0:1789760795.508515:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c708e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68059136->68063231] (req 68059136->68063231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.508527:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c708e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68059136->68063231] (req 68059136->68063231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.508537:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c708e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68059136->68063231] (req 68059136->68063231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.508544:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760795.508548:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c708e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68059136->68063231] (req 68059136->68063231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.508557:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c708e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68059136->68063231] (req 68059136->68063231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.508576:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c708e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68059136->68063231] (req 68059136->68063231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.508593:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c708e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68059136->68063231] (req 68059136->68063231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.508689:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3f9ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554894848:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6909us (7827us total) trans 4294983925 rc 0/0 00000100:00100000:2.0:1789760795.508700:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16627 00000100:00100000:2.0:1789760795.508725:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738733477030 time 738747221571 state 0x2 ref 0 00000100:00100000:2.0:1789760795.517479:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760795.517600:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554894976 00000100:00100000:1.0:1789760795.517624:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554894976 00000100:00100000:1.0:1789760795.517679:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738733477030 time 738747221571 state 0x0 ref 1 00000100:00100000:1.0:1789760795.517694:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738747481987 time 738747221571 state 0x1 ref 1 00000100:00100000:1.0:1789760795.517725:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738747481987 time 738763344017 state 0x0 ref 1 00000100:00100000:1.0:1789760795.517738:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3f9e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554894976:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760795.517786:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68063232, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20760 00010000:00010000:1.0:1789760795.517795:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7095 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.517810:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760795.517814:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.517821:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7095 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68063232->68067327] (req 68063232->68067327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.517835:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7095 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68063232->68067327] (req 68063232->68067327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.522919:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983926 is committed 00002000:00010000:0.0:1789760795.523193:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68063232 -> 68067328 00002000:00010000:0.0:1789760795.523198:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132960 -> 132968 00010000:00010000:0.0:1789760795.523211:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7095 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68063232->68067327] (req 68063232->68067327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.523223:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7095 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68063232->68067327] (req 68063232->68067327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.523231:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7095 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68063232->68067327] (req 68063232->68067327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.523243:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760795.523247:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7095 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68063232->68067327] (req 68063232->68067327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.523256:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7095 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68063232->68067327] (req 68063232->68067327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.523275:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7095 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68063232->68067327] (req 68063232->68067327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.523282:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7095 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68063232->68067327] (req 68063232->68067327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.523455:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3f9e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554894976:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5714us (5981us total) trans 4294983926 rc 0/0 00000100:00100000:0.0:1789760795.523466:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16628 00000100:00100000:0.0:1789760795.523475:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738747481987 time 738763344017 state 0x2 ref 0 00000100:00100000:1.0:1789760795.529129:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760795.529851:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554895104 00000100:00100000:2.0:1789760795.529861:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554895104 00000100:00100000:2.0:1789760795.529918:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738747481987 time 738763344017 state 0x0 ref 1 00000100:00100000:2.0:1789760795.529934:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738763604433 time 738763344017 state 0x1 ref 1 00000100:00100000:2.0:1789760795.529967:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738763604433 time 738775585602 state 0x0 ref 1 00000100:00100000:2.0:1789760795.529980:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06448e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554895104:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760795.530023:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68067328, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20768 00010000:00010000:2.0:1789760795.530032:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c709c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.530047:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760795.530051:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.530058:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c709c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68067328->68071423] (req 68067328->68071423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.530071:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c709c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68067328->68071423] (req 68067328->68071423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.538466:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983927 is committed 00002000:00010000:1.0:1789760795.539064:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68067328 -> 68071424 00002000:00010000:1.0:1789760795.539071:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132968 -> 132976 00010000:00010000:1.0:1789760795.539084:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c709c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68067328->68071423] (req 68067328->68071423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.539096:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c709c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68067328->68071423] (req 68067328->68071423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.539107:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c709c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68067328->68071423] (req 68067328->68071423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.539114:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.539118:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c709c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68067328->68071423] (req 68067328->68071423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.539128:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c709c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68067328->68071423] (req 68067328->68071423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.539148:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c709c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68067328->68071423] (req 68067328->68071423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.539156:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c709c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68067328->68071423] (req 68067328->68071423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.539305:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06448e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554895104:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9322us (10181us total) trans 4294983927 rc 0/0 00000100:00100000:1.0:1789760795.539315:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16629 00000100:00100000:1.0:1789760795.539324:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738763604433 time 738775585602 state 0x2 ref 0 00000100:00100000:2.0:1789760795.543861:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760795.549513:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554895232 00000100:00100000:1.0:1789760795.549529:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554895232 00000100:00100000:0.0:1789760795.549697:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738763604433 time 738775585602 state 0x0 ref 1 00000100:00100000:0.0:1789760795.549903:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738775846018 time 738775585602 state 0x1 ref 1 00000100:00100000:0.0:1789760795.549942:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738775846018 time 738795560121 state 0x0 ref 1 00000100:00100000:0.0:1789760795.549955:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3f9e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554895232:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760795.550004:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68071424, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20770 00010000:00010000:0.0:1789760795.550014:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70a3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.550031:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760795.550035:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.550041:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68071424->68075519] (req 68071424->68075519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.550056:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68071424->68075519] (req 68071424->68075519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.558373:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983928 is committed 00002000:00010000:0.0:1789760795.558525:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68071424 -> 68075520 00002000:00010000:0.0:1789760795.558530:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132976 -> 132984 00010000:00010000:0.0:1789760795.558544:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70a3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68071424->68075519] (req 68071424->68075519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.558558:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70a3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68071424->68075519] (req 68071424->68075519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.558567:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68071424->68075519] (req 68071424->68075519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.558573:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760795.558578:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70a3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68071424->68075519] (req 68071424->68075519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.558609:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70a3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68071424->68075519] (req 68071424->68075519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.558630:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70a3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68071424->68075519] (req 68071424->68075519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.558636:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70a3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68071424->68075519] (req 68071424->68075519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.558726:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3f9e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554895232:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8769us (14875us total) trans 4294983928 rc 0/0 00000100:00100000:0.0:1789760795.558737:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16630 00000100:00100000:0.0:1789760795.558747:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738775846018 time 738795560121 state 0x2 ref 0 00000100:00100000:2.0:1789760795.565523:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760795.565647:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554895360 00000100:00100000:1.0:1789760795.565659:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554895360 00000100:00100000:1.0:1789760795.565713:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738775846018 time 738795560121 state 0x0 ref 1 00000100:00100000:1.0:1789760795.565728:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738795820537 time 738795560121 state 0x1 ref 1 00000100:00100000:1.0:1789760795.565767:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738795820537 time 738811385732 state 0x0 ref 1 00000100:00100000:1.0:1789760795.565780:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554895360:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760795.565827:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68075520, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20778 00010000:00010000:1.0:1789760795.565846:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c70aa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.565862:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:1.0:1789760795.565866:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.565872:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c70aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68075520->68079615] (req 68075520->68079615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.565886:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c70aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68075520->68079615] (req 68075520->68079615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.571940:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983929 is committed 00002000:00010000:0.0:1789760795.572944:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68075520 -> 68079616 00002000:00010000:0.0:1789760795.572952:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132984 -> 132992 00010000:00010000:0.0:1789760795.572965:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c70aa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68075520->68079615] (req 68075520->68079615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.572977:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c70aa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68075520->68079615] (req 68075520->68079615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.572986:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c70aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68075520->68079615] (req 68075520->68079615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.572998:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760795.573003:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c70aa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68075520->68079615] (req 68075520->68079615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.573012:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c70aa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68075520->68079615] (req 68075520->68079615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.573032:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c70aa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68075520->68079615] (req 68075520->68079615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.573040:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c70aa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68075520->68079615] (req 68075520->68079615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.573138:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554895360:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7356us (7620us total) trans 4294983929 rc 0/0 00000100:00100000:0.0:1789760795.573149:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16631 00000100:00100000:0.0:1789760795.573159:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738795820537 time 738811385732 state 0x2 ref 0 00000100:00100000:1.0:1789760795.577987:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760795.578084:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554895488 00000100:00100000:2.0:1789760795.578094:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554895488 00000100:00100000:2.0:1789760795.578147:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738795820537 time 738811385732 state 0x0 ref 1 00000100:00100000:2.0:1789760795.578166:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738811646148 time 738811385732 state 0x1 ref 1 00000100:00100000:2.0:1789760795.578203:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738811646148 time 738823821184 state 0x0 ref 1 00000100:00100000:2.0:1789760795.578214:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06449500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554895488:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760795.578263:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68079616, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20780 00010000:00010000:2.0:1789760795.578273:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c70b1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.578289:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760795.578293:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.578300:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c70b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68079616->68083711] (req 68079616->68083711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.578313:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c70b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68079616->68083711] (req 68079616->68083711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.586942:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983930 is committed 00002000:00010000:2.0:1789760795.587424:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68079616 -> 68083712 00002000:00010000:2.0:1789760795.587429:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 132992 -> 133000 00010000:00010000:2.0:1789760795.587442:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c70b1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68079616->68083711] (req 68079616->68083711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.587461:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c70b1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68079616->68083711] (req 68079616->68083711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.587476:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c70b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68079616->68083711] (req 68079616->68083711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.587483:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760795.587488:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c70b1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68079616->68083711] (req 68079616->68083711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.587498:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c70b1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68079616->68083711] (req 68079616->68083711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.587519:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c70b1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68079616->68083711] (req 68079616->68083711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.587527:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c70b1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68079616->68083711] (req 68079616->68083711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.587939:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06449500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554895488:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9719us (9954us total) trans 4294983930 rc 0/0 00000100:00100000:2.0:1789760795.587951:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16632 00000100:00100000:2.0:1789760795.587970:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738811646148 time 738823821184 state 0x2 ref 0 00000100:00100000:2.0:1789760795.592236:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760795.592314:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554895616 00000100:00100000:3.0:1789760795.592324:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554895616 00000100:00100000:3.0:1789760795.592423:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738811646148 time 738823821184 state 0x0 ref 1 00000100:00100000:3.0:1789760795.592438:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738824081600 time 738823821184 state 0x1 ref 1 00000100:00100000:3.0:1789760795.592470:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738824081600 time 738838088475 state 0x0 ref 1 00000100:00100000:3.0:1789760795.592483:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554895616:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760795.592536:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68083712, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20788 00010000:00010000:3.0:1789760795.592544:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c70b8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.592559:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760795.592564:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.592570:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c70b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68083712->68087807] (req 68083712->68087807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.592591:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c70b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68083712->68087807] (req 68083712->68087807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.601811:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983931 is committed 00002000:00010000:1.0:1789760795.602245:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68083712 -> 68087808 00002000:00010000:1.0:1789760795.602251:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133000 -> 133008 00010000:00010000:1.0:1789760795.602265:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c70b8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68083712->68087807] (req 68083712->68087807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.602277:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c70b8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68083712->68087807] (req 68083712->68087807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.602285:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c70b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68083712->68087807] (req 68083712->68087807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.602292:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.602297:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c70b8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68083712->68087807] (req 68083712->68087807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.602307:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c70b8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68083712->68087807] (req 68083712->68087807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.602326:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c70b8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68083712->68087807] (req 68083712->68087807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.602333:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c70b8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68083712->68087807] (req 68083712->68087807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.602548:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554895616:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10061us (10313us total) trans 4294983931 rc 0/0 00000100:00100000:1.0:1789760795.602559:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16633 00000100:00100000:1.0:1789760795.602568:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738824081600 time 738838088475 state 0x2 ref 0 00000100:00100000:1.0:1789760795.611483:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760795.611559:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554895744 00000100:00100000:0.0:1789760795.611592:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554895744 00000100:00100000:0.0:1789760795.611649:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738824081600 time 738838088475 state 0x0 ref 1 00000100:00100000:0.0:1789760795.611664:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738838348891 time 738838088475 state 0x1 ref 1 00000100:00100000:0.0:1789760795.611695:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738838348891 time 738857313666 state 0x0 ref 1 00000100:00100000:0.0:1789760795.611711:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb0644b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554895744:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760795.611758:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68087808, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20790 00010000:00010000:0.0:1789760795.611767:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70bf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.611782:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760795.611786:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.611792:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68087808->68091903] (req 68087808->68091903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.611806:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68087808->68091903] (req 68087808->68091903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760795.618496:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983932 is committed 00002000:00010000:0.0:1789760795.618859:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68087808 -> 68091904 00002000:00010000:0.0:1789760795.618865:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133008 -> 133016 00010000:00010000:0.0:1789760795.618878:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70bf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68087808->68091903] (req 68087808->68091903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.618890:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70bf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68087808->68091903] (req 68087808->68091903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.618900:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68087808->68091903] (req 68087808->68091903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.618908:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760795.618914:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70bf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68087808->68091903] (req 68087808->68091903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.618924:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70bf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68087808->68091903] (req 68087808->68091903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.618944:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70bf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68087808->68091903] (req 68087808->68091903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.618951:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70bf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68087808->68091903] (req 68087808->68091903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.619161:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb0644b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554895744:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7447us (7682us total) trans 4294983932 rc 0/0 00000100:00100000:0.0:1789760795.619172:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16634 00000100:00100000:0.0:1789760795.619181:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738838348891 time 738857313666 state 0x2 ref 0 00000100:00100000:2.0:1789760795.624104:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760795.624208:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554895872 00000100:00100000:0.0:1789760795.624218:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554895872 00000100:00100000:0.0:1789760795.624274:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738838348891 time 738857313666 state 0x0 ref 1 00000100:00100000:0.0:1789760795.624290:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738857574082 time 738857313666 state 0x1 ref 1 00000100:00100000:0.0:1789760795.624374:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738857574082 time 738869992056 state 0x0 ref 1 00000100:00100000:0.0:1789760795.624414:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554895872:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760795.624464:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68091904, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x20798 00010000:00010000:0.0:1789760795.624474:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c70c6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.624490:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760795.624494:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.624500:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c70c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68091904->68095999] (req 68091904->68095999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.624512:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c70c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68091904->68095999] (req 68091904->68095999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.631067:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983933 is committed 00002000:00010000:0.0:1789760795.631236:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68091904 -> 68096000 00002000:00010000:0.0:1789760795.631241:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133016 -> 133024 00010000:00010000:0.0:1789760795.631303:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c70c6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68091904->68095999] (req 68091904->68095999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.631316:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c70c6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68091904->68095999] (req 68091904->68095999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.631329:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c70c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68091904->68095999] (req 68091904->68095999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.631429:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760795.631435:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c70c6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68091904->68095999] (req 68091904->68095999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.631445:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c70c6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68091904->68095999] (req 68091904->68095999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.631465:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c70c6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68091904->68095999] (req 68091904->68095999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.631472:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c70c6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68091904->68095999] (req 68091904->68095999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.631552:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554895872:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7136us (7453us total) trans 4294983933 rc 0/0 00000100:00100000:0.0:1789760795.631562:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16635 00000100:00100000:0.0:1789760795.632027:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738857574082 time 738869992056 state 0x2 ref 0 00000100:00100000:2.0:1789760795.636859:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760795.637011:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554896000 00000100:00100000:0.0:1789760795.637021:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554896000 00000100:00100000:0.0:1789760795.637077:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738857574082 time 738869992056 state 0x0 ref 1 00000100:00100000:0.0:1789760795.637091:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738870252472 time 738869992056 state 0x1 ref 1 00000100:00100000:0.0:1789760795.637158:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738870252472 time 738882775727 state 0x0 ref 1 00000100:00100000:0.0:1789760795.637171:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554896000:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760795.637221:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68096000, mtime: 0x6aad951a, atime: 0x0, ctime: 0x6aad951a, blocks: 0x207a0 00010000:00010000:0.0:1789760795.637231:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70cd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.637260:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760795.637266:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.637272:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68096000->68100095] (req 68096000->68100095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.637286:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68096000->68100095] (req 68096000->68100095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.645754:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983934 is committed 00002000:00010000:0.0:1789760795.647496:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68096000 -> 68100096 00002000:00010000:0.0:1789760795.647504:0:8443:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760794 -> 1789760795 00002000:00010000:0.0:1789760795.647507:0:8443:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760794 -> 1789760795 00002000:00010000:0.0:1789760795.647509:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133024 -> 133032 00010000:00010000:0.0:1789760795.647523:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70cd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68096000->68100095] (req 68096000->68100095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.647535:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70cd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68096000->68100095] (req 68096000->68100095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.647543:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68096000->68100095] (req 68096000->68100095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.647551:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760795.647555:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70cd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68096000->68100095] (req 68096000->68100095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.647564:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70cd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68096000->68100095] (req 68096000->68100095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.648048:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70cd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68096000->68100095] (req 68096000->68100095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.648057:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70cd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68096000->68100095] (req 68096000->68100095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.648206:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554896000:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11033us (11352us total) trans 4294983934 rc 0/0 00000100:00100000:0.0:1789760795.648218:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16636 00000100:00100000:0.0:1789760795.648227:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738870252472 time 738882775727 state 0x2 ref 0 00000100:00100000:1.0:1789760795.660908:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760795.661942:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554896128 00000100:00100000:3.0:1789760795.661954:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554896128 00000100:00100000:3.0:1789760795.662031:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738870252472 time 738882775727 state 0x0 ref 1 00000100:00100000:3.0:1789760795.662047:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738883036143 time 738882775727 state 0x1 ref 1 00000100:00100000:3.0:1789760795.662082:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738883036143 time 738907700708 state 0x0 ref 1 00000100:00100000:3.0:1789760795.662095:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554896128:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760795.662144:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68100096, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x207a8 00010000:00010000:3.0:1789760795.662154:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c70d4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.662170:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760795.662174:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.662180:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c70d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68100096->68104191] (req 68100096->68104191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.662194:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c70d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68100096->68104191] (req 68100096->68104191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.671305:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983935 is committed 00002000:00010000:3.0:1789760795.671489:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68100096 -> 68104192 00002000:00010000:3.0:1789760795.671494:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133032 -> 133040 00010000:00010000:3.0:1789760795.671509:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c70d4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68100096->68104191] (req 68100096->68104191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.671522:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c70d4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68100096->68104191] (req 68100096->68104191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.671531:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c70d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68100096->68104191] (req 68100096->68104191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.671539:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760795.671544:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c70d4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68100096->68104191] (req 68100096->68104191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.671553:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c70d4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68100096->68104191] (req 68100096->68104191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.671572:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c70d4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68100096->68104191] (req 68100096->68104191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.671589:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c70d4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68100096->68104191] (req 68100096->68104191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.671686:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554896128:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9589us (10783us total) trans 4294983935 rc 0/0 00000100:00100000:3.0:1789760795.671697:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16637 00000100:00100000:3.0:1789760795.671706:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738883036143 time 738907700708 state 0x2 ref 0 00000100:00100000:2.0:1789760795.675968:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760795.676063:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554896256 00000100:00100000:3.0:1789760795.676072:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554896256 00000100:00100000:3.0:1789760795.676128:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738883036143 time 738907700708 state 0x0 ref 1 00000100:00100000:3.0:1789760795.676143:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738907961124 time 738907700708 state 0x1 ref 1 00000100:00100000:3.0:1789760795.676177:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738907961124 time 738921795698 state 0x0 ref 1 00000100:00100000:3.0:1789760795.676189:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554896256:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760795.676239:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68104192, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x207b0 00010000:00010000:3.0:1789760795.676249:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c70db lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.676266:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760795.676270:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.676276:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c70db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68104192->68108287] (req 68104192->68108287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.676290:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c70db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68104192->68108287] (req 68104192->68108287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.686292:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983936 is committed 00002000:00010000:3.0:1789760795.686736:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68104192 -> 68108288 00002000:00010000:3.0:1789760795.686741:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133040 -> 133048 00010000:00010000:3.0:1789760795.686756:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c70db lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68104192->68108287] (req 68104192->68108287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.686776:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c70db lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68104192->68108287] (req 68104192->68108287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.686784:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c70db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68104192->68108287] (req 68104192->68108287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.686792:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760795.686797:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c70db lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68104192->68108287] (req 68104192->68108287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.686807:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c70db lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68104192->68108287] (req 68104192->68108287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.686827:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c70db lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68104192->68108287] (req 68104192->68108287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.686836:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c70db lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68104192->68108287] (req 68104192->68108287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.686935:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554896256:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10743us (10983us total) trans 4294983936 rc 0/0 00000100:00100000:3.0:1789760795.686947:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16638 00000100:00100000:3.0:1789760795.686957:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738907961124 time 738921795698 state 0x2 ref 0 00000100:00100000:2.0:1789760795.692081:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760795.692179:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554896384 00000100:00100000:0.0:1789760795.692189:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554896384 00000100:00100000:0.0:1789760795.692253:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738907961124 time 738921795698 state 0x0 ref 1 00000100:00100000:0.0:1789760795.692271:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738922056114 time 738921795698 state 0x1 ref 1 00000100:00100000:0.0:1789760795.692302:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738922056114 time 738937920263 state 0x0 ref 1 00000100:00100000:0.0:1789760795.692314:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554896384:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760795.692400:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68108288, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x207b8 00010000:00010000:0.0:1789760795.692410:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70e2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.692430:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:0.0:1789760795.692435:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.692442:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68108288->68112383] (req 68108288->68112383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.692462:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68108288->68112383] (req 68108288->68112383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.699637:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983937 is committed 00002000:00010000:0.0:1789760795.699948:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68108288 -> 68112384 00002000:00010000:0.0:1789760795.699954:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133048 -> 133056 00010000:00010000:0.0:1789760795.699966:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70e2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68108288->68112383] (req 68108288->68112383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.699979:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70e2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68108288->68112383] (req 68108288->68112383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.699989:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68108288->68112383] (req 68108288->68112383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.699996:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760795.700000:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70e2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68108288->68112383] (req 68108288->68112383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.700009:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70e2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68108288->68112383] (req 68108288->68112383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.700036:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70e2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68108288->68112383] (req 68108288->68112383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.700043:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c70e2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68108288->68112383] (req 68108288->68112383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.700136:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554896384:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7819us (8059us total) trans 4294983937 rc 0/0 00000100:00100000:0.0:1789760795.700147:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16639 00000100:00100000:0.0:1789760795.700157:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738922056114 time 738937920263 state 0x2 ref 0 00000100:00100000:1.0:1789760795.707464:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760795.707557:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554896512 00000100:00100000:2.0:1789760795.707567:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554896512 00000100:00100000:2.0:1789760795.707645:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738922056114 time 738937920263 state 0x0 ref 1 00000100:00100000:2.0:1789760795.707661:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738938180679 time 738937920263 state 0x1 ref 1 00000100:00100000:2.0:1789760795.707692:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738938180679 time 738953310299 state 0x0 ref 1 00000100:00100000:2.0:1789760795.707705:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554896512:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760795.707760:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68112384, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x207c0 00010000:00010000:2.0:1789760795.707770:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c70e9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.707785:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:2.0:1789760795.707790:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.707796:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c70e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68112384->68116479] (req 68112384->68116479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.707810:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c70e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68112384->68116479] (req 68112384->68116479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.715364:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983938 is committed 00002000:00010000:2.0:1789760795.719788:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68112384 -> 68116480 00002000:00010000:2.0:1789760795.719800:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133056 -> 133064 00010000:00010000:2.0:1789760795.719813:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c70e9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68112384->68116479] (req 68112384->68116479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.719826:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c70e9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68112384->68116479] (req 68112384->68116479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.719835:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c70e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68112384->68116479] (req 68112384->68116479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.719842:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760795.719846:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c70e9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68112384->68116479] (req 68112384->68116479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.719854:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c70e9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68112384->68116479] (req 68112384->68116479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.719873:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c70e9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68112384->68116479] (req 68112384->68116479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.719881:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c70e9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68112384->68116479] (req 68112384->68116479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.719985:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554896512:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12277us (12524us total) trans 4294983938 rc 0/0 00000100:00100000:2.0:1789760795.719995:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16640 00000100:00100000:2.0:1789760795.720005:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738938180679 time 738953310299 state 0x2 ref 0 00000100:00100000:2.0:1789760795.726467:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760795.727687:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554896640 00000100:00100000:3.0:1789760795.727699:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554896640 00000100:00100000:3.0:1789760795.727770:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738938180679 time 738953310299 state 0x0 ref 1 00000100:00100000:3.0:1789760795.727788:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738953570715 time 738953310299 state 0x1 ref 1 00000100:00100000:3.0:1789760795.727826:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738953570715 time 738973443919 state 0x0 ref 1 00000100:00100000:3.0:1789760795.727848:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554896640:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760795.727895:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68116480, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x207c8 00010000:00010000:3.0:1789760795.727906:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c70f0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.727922:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760795.727926:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.727932:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c70f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68116480->68120575] (req 68116480->68120575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.727947:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c70f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68116480->68120575] (req 68116480->68120575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.739894:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983939 is committed 00002000:00010000:1.0:1789760795.740351:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68116480 -> 68120576 00002000:00010000:1.0:1789760795.740356:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133064 -> 133072 00010000:00010000:1.0:1789760795.740368:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c70f0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68116480->68120575] (req 68116480->68120575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.740403:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c70f0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68116480->68120575] (req 68116480->68120575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.740417:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c70f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68116480->68120575] (req 68116480->68120575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.740425:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.740430:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c70f0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68116480->68120575] (req 68116480->68120575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.740439:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c70f0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68116480->68120575] (req 68116480->68120575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.740458:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c70f0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68116480->68120575] (req 68116480->68120575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.740465:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c70f0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68116480->68120575] (req 68116480->68120575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.740556:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554896640:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12706us (14093us total) trans 4294983939 rc 0/0 00000100:00100000:1.0:1789760795.740567:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16641 00000100:00100000:1.0:1789760795.740576:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738953570715 time 738973443919 state 0x2 ref 0 00000100:00100000:0.0:1789760795.747972:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760795.748057:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554896768 00000100:00100000:3.0:1789760795.748066:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554896768 00000100:00100000:3.0:1789760795.748119:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738953570715 time 738973443919 state 0x0 ref 1 00000100:00100000:3.0:1789760795.748135:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738973704335 time 738973443919 state 0x1 ref 1 00000100:00100000:3.0:1789760795.748173:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738973704335 time 738993792060 state 0x0 ref 1 00000100:00100000:3.0:1789760795.748199:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e57800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554896768:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760795.748243:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68120576, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x207d0 00010000:00010000:3.0:1789760795.748252:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c70f7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.748267:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=738s v=(0 0) 00010000:00010000:3.0:1789760795.748272:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.748278:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c70f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68120576->68124671] (req 68120576->68124671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.748291:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c70f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68120576->68124671] (req 68120576->68124671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.753966:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983940 is committed 00002000:00010000:3.0:1789760795.754144:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68120576 -> 68124672 00002000:00010000:3.0:1789760795.754149:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133072 -> 133080 00010000:00010000:3.0:1789760795.754161:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c70f7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68120576->68124671] (req 68120576->68124671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.754173:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c70f7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68120576->68124671] (req 68120576->68124671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.754183:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c70f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68120576->68124671] (req 68120576->68124671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.754251:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760795.754258:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c70f7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68120576->68124671] (req 68120576->68124671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.754268:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c70f7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68120576->68124671] (req 68120576->68124671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.754286:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c70f7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68120576->68124671] (req 68120576->68124671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.754295:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c70f7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68120576->68124671] (req 68120576->68124671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.754607:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e57800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554896768:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6414us (6639us total) trans 4294983940 rc 0/0 00000100:00100000:3.0:1789760795.754619:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16642 00000100:00100000:3.0:1789760795.754628:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738973704335 time 738993792060 state 0x2 ref 0 00000100:00100000:2.0:1789760795.758150:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760795.758244:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554896896 00000100:00100000:0.0:1789760795.758253:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554896896 00000100:00100000:0.0:1789760795.758303:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738973704335 time 738993792060 state 0x0 ref 1 00000100:00100000:0.0:1789760795.758316:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738994052476 time 738993792060 state 0x1 ref 1 00000100:00100000:0.0:1789760795.758351:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738994052476 time 739003969957 state 0x0 ref 1 00000100:00100000:0.0:1789760795.758361:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554896896:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760795.758429:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68124672, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x207d8 00010000:00010000:0.0:1789760795.758439:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c70fe lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.758460:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760795.758464:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.758470:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c70fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68124672->68128767] (req 68124672->68128767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.758487:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c70fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68124672->68128767] (req 68124672->68128767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.764006:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983941 is committed 00002000:00010000:1.0:1789760795.766303:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68124672 -> 68128768 00002000:00010000:1.0:1789760795.766312:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133080 -> 133088 00010000:00010000:1.0:1789760795.766326:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c70fe lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68124672->68128767] (req 68124672->68128767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.766355:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c70fe lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68124672->68128767] (req 68124672->68128767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.766363:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c70fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68124672->68128767] (req 68124672->68128767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.766370:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.766398:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c70fe lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68124672->68128767] (req 68124672->68128767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.766408:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c70fe lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68124672->68128767] (req 68124672->68128767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.766428:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c70fe lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68124672->68128767] (req 68124672->68128767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.766437:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c70fe lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68124672->68128767] (req 68124672->68128767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.766528:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554896896:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8163us (8382us total) trans 4294983941 rc 0/0 00000100:00100000:1.0:1789760795.766540:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16643 00000100:00100000:1.0:1789760795.766549:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738994052476 time 739003969957 state 0x2 ref 0 00000100:00100000:2.0:1789760795.771980:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760795.772068:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554897024 00000100:00100000:0.0:1789760795.772141:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554897024 00000100:00100000:0.0:1789760795.772195:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 738994052476 time 739003969957 state 0x0 ref 1 00000100:00100000:0.0:1789760795.772211:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739004230373 time 739003969957 state 0x1 ref 1 00000100:00100000:0.0:1789760795.772244:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739004230373 time 739017862823 state 0x0 ref 1 00000100:00100000:0.0:1789760795.772257:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554897024:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760795.772300:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68128768, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x207e0 00010000:00010000:0.0:1789760795.772309:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c7105 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.772325:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760795.772330:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.772336:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c7105 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68128768->68132863] (req 68128768->68132863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.772355:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c7105 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68128768->68132863] (req 68128768->68132863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.777354:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983942 is committed 00002000:00010000:1.0:1789760795.777620:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68128768 -> 68132864 00002000:00010000:1.0:1789760795.777626:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133088 -> 133096 00010000:00010000:1.0:1789760795.777639:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c7105 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68128768->68132863] (req 68128768->68132863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.777651:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c7105 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68128768->68132863] (req 68128768->68132863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.777661:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c7105 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68128768->68132863] (req 68128768->68132863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.777669:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.777674:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c7105 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68128768->68132863] (req 68128768->68132863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.777683:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c7105 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68128768->68132863] (req 68128768->68132863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.777702:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c7105 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68128768->68132863] (req 68128768->68132863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.777710:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c7105 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68128768->68132863] (req 68128768->68132863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.777797:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554897024:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5537us (5821us total) trans 4294983942 rc 0/0 00000100:00100000:1.0:1789760795.777808:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16644 00000100:00100000:1.0:1789760795.777818:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739004230373 time 739017862823 state 0x2 ref 0 00000100:00100000:0.0:1789760795.789434:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760795.789544:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554897152 00000100:00100000:1.0:1789760795.789555:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554897152 00000100:00100000:1.0:1789760795.789620:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739004230373 time 739017862823 state 0x0 ref 1 00000100:00100000:1.0:1789760795.789636:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739018123239 time 739017862823 state 0x1 ref 1 00000100:00100000:1.0:1789760795.789671:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739018123239 time 739035288510 state 0x0 ref 1 00000100:00100000:1.0:1789760795.789683:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554897152:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760795.789730:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68132864, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x207e8 00010000:00010000:1.0:1789760795.789740:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c710c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.789756:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760795.789760:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.789766:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c710c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68132864->68136959] (req 68132864->68136959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.789779:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c710c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68132864->68136959] (req 68132864->68136959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.794618:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983943 is committed 00002000:00010000:1.0:1789760795.794729:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68132864 -> 68136960 00002000:00010000:1.0:1789760795.794734:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133096 -> 133104 00010000:00010000:1.0:1789760795.794746:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c710c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68132864->68136959] (req 68132864->68136959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.794764:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c710c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68132864->68136959] (req 68132864->68136959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.794773:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c710c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68132864->68136959] (req 68132864->68136959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.794780:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.794784:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c710c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68132864->68136959] (req 68132864->68136959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.794792:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c710c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68132864->68136959] (req 68132864->68136959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.794811:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c710c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68132864->68136959] (req 68132864->68136959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.794819:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c710c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68132864->68136959] (req 68132864->68136959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.795059:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554897152:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5373us (5630us total) trans 4294983943 rc 0/0 00000100:00100000:1.0:1789760795.795069:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16645 00000100:00100000:1.0:1789760795.795080:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739018123239 time 739035288510 state 0x2 ref 0 00000100:00100000:2.0:1789760795.802736:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760795.802871:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554897280 00000100:00100000:3.0:1789760795.802881:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554897280 00000100:00100000:3.0:1789760795.802936:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739018123239 time 739035288510 state 0x0 ref 1 00000100:00100000:3.0:1789760795.802952:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739035548926 time 739035288510 state 0x1 ref 1 00000100:00100000:3.0:1789760795.802991:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739035548926 time 739048609608 state 0x0 ref 1 00000100:00100000:3.0:1789760795.803004:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554897280:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760795.803051:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68136960, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x207f0 00010000:00010000:3.0:1789760795.803062:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c7113 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.803079:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760795.803084:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.803092:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c7113 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68136960->68141055] (req 68136960->68141055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.803108:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c7113 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68136960->68141055] (req 68136960->68141055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.819187:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983944 is committed 00002000:00010000:2.0:1789760795.819392:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68136960 -> 68141056 00002000:00010000:2.0:1789760795.819398:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133104 -> 133112 00010000:00010000:2.0:1789760795.819412:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c7113 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68136960->68141055] (req 68136960->68141055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.819425:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c7113 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68136960->68141055] (req 68136960->68141055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.819433:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c7113 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68136960->68141055] (req 68136960->68141055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.819441:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760795.819446:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c7113 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68136960->68141055] (req 68136960->68141055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.819455:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c7113 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68136960->68141055] (req 68136960->68141055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.819480:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c7113 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68136960->68141055] (req 68136960->68141055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.819488:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c7113 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68136960->68141055] (req 68136960->68141055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.819629:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554897280:12345-192.168.203.22@tcp:4:dd.0 Request processed in 16623us (16899us total) trans 4294983944 rc 0/0 00000100:00100000:2.0:1789760795.819640:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16646 00000100:00100000:2.0:1789760795.819650:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739035548926 time 739048609608 state 0x2 ref 0 00000100:00100000:2.0:1789760795.827234:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760795.827327:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554897408 00000100:00100000:1.0:1789760795.827336:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554897408 00000100:00100000:1.0:1789760795.827417:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739035548926 time 739048609608 state 0x0 ref 1 00000100:00100000:1.0:1789760795.827433:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739048870024 time 739048609608 state 0x1 ref 1 00000100:00100000:1.0:1789760795.827462:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739048870024 time 739073080561 state 0x0 ref 1 00000100:00100000:1.0:1789760795.827473:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554897408:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760795.827519:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68141056, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x207f8 00010000:00010000:1.0:1789760795.827527:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c711a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.827541:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760795.827546:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.827551:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c711a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68141056->68145151] (req 68141056->68145151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.827566:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c711a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68141056->68145151] (req 68141056->68145151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.833775:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983945 is committed 00002000:00010000:1.0:1789760795.833887:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68141056 -> 68145152 00002000:00010000:1.0:1789760795.833892:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133112 -> 133120 00010000:00010000:1.0:1789760795.833915:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c711a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68141056->68145151] (req 68141056->68145151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.833928:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c711a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68141056->68145151] (req 68141056->68145151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.833943:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c711a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68141056->68145151] (req 68141056->68145151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.833951:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.833963:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c711a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68141056->68145151] (req 68141056->68145151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.833973:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c711a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68141056->68145151] (req 68141056->68145151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.833994:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c711a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68141056->68145151] (req 68141056->68145151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.834000:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c711a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68141056->68145151] (req 68141056->68145151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.834095:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554897408:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6618us (6865us total) trans 4294983945 rc 0/0 00000100:00100000:1.0:1789760795.834105:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16647 00000100:00100000:1.0:1789760795.834124:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739048870024 time 739073080561 state 0x2 ref 0 00000100:00100000:0.0:1789760795.840505:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760795.840659:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554897536 00000100:00100000:1.0:1789760795.840669:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554897536 00000100:00100000:1.0:1789760795.840725:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739048870024 time 739073080561 state 0x0 ref 1 00000100:00100000:1.0:1789760795.840741:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739073340977 time 739073080561 state 0x1 ref 1 00000100:00100000:1.0:1789760795.840778:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739073340977 time 739086395597 state 0x0 ref 1 00000100:00100000:1.0:1789760795.840790:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e54a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554897536:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760795.840836:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68145152, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20800 00010000:00010000:1.0:1789760795.840845:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c7121 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.840862:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760795.840865:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.840872:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c7121 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68145152->68149247] (req 68145152->68149247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.840885:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c7121 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68145152->68149247] (req 68145152->68149247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.850835:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983946 is committed 00002000:00010000:1.0:1789760795.851016:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68145152 -> 68149248 00002000:00010000:1.0:1789760795.851021:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133120 -> 133128 00010000:00010000:1.0:1789760795.851035:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c7121 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68145152->68149247] (req 68145152->68149247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.851049:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c7121 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68145152->68149247] (req 68145152->68149247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.851058:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c7121 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68145152->68149247] (req 68145152->68149247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.851072:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.851077:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c7121 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68145152->68149247] (req 68145152->68149247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.851101:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c7121 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68145152->68149247] (req 68145152->68149247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.851120:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c7121 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68145152->68149247] (req 68145152->68149247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.851127:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c7121 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68145152->68149247] (req 68145152->68149247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.851228:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554897536:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10426us (10721us total) trans 4294983946 rc 0/0 00000100:00100000:1.0:1789760795.851240:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16648 00000100:00100000:1.0:1789760795.851250:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739073340977 time 739086395597 state 0x2 ref 0 00000100:00100000:2.0:1789760795.859988:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760795.860111:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554897664 00000100:00100000:1.0:1789760795.860120:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554897664 00000100:00100000:1.0:1789760795.860174:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739073340977 time 739086395597 state 0x0 ref 1 00000100:00100000:1.0:1789760795.860188:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739086656013 time 739086395597 state 0x1 ref 1 00000100:00100000:1.0:1789760795.860250:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739086656013 time 739105868309 state 0x0 ref 1 00000100:00100000:1.0:1789760795.860264:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554897664:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760795.860310:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68149248, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20808 00010000:00010000:1.0:1789760795.860318:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c7128 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.860335:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760795.860339:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.860345:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c7128 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68149248->68153343] (req 68149248->68153343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.860358:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c7128 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68149248->68153343] (req 68149248->68153343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.867131:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983947 is committed 00002000:00010000:1.0:1789760795.869336:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68149248 -> 68153344 00002000:00010000:1.0:1789760795.869346:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133128 -> 133136 00010000:00010000:1.0:1789760795.869360:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c7128 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68149248->68153343] (req 68149248->68153343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.869373:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c7128 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68149248->68153343] (req 68149248->68153343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.869434:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c7128 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68149248->68153343] (req 68149248->68153343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.869447:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.869452:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c7128 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68149248->68153343] (req 68149248->68153343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.869463:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c7128 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68149248->68153343] (req 68149248->68153343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.869483:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c7128 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68149248->68153343] (req 68149248->68153343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.869490:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000ebab2af/0x1f99687d854c7128 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68149248->68153343] (req 68149248->68153343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.869595:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554897664:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9326us (9612us total) trans 4294983947 rc 0/0 00000100:00100000:1.0:1789760795.869607:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16649 00000100:00100000:1.0:1789760795.869616:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739086656013 time 739105868309 state 0x2 ref 0 00000100:00100000:2.0:1789760795.878576:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760795.878976:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554897792 00000100:00100000:0.0:1789760795.878987:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554897792 00000100:00100000:0.0:1789760795.879044:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739086656013 time 739105868309 state 0x0 ref 1 00000100:00100000:0.0:1789760795.879070:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739106128725 time 739105868309 state 0x1 ref 1 00000100:00100000:0.0:1789760795.879110:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739106128725 time 739124728525 state 0x0 ref 1 00000100:00100000:0.0:1789760795.879124:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554897792:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760795.879171:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68153344, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20810 00010000:00010000:0.0:1789760795.879181:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c712f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.879197:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760795.879202:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760795.879208:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c712f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68153344->68157439] (req 68153344->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.879560:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c712f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68153344->68157439] (req 68153344->68157439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.889886:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983948 is committed 00002000:00010000:2.0:1789760795.890106:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68153344 -> 68157440 00002000:00010000:2.0:1789760795.890112:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133136 -> 133144 00010000:00010000:2.0:1789760795.890125:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c712f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68153344->68157439] (req 68153344->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.890139:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c712f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68153344->68157439] (req 68153344->68157439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.890149:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c712f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68153344->68157439] (req 68153344->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.890162:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760795.890167:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c712f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68153344->68157439] (req 68153344->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.890177:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c712f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68153344->68157439] (req 68153344->68157439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.890195:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c712f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68153344->68157439] (req 68153344->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.890204:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c712f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68153344->68157439] (req 68153344->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.890300:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554897792:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11175us (11730us total) trans 4294983948 rc 0/0 00000100:00100000:2.0:1789760795.890312:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16650 00000100:00100000:2.0:1789760795.890322:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739106128725 time 739124728525 state 0x2 ref 0 00000100:00100000:0.0:1789760795.902290:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760795.909220:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554897920 00000100:00100000:2.0:1789760795.909234:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554897920 00000100:00100000:2.0:1789760795.909299:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739106128725 time 739124728525 state 0x0 ref 1 00000100:00100000:2.0:1789760795.909315:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739124988941 time 739124728525 state 0x1 ref 1 00000100:00100000:2.0:1789760795.909351:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739124988941 time 739154969277 state 0x0 ref 1 00000100:00100000:2.0:1789760795.909368:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554897920:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760795.909844:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68157440, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20818 00010000:00010000:2.0:1789760795.909854:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7136 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.909870:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760795.909875:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.909881:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7136 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68157440->68161535] (req 68157440->68161535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.909895:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7136 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68157440->68161535] (req 68157440->68161535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.917948:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983949 is committed 00002000:00010000:2.0:1789760795.918232:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68157440 -> 68161536 00002000:00010000:2.0:1789760795.918238:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133144 -> 133152 00010000:00010000:2.0:1789760795.918251:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7136 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68157440->68161535] (req 68157440->68161535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.918266:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7136 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68157440->68161535] (req 68157440->68161535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.918276:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7136 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68157440->68161535] (req 68157440->68161535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.918285:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760795.918291:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7136 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68157440->68161535] (req 68157440->68161535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.918300:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7136 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68157440->68161535] (req 68157440->68161535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.918321:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7136 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68157440->68161535] (req 68157440->68161535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.918330:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7136 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68157440->68161535] (req 68157440->68161535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.919514:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554897920:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10141us (17225us total) trans 4294983949 rc 0/0 00000100:00100000:2.0:1789760795.919528:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16651 00000100:00100000:2.0:1789760795.919537:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739124988941 time 739154969277 state 0x2 ref 0 00000100:00100000:2.0:1789760795.929169:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760795.931705:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554898048 00000100:00100000:3.0:1789760795.931719:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554898048 00000100:00100000:3.0:1789760795.931776:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739124988941 time 739154969277 state 0x0 ref 1 00000100:00100000:3.0:1789760795.931791:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739155229693 time 739154969277 state 0x1 ref 1 00000100:00100000:3.0:1789760795.931823:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739155229693 time 739177442262 state 0x0 ref 1 00000100:00100000:3.0:1789760795.931836:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554898048:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760795.931883:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68161536, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20820 00010000:00010000:3.0:1789760795.931892:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c713d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.931908:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760795.931913:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.931920:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c713d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68161536->68165631] (req 68161536->68165631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.931935:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c713d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68161536->68165631] (req 68161536->68165631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.938165:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983950 is committed 00002000:00010000:3.0:1789760795.938274:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68161536 -> 68165632 00002000:00010000:3.0:1789760795.938279:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133152 -> 133160 00010000:00010000:3.0:1789760795.938293:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c713d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68161536->68165631] (req 68161536->68165631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.938306:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c713d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68161536->68165631] (req 68161536->68165631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.938315:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c713d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68161536->68165631] (req 68161536->68165631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.938321:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760795.938327:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c713d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68161536->68165631] (req 68161536->68165631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.938335:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c713d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68161536->68165631] (req 68161536->68165631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.938354:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c713d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68161536->68165631] (req 68161536->68165631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.938362:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c713d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68161536->68165631] (req 68161536->68165631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760795.938470:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554898048:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6632us (9306us total) trans 4294983950 rc 0/0 00000100:00100000:3.0:1789760795.938489:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16652 00000100:00100000:3.0:1789760795.938498:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739155229693 time 739177442262 state 0x2 ref 0 00000100:00100000:1.0:1789760795.949858:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760795.950189:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554898176 00000100:00100000:2.0:1789760795.950199:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554898176 00000100:00100000:2.0:1789760795.950250:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739155229693 time 739177442262 state 0x0 ref 1 00000100:00100000:2.0:1789760795.950264:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739177702678 time 739177442262 state 0x1 ref 1 00000100:00100000:2.0:1789760795.950297:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739177702678 time 739195915423 state 0x0 ref 1 00000100:00100000:2.0:1789760795.950309:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3786cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554898176:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760795.950354:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68165632, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20828 00010000:00010000:2.0:1789760795.950363:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7144 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.950658:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760795.950665:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.950672:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7144 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68165632->68169727] (req 68165632->68169727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.950686:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7144 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68165632->68169727] (req 68165632->68169727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760795.959923:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983951 is committed 00002000:00010000:0.0:1789760795.960079:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68165632 -> 68169728 00002000:00010000:0.0:1789760795.960085:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133160 -> 133168 00010000:00010000:0.0:1789760795.960099:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7144 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68165632->68169727] (req 68165632->68169727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.960112:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7144 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68165632->68169727] (req 68165632->68169727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.960122:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7144 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68165632->68169727] (req 68165632->68169727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.960131:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760795.960135:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7144 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68165632->68169727] (req 68165632->68169727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.960145:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7144 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68165632->68169727] (req 68165632->68169727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.960165:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7144 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68165632->68169727] (req 68165632->68169727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760795.960174:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7144 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68165632->68169727] (req 68165632->68169727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760795.960266:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3786cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554898176:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9954us (10412us total) trans 4294983951 rc 0/0 00000100:00100000:0.0:1789760795.960277:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16653 00000100:00100000:0.0:1789760795.960288:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739177702678 time 739195915423 state 0x2 ref 0 00000100:00100000:0.0:1789760795.968227:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760795.968515:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554898304 00000100:00100000:2.0:1789760795.968525:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554898304 00000100:00100000:2.0:1789760795.968592:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739177702678 time 739195915423 state 0x0 ref 1 00000100:00100000:2.0:1789760795.968608:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739196175839 time 739195915423 state 0x1 ref 1 00000100:00100000:2.0:1789760795.968639:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739196175839 time 739214258129 state 0x0 ref 1 00000100:00100000:2.0:1789760795.968651:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e57b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554898304:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760795.968698:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68169728, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20830 00010000:00010000:2.0:1789760795.968706:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c714b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.968728:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760795.968732:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760795.968738:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c714b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68169728->68173823] (req 68169728->68173823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.968752:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c714b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68169728->68173823] (req 68169728->68173823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.975656:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983952 is committed 00002000:00010000:2.0:1789760795.975771:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68169728 -> 68173824 00002000:00010000:2.0:1789760795.975776:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133168 -> 133176 00010000:00010000:2.0:1789760795.975788:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c714b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68169728->68173823] (req 68169728->68173823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.975800:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c714b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68169728->68173823] (req 68169728->68173823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.975810:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c714b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68169728->68173823] (req 68169728->68173823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.975817:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760795.975822:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c714b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68169728->68173823] (req 68169728->68173823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.975831:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c714b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68169728->68173823] (req 68169728->68173823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.975853:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c714b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68169728->68173823] (req 68169728->68173823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760795.975861:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c714b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68169728->68173823] (req 68169728->68173823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760795.975961:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e57b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554898304:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7301us (7732us total) trans 4294983952 rc 0/0 00000100:00100000:2.0:1789760795.975970:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16654 00000100:00100000:2.0:1789760795.975979:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739196175839 time 739214258129 state 0x2 ref 0 00000100:00100000:2.0:1789760795.981394:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760795.981562:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554898432 00000100:00100000:1.0:1789760795.981572:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554898432 00000100:00100000:1.0:1789760795.981699:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739196175839 time 739214258129 state 0x0 ref 1 00000100:00100000:1.0:1789760795.981714:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739214518545 time 739214258129 state 0x1 ref 1 00000100:00100000:1.0:1789760795.981747:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739214518545 time 739227364880 state 0x0 ref 1 00000100:00100000:1.0:1789760795.981759:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554898432:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760795.981951:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68173824, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20838 00010000:00010000:1.0:1789760795.982150:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c7152 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.982167:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760795.982171:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760795.982177:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c7152 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68173824->68177919] (req 68173824->68177919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.982190:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c7152 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68173824->68177919] (req 68173824->68177919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760795.988228:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983953 is committed 00002000:00010000:1.0:1789760795.988839:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68173824 -> 68177920 00002000:00010000:1.0:1789760795.988846:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133176 -> 133184 00010000:00010000:1.0:1789760795.988862:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c7152 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68173824->68177919] (req 68173824->68177919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.988877:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c7152 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68173824->68177919] (req 68173824->68177919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.988886:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c7152 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68173824->68177919] (req 68173824->68177919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.988894:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760795.988899:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c7152 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68173824->68177919] (req 68173824->68177919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.988914:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c7152 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68173824->68177919] (req 68173824->68177919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.988934:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c7152 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68173824->68177919] (req 68173824->68177919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760795.988942:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c7152 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68173824->68177919] (req 68173824->68177919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760795.989099:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554898432:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7330us (7720us total) trans 4294983953 rc 0/0 00000100:00100000:1.0:1789760795.989110:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16655 00000100:00100000:1.0:1789760795.989119:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739214518545 time 739227364880 state 0x2 ref 0 00000100:00100000:1.0:1789760795.995980:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760795.996052:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554898560 00000100:00100000:3.0:1789760795.996062:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554898560 00000100:00100000:3.0:1789760795.996116:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739214518545 time 739227364880 state 0x0 ref 1 00000100:00100000:3.0:1789760795.996132:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739227625296 time 739227364880 state 0x1 ref 1 00000100:00100000:3.0:1789760795.996158:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739227625296 time 739241777449 state 0x0 ref 1 00000100:00100000:3.0:1789760795.996170:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554898560:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760795.996215:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68177920, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20840 00010000:00010000:3.0:1789760795.996236:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7159 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.996252:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760795.996256:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760795.996263:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7159 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68177920->68182015] (req 68177920->68182015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760795.996277:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7159 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68177920->68182015] (req 68177920->68182015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.006555:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983954 is committed 00002000:00010000:2.0:1789760796.014546:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68177920 -> 68182016 00002000:00010000:2.0:1789760796.014556:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133184 -> 133192 00010000:00010000:2.0:1789760796.014570:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7159 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68177920->68182015] (req 68177920->68182015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.014596:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7159 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68177920->68182015] (req 68177920->68182015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.014606:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7159 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68177920->68182015] (req 68177920->68182015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.014891:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760796.014899:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7159 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68177920->68182015] (req 68177920->68182015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.014910:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7159 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68177920->68182015] (req 68177920->68182015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.014931:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7159 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68177920->68182015] (req 68177920->68182015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.014940:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7159 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68177920->68182015] (req 68177920->68182015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.015125:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554898560:12345-192.168.203.22@tcp:4:dd.0 Request processed in 18949us (19148us total) trans 4294983954 rc 0/0 00000100:00100000:2.0:1789760796.015137:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16656 00000100:00100000:2.0:1789760796.015147:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739227625296 time 739241777449 state 0x2 ref 0 00000100:00100000:0.0:1789760796.022292:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760796.022505:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554898688 00000100:00100000:1.0:1789760796.022515:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554898688 00000100:00100000:1.0:1789760796.022570:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739227625296 time 739241777449 state 0x0 ref 1 00000100:00100000:1.0:1789760796.022595:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739242037865 time 739241777449 state 0x1 ref 1 00000100:00100000:1.0:1789760796.022670:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739242037865 time 739268288179 state 0x0 ref 1 00000100:00100000:1.0:1789760796.022694:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e57480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554898688:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760796.022745:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68182016, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20848 00010000:00010000:1.0:1789760796.022756:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c7160 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.022772:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760796.022776:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.022783:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c7160 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68182016->68186111] (req 68182016->68186111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.022797:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c7160 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68182016->68186111] (req 68182016->68186111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.039987:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983955 is committed 00002000:00010000:1.0:1789760796.040283:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68182016 -> 68186112 00002000:00010000:1.0:1789760796.040288:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133192 -> 133200 00010000:00010000:1.0:1789760796.040300:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c7160 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68182016->68186111] (req 68182016->68186111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.040313:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c7160 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68182016->68186111] (req 68182016->68186111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.040322:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c7160 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68182016->68186111] (req 68182016->68186111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.040329:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760796.040334:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c7160 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68182016->68186111] (req 68182016->68186111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.040344:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c7160 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68182016->68186111] (req 68182016->68186111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.040363:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c7160 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68182016->68186111] (req 68182016->68186111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.040370:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c7160 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68182016->68186111] (req 68182016->68186111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.040637:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e57480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554898688:12345-192.168.203.22@tcp:4:dd.0 Request processed in 17941us (18350us total) trans 4294983955 rc 0/0 00000100:00100000:1.0:1789760796.040653:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16657 00000100:00100000:1.0:1789760796.040730:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739242037865 time 739268288179 state 0x2 ref 0 00000100:00100000:1.0:1789760796.050048:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb054c8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573923456:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1789760796.050119:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760796.051149:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573923456 00000100:00100000:0.0:1789760796.051160:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573923456 00000100:00100000:0.0:1789760796.051213:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 397 00000100:00100000:0.0:1789760796.051232:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573923456:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1789760796.051268:0:6526:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446383616 avail 1300144128 00000100:00100000:2.0:1789760796.051419:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760796.051446:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554898816 00000100:00100000:2.0:1789760796.051452:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554898816 00000100:00100000:2.0:1789760796.051489:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739242037865 time 739268288179 state 0x0 ref 1 00000100:00100000:2.0:1789760796.051501:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739268548595 time 739268288179 state 0x1 ref 1 00000100:00100000:2.0:1789760796.051521:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739268548595 time 739297139950 state 0x0 ref 1 00000100:00100000:2.0:1789760796.051531:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554898816:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760796.051587:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68186112, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20850 00010000:00010000:2.0:1789760796.051596:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7167 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.051612:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760796.051617:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.051637:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7167 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68186112->68190207] (req 68186112->68190207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.052034:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7167 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68186112->68190207] (req 68186112->68190207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.055366:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573923456:12345-0@lo:41:osp-pre-1-0.0 Request processed in 4126us (5236us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760796.056050:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 397 00000100:00100000:1.0:1789760796.056312:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb054c8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573923456:0@lo:41:osp-pre-1-0.0 00000001:00080000:2.0:1789760796.062892:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983956 is committed 00002000:00010000:3.0:1789760796.063047:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68186112 -> 68190208 00002000:00010000:3.0:1789760796.063053:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133200 -> 133208 00010000:00010000:3.0:1789760796.063067:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7167 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68186112->68190207] (req 68186112->68190207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.063081:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7167 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68186112->68190207] (req 68186112->68190207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.063091:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7167 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68186112->68190207] (req 68186112->68190207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.063099:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760796.063103:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7167 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68186112->68190207] (req 68186112->68190207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.063112:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7167 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68186112->68190207] (req 68186112->68190207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.063132:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7167 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68186112->68190207] (req 68186112->68190207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.063330:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7167 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68186112->68190207] (req 68186112->68190207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.063677:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554898816:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12140us (12253us total) trans 4294983956 rc 0/0 00000100:00100000:2.0:1789760796.063690:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16658 00000100:00100000:2.0:1789760796.063700:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739268548595 time 739297139950 state 0x2 ref 0 00000100:00100000:2.0:1789760796.071371:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760796.071620:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554898944 00000100:00100000:1.0:1789760796.071630:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554898944 00000100:00100000:1.0:1789760796.071680:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739268548595 time 739297139950 state 0x0 ref 1 00000100:00100000:1.0:1789760796.071695:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739297400366 time 739297139950 state 0x1 ref 1 00000100:00100000:1.0:1789760796.071764:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739297400366 time 739317382493 state 0x0 ref 1 00000100:00100000:1.0:1789760796.071777:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554898944:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760796.071827:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68190208, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20858 00010000:00010000:1.0:1789760796.071838:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c716e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.071855:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760796.071859:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.071866:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c716e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68190208->68194303] (req 68190208->68194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.071880:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c716e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68190208->68194303] (req 68190208->68194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.079034:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983957 is committed 00002000:00010000:1.0:1789760796.079142:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68190208 -> 68194304 00002000:00010000:1.0:1789760796.079147:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133208 -> 133216 00010000:00010000:1.0:1789760796.079160:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c716e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68190208->68194303] (req 68190208->68194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.079172:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c716e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68190208->68194303] (req 68190208->68194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.079180:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c716e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68190208->68194303] (req 68190208->68194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.079187:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760796.079192:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c716e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68190208->68194303] (req 68190208->68194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.079202:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c716e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68190208->68194303] (req 68190208->68194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.079223:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c716e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68190208->68194303] (req 68190208->68194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.079231:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c716e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68190208->68194303] (req 68190208->68194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.079326:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554898944:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7546us (7960us total) trans 4294983957 rc 0/0 00000100:00100000:1.0:1789760796.079338:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16659 00000100:00100000:1.0:1789760796.079347:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739297400366 time 739317382493 state 0x2 ref 0 00000100:00100000:0.0:1789760796.088663:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760796.088767:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554899072 00000100:00100000:3.0:1789760796.088777:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554899072 00000100:00100000:3.0:1789760796.088826:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739297400366 time 739317382493 state 0x0 ref 1 00000100:00100000:3.0:1789760796.088841:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739317642909 time 739317382493 state 0x1 ref 1 00000100:00100000:3.0:1789760796.088871:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739317642909 time 739334490178 state 0x0 ref 1 00000100:00100000:3.0:1789760796.088882:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e54700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554899072:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760796.088923:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68194304, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20860 00010000:00010000:3.0:1789760796.088931:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c7175 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.088946:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760796.088950:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.088964:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c7175 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68194304->68198399] (req 68194304->68198399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.088979:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c7175 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68194304->68198399] (req 68194304->68198399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.099713:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983958 is committed 00002000:00010000:2.0:1789760796.102362:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68194304 -> 68198400 00002000:00010000:2.0:1789760796.102372:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133216 -> 133224 00010000:00010000:2.0:1789760796.102541:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c7175 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68194304->68198399] (req 68194304->68198399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.102556:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c7175 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68194304->68198399] (req 68194304->68198399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.102571:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c7175 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68194304->68198399] (req 68194304->68198399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.102587:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760796.102592:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c7175 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68194304->68198399] (req 68194304->68198399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.102664:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c7175 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68194304->68198399] (req 68194304->68198399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.102684:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c7175 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68194304->68198399] (req 68194304->68198399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.102692:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c7175 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68194304->68198399] (req 68194304->68198399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.102900:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554899072:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14013us (14515us total) trans 4294983958 rc 0/0 00000100:00100000:2.0:1789760796.102911:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16660 00000100:00100000:2.0:1789760796.102919:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739317642909 time 739334490178 state 0x2 ref 0 00000100:00100000:2.0:1789760796.106931:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760796.107019:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554899200 00000100:00100000:0.0:1789760796.107028:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554899200 00000100:00100000:0.0:1789760796.107080:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739317642909 time 739334490178 state 0x0 ref 1 00000100:00100000:0.0:1789760796.107099:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739334750594 time 739334490178 state 0x1 ref 1 00000100:00100000:0.0:1789760796.107177:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739334750594 time 739352795195 state 0x0 ref 1 00000100:00100000:0.0:1789760796.107190:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554899200:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760796.107237:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68198400, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20868 00010000:00010000:0.0:1789760796.107247:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c717c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.107264:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760796.107268:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.107275:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c717c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68198400->68202495] (req 68198400->68202495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.107291:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c717c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68198400->68202495] (req 68198400->68202495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.114933:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983959 is committed 00002000:00010000:0.0:1789760796.115051:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68198400 -> 68202496 00002000:00010000:0.0:1789760796.115057:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133224 -> 133232 00010000:00010000:0.0:1789760796.115070:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c717c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68198400->68202495] (req 68198400->68202495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.115084:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c717c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68198400->68202495] (req 68198400->68202495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.115094:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c717c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68198400->68202495] (req 68198400->68202495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.115102:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760796.115107:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c717c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68198400->68202495] (req 68198400->68202495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.115116:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c717c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68198400->68202495] (req 68198400->68202495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.115136:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c717c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68198400->68202495] (req 68198400->68202495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.115144:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c717c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68198400->68202495] (req 68198400->68202495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.115240:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554899200:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8047us (8314us total) trans 4294983959 rc 0/0 00000100:00100000:0.0:1789760796.115251:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16661 00000100:00100000:0.0:1789760796.115261:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739334750594 time 739352795195 state 0x2 ref 0 00000100:00100000:1.0:1789760796.129778:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760796.130065:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554899328 00000100:00100000:2.0:1789760796.130076:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554899328 00000100:00100000:2.0:1789760796.130125:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739334750594 time 739352795195 state 0x0 ref 1 00000100:00100000:2.0:1789760796.130143:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739353055611 time 739352795195 state 0x1 ref 1 00000100:00100000:2.0:1789760796.130176:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739353055611 time 739375794037 state 0x0 ref 1 00000100:00100000:2.0:1789760796.130189:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554899328:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760796.130244:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68202496, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20870 00010000:00010000:2.0:1789760796.130254:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7183 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.130270:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760796.130274:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.130280:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7183 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68202496->68206591] (req 68202496->68206591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.130292:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7183 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68202496->68206591] (req 68202496->68206591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.139124:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983960 is committed 00002000:00010000:2.0:1789760796.139226:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68202496 -> 68206592 00002000:00010000:2.0:1789760796.139231:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133232 -> 133240 00010000:00010000:2.0:1789760796.139244:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7183 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68202496->68206591] (req 68202496->68206591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.139256:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7183 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68202496->68206591] (req 68202496->68206591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.139265:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7183 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68202496->68206591] (req 68202496->68206591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.139273:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760796.139278:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7183 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68202496->68206591] (req 68202496->68206591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.139286:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7183 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68202496->68206591] (req 68202496->68206591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.139303:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7183 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68202496->68206591] (req 68202496->68206591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.140021:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7183 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68202496->68206591] (req 68202496->68206591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.140180:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554899328:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9987us (10406us total) trans 4294983960 rc 0/0 00000100:00100000:2.0:1789760796.140190:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16662 00000100:00100000:2.0:1789760796.140200:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739353055611 time 739375794037 state 0x2 ref 0 00000100:00100000:2.0:1789760796.146832:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760796.148743:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554899456 00000100:00100000:1.0:1789760796.148767:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554899456 00000100:00100000:1.0:1789760796.148829:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739353055611 time 739375794037 state 0x0 ref 1 00000100:00100000:1.0:1789760796.148845:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739376054453 time 739375794037 state 0x1 ref 1 00000100:00100000:1.0:1789760796.148880:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739376054453 time 739394498399 state 0x0 ref 1 00000100:00100000:1.0:1789760796.148892:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554899456:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760796.148939:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68206592, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20878 00010000:00010000:1.0:1789760796.148948:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c718a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.149145:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760796.149151:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.149158:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c718a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68206592->68210687] (req 68206592->68210687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.149174:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c718a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68206592->68210687] (req 68206592->68210687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.158109:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983961 is committed 00002000:00010000:1.0:1789760796.158429:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68206592 -> 68210688 00002000:00010000:1.0:1789760796.158434:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133240 -> 133248 00010000:00010000:1.0:1789760796.158447:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c718a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68206592->68210687] (req 68206592->68210687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.158460:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c718a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68206592->68210687] (req 68206592->68210687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.158470:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c718a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68206592->68210687] (req 68206592->68210687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.158477:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760796.158482:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c718a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68206592->68210687] (req 68206592->68210687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.158491:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c718a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68206592->68210687] (req 68206592->68210687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.158516:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c718a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68206592->68210687] (req 68206592->68210687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.158524:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008712c50d/0x1f99687d854c718a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68206592->68210687] (req 68206592->68210687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.158685:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554899456:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9789us (11857us total) trans 4294983961 rc 0/0 00000100:00100000:1.0:1789760796.158695:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16663 00000100:00100000:1.0:1789760796.158704:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739376054453 time 739394498399 state 0x2 ref 0 00000100:00100000:1.0:1789760796.165687:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760796.165829:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554899584 00000100:00100000:3.0:1789760796.165840:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554899584 00000100:00100000:3.0:1789760796.165895:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739376054453 time 739394498399 state 0x0 ref 1 00000100:00100000:3.0:1789760796.165910:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739394758815 time 739394498399 state 0x1 ref 1 00000100:00100000:3.0:1789760796.166047:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739394758815 time 739411665012 state 0x0 ref 1 00000100:00100000:3.0:1789760796.166060:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554899584:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760796.166102:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68210688, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20880 00010000:00010000:3.0:1789760796.166112:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c7191 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.166128:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760796.166132:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.166138:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c7191 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68210688->68214783] (req 68210688->68214783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.166150:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c7191 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68210688->68214783] (req 68210688->68214783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.174699:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983962 is committed 00002000:00010000:2.0:1789760796.174819:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68210688 -> 68214784 00002000:00010000:2.0:1789760796.174825:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133248 -> 133256 00010000:00010000:2.0:1789760796.174838:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c7191 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68210688->68214783] (req 68210688->68214783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.174851:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c7191 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68210688->68214783] (req 68210688->68214783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.174861:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c7191 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68210688->68214783] (req 68210688->68214783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.174869:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760796.174874:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c7191 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68210688->68214783] (req 68210688->68214783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.174884:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c7191 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68210688->68214783] (req 68210688->68214783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.175112:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c7191 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68210688->68214783] (req 68210688->68214783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.175122:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f57987f0/0x1f99687d854c7191 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68210688->68214783] (req 68210688->68214783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.175257:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554899584:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9194us (9574us total) trans 4294983962 rc 0/0 00000100:00100000:2.0:1789760796.175268:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16664 00000100:00100000:2.0:1789760796.175277:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739394758815 time 739411665012 state 0x2 ref 0 00000100:00100000:2.0:1789760796.179287:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760796.179416:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554899712 00000100:00100000:0.0:1789760796.179427:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554899712 00000100:00100000:0.0:1789760796.179482:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739394758815 time 739411665012 state 0x0 ref 1 00000100:00100000:0.0:1789760796.179502:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739411925428 time 739411665012 state 0x1 ref 1 00000100:00100000:0.0:1789760796.179523:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739411925428 time 739425142386 state 0x0 ref 1 00000100:00100000:0.0:1789760796.179534:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554899712:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760796.179590:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68214784, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20888 00010000:00010000:0.0:1789760796.179600:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c7198 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.179615:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760796.179619:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.179626:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c7198 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68214784->68218879] (req 68214784->68218879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.179639:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c7198 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68214784->68218879] (req 68214784->68218879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.187604:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983963 is committed 00002000:00010000:0.0:1789760796.189122:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68214784 -> 68218880 00002000:00010000:0.0:1789760796.189132:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133256 -> 133264 00010000:00010000:0.0:1789760796.189146:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c7198 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68214784->68218879] (req 68214784->68218879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.189158:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c7198 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68214784->68218879] (req 68214784->68218879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.189167:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c7198 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68214784->68218879] (req 68214784->68218879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.189175:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760796.189180:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c7198 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68214784->68218879] (req 68214784->68218879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.189189:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c7198 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68214784->68218879] (req 68214784->68218879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.189210:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c7198 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68214784->68218879] (req 68214784->68218879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.189218:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c7198 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68214784->68218879] (req 68214784->68218879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.189560:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554899712:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10022us (10281us total) trans 4294983963 rc 0/0 00000100:00100000:0.0:1789760796.189572:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16665 00000100:00100000:0.0:1789760796.189592:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739411925428 time 739425142386 state 0x2 ref 0 00000100:00100000:0.0:1789760796.193485:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760796.193625:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554899840 00000100:00100000:1.0:1789760796.193636:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554899840 00000100:00100000:1.0:1789760796.193697:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739411925428 time 739425142386 state 0x0 ref 1 00000100:00100000:1.0:1789760796.193710:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739425402802 time 739425142386 state 0x1 ref 1 00000100:00100000:1.0:1789760796.194137:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739425402802 time 739439755192 state 0x0 ref 1 00000100:00100000:1.0:1789760796.194150:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e57800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554899840:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760796.194197:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68218880, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20890 00010000:00010000:1.0:1789760796.194207:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c719f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.194228:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760796.194232:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.194239:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c719f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68218880->68222975] (req 68218880->68222975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.194253:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c719f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68218880->68222975] (req 68218880->68222975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.208586:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983964 is committed 00002000:00010000:1.0:1789760796.208706:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68218880 -> 68222976 00002000:00010000:1.0:1789760796.208711:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133264 -> 133272 00010000:00010000:1.0:1789760796.208724:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c719f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68218880->68222975] (req 68218880->68222975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.208737:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c719f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68218880->68222975] (req 68218880->68222975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.208747:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c719f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68218880->68222975] (req 68218880->68222975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.208755:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760796.208760:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c719f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68218880->68222975] (req 68218880->68222975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.208769:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c719f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68218880->68222975] (req 68218880->68222975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.208804:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c719f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68218880->68222975] (req 68218880->68222975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.208811:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c719f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68218880->68222975] (req 68218880->68222975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.208908:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e57800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554899840:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14756us (15429us total) trans 4294983964 rc 0/0 00000100:00100000:1.0:1789760796.208919:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16666 00000100:00100000:1.0:1789760796.208930:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739425402802 time 739439755192 state 0x2 ref 0 00000100:00100000:1.0:1789760796.213064:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760796.213251:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554899968 00000100:00100000:2.0:1789760796.213260:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554899968 00000100:00100000:2.0:1789760796.213314:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739425402802 time 739439755192 state 0x0 ref 1 00000100:00100000:2.0:1789760796.213344:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739440015608 time 739439755192 state 0x1 ref 1 00000100:00100000:2.0:1789760796.213409:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739440015608 time 739458994920 state 0x0 ref 1 00000100:00100000:2.0:1789760796.213423:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554899968:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760796.213475:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68222976, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20898 00010000:00010000:2.0:1789760796.213484:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71a6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.213500:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760796.213505:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.213511:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68222976->68227071] (req 68222976->68227071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.213525:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68222976->68227071] (req 68222976->68227071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.221930:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983965 is committed 00002000:00010000:2.0:1789760796.222116:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68222976 -> 68227072 00002000:00010000:2.0:1789760796.222122:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133272 -> 133280 00010000:00010000:2.0:1789760796.222136:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71a6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68222976->68227071] (req 68222976->68227071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.222150:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71a6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68222976->68227071] (req 68222976->68227071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.222159:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68222976->68227071] (req 68222976->68227071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.222168:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760796.222173:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71a6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68222976->68227071] (req 68222976->68227071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.222183:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71a6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68222976->68227071] (req 68222976->68227071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.222202:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71a6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68222976->68227071] (req 68222976->68227071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.222303:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71a6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68222976->68227071] (req 68222976->68227071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.222647:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554899968:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9220us (9589us total) trans 4294983965 rc 0/0 00000100:00100000:2.0:1789760796.222657:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16667 00000100:00100000:2.0:1789760796.222666:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739440015608 time 739458994920 state 0x2 ref 0 00000100:00100000:2.0:1789760796.232552:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760796.232663:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554900096 00000100:00100000:3.0:1789760796.232675:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554900096 00000100:00100000:3.0:1789760796.232734:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739440015608 time 739458994920 state 0x0 ref 1 00000100:00100000:3.0:1789760796.232752:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739459255336 time 739458994920 state 0x1 ref 1 00000100:00100000:3.0:1789760796.232847:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739459255336 time 739478463354 state 0x0 ref 1 00000100:00100000:3.0:1789760796.232865:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554900096:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760796.232915:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68227072, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x208a0 00010000:00010000:3.0:1789760796.232924:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c71ad lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.232939:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760796.232944:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.232950:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c71ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68227072->68231167] (req 68227072->68231167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.232965:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c71ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68227072->68231167] (req 68227072->68231167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.240243:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983966 is committed 00002000:00010000:3.0:1789760796.240374:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68227072 -> 68231168 00002000:00010000:3.0:1789760796.240416:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133280 -> 133288 00010000:00010000:3.0:1789760796.240429:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c71ad lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68227072->68231167] (req 68227072->68231167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.240441:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c71ad lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68227072->68231167] (req 68227072->68231167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.240450:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c71ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68227072->68231167] (req 68227072->68231167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.240458:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760796.240464:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c71ad lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68227072->68231167] (req 68227072->68231167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.240473:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c71ad lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68227072->68231167] (req 68227072->68231167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.240493:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c71ad lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68227072->68231167] (req 68227072->68231167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.240501:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c71ad lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68227072->68231167] (req 68227072->68231167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.240610:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554900096:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7743us (8062us total) trans 4294983966 rc 0/0 00000100:00100000:3.0:1789760796.240622:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16668 00000100:00100000:3.0:1789760796.240631:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739459255336 time 739478463354 state 0x2 ref 0 00010000:00010000:3.0:1789760796.242532:0:8453:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789760796.247820:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760796.247909:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554900224 00000100:00100000:0.0:1789760796.247920:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554900224 00000100:00100000:0.0:1789760796.247977:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739459255336 time 739478463354 state 0x0 ref 1 00000100:00100000:0.0:1789760796.247993:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739478723770 time 739478463354 state 0x1 ref 1 00000100:00100000:0.0:1789760796.248031:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739478723770 time 739493648598 state 0x0 ref 1 00000100:00100000:0.0:1789760796.248044:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554900224:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760796.248095:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68231168, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x208a8 00010000:00010000:0.0:1789760796.248104:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c71b4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.248119:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760796.248123:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.248129:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c71b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68231168->68235263] (req 68231168->68235263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.248142:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c71b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68231168->68235263] (req 68231168->68235263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.253246:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983967 is committed 00002000:00010000:1.0:1789760796.253553:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68231168 -> 68235264 00002000:00010000:1.0:1789760796.253559:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133288 -> 133296 00010000:00010000:1.0:1789760796.253571:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c71b4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68231168->68235263] (req 68231168->68235263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.253593:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c71b4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68231168->68235263] (req 68231168->68235263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.253603:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c71b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68231168->68235263] (req 68231168->68235263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.253619:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760796.253624:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c71b4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68231168->68235263] (req 68231168->68235263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.253632:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c71b4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68231168->68235263] (req 68231168->68235263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.253652:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c71b4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68231168->68235263] (req 68231168->68235263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.253660:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000079bb0e11/0x1f99687d854c71b4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68231168->68235263] (req 68231168->68235263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.253794:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554900224:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5747us (5980us total) trans 4294983967 rc 0/0 00000100:00100000:1.0:1789760796.253805:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16669 00000100:00100000:1.0:1789760796.253815:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739478723770 time 739493648598 state 0x2 ref 0 00000100:00100000:1.0:1789760796.258809:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760796.259323:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554900352 00000100:00100000:2.0:1789760796.259334:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554900352 00000100:00100000:2.0:1789760796.259417:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739478723770 time 739493648598 state 0x0 ref 1 00000100:00100000:2.0:1789760796.259435:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739493909014 time 739493648598 state 0x1 ref 1 00000100:00100000:2.0:1789760796.259468:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739493909014 time 739505086551 state 0x0 ref 1 00000100:00100000:2.0:1789760796.259482:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554900352:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760796.259538:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68235264, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x208b0 00010000:00010000:2.0:1789760796.259547:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c71bb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.259564:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760796.259569:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.259585:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c71bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68235264->68239359] (req 68235264->68239359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.259599:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c71bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68235264->68239359] (req 68235264->68239359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.270747:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983968 is committed 00002000:00010000:2.0:1789760796.271923:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68235264 -> 68239360 00002000:00010000:2.0:1789760796.271931:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133296 -> 133304 00010000:00010000:2.0:1789760796.271946:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c71bb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68235264->68239359] (req 68235264->68239359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.271957:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c71bb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68235264->68239359] (req 68235264->68239359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.271966:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c71bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68235264->68239359] (req 68235264->68239359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.271973:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760796.271978:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c71bb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68235264->68239359] (req 68235264->68239359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.271987:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c71bb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68235264->68239359] (req 68235264->68239359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.272007:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c71bb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68235264->68239359] (req 68235264->68239359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.272015:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c71bb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68235264->68239359] (req 68235264->68239359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.272306:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554900352:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12821us (13501us total) trans 4294983968 rc 0/0 00000100:00100000:2.0:1789760796.272320:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16670 00000100:00100000:2.0:1789760796.272330:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739493909014 time 739505086551 state 0x2 ref 0 00000100:00100000:2.0:1789760796.275142:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760796.275264:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554900480 00000100:00100000:1.0:1789760796.275274:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554900480 00000100:00100000:1.0:1789760796.275325:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739493909014 time 739505086551 state 0x0 ref 1 00000100:00100000:1.0:1789760796.275340:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739505346967 time 739505086551 state 0x1 ref 1 00000100:00100000:1.0:1789760796.275444:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739505346967 time 739520993605 state 0x0 ref 1 00000100:00100000:1.0:1789760796.275459:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554900480:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760796.275507:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68239360, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x208b8 00010000:00010000:1.0:1789760796.275520:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c71c2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.275606:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760796.275610:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.275618:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c71c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68239360->68243455] (req 68239360->68243455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.275632:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c71c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68239360->68243455] (req 68239360->68243455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.282810:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983969 is committed 00002000:00010000:1.0:1789760796.283269:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68239360 -> 68243456 00002000:00010000:1.0:1789760796.283275:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133304 -> 133312 00010000:00010000:1.0:1789760796.283291:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c71c2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68239360->68243455] (req 68239360->68243455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.283304:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c71c2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68239360->68243455] (req 68239360->68243455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.283314:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c71c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68239360->68243455] (req 68239360->68243455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.283321:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760796.283328:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c71c2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68239360->68243455] (req 68239360->68243455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.283337:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c71c2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68239360->68243455] (req 68239360->68243455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.283359:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c71c2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68239360->68243455] (req 68239360->68243455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.283368:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c71c2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68239360->68243455] (req 68239360->68243455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.283546:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554900480:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8084us (8407us total) trans 4294983969 rc 0/0 00000100:00100000:1.0:1789760796.283559:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16671 00000100:00100000:1.0:1789760796.283571:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739505346967 time 739520993605 state 0x2 ref 0 00000100:00100000:3.0:1789760796.288370:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760796.288605:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554900608 00000100:00100000:2.0:1789760796.288614:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554900608 00000100:00100000:2.0:1789760796.288665:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739505346967 time 739520993605 state 0x0 ref 1 00000100:00100000:2.0:1789760796.288680:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739521254021 time 739520993605 state 0x1 ref 1 00000100:00100000:2.0:1789760796.288708:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739521254021 time 739534326158 state 0x0 ref 1 00000100:00100000:2.0:1789760796.288720:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554900608:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760796.288781:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68243456, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x208c0 00010000:00010000:2.0:1789760796.288790:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71c9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.288805:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760796.288809:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.288814:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68243456->68247551] (req 68243456->68247551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.288828:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68243456->68247551] (req 68243456->68247551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.296285:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983970 is committed 00002000:00010000:2.0:1789760796.296440:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68243456 -> 68247552 00002000:00010000:2.0:1789760796.296446:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133312 -> 133320 00010000:00010000:2.0:1789760796.296459:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71c9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68243456->68247551] (req 68243456->68247551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.296472:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71c9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68243456->68247551] (req 68243456->68247551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.296480:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68243456->68247551] (req 68243456->68247551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.296487:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760796.296492:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71c9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68243456->68247551] (req 68243456->68247551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.296506:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71c9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68243456->68247551] (req 68243456->68247551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.296522:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71c9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68243456->68247551] (req 68243456->68247551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.296527:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71c9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68243456->68247551] (req 68243456->68247551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.296702:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554900608:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7977us (8335us total) trans 4294983970 rc 0/0 00000100:00100000:2.0:1789760796.296713:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16672 00000100:00100000:2.0:1789760796.296722:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739521254021 time 739534326158 state 0x2 ref 0 00000100:00100000:2.0:1789760796.308476:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760796.308651:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554900736 00000100:00100000:1.0:1789760796.308661:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554900736 00000100:00100000:1.0:1789760796.308719:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739521254021 time 739534326158 state 0x0 ref 1 00000100:00100000:1.0:1789760796.308735:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739534586574 time 739534326158 state 0x1 ref 1 00000100:00100000:1.0:1789760796.308770:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739534586574 time 739554388154 state 0x0 ref 1 00000100:00100000:1.0:1789760796.308783:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554900736:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760796.308838:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68247552, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x208c8 00010000:00010000:1.0:1789760796.308848:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c71d0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.308863:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760796.308868:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.308875:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c71d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68247552->68251647] (req 68247552->68251647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.308888:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c71d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68247552->68251647] (req 68247552->68251647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.314434:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983971 is committed 00002000:00010000:1.0:1789760796.314550:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68247552 -> 68251648 00002000:00010000:1.0:1789760796.314555:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133320 -> 133328 00010000:00010000:1.0:1789760796.314568:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c71d0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68247552->68251647] (req 68247552->68251647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.314590:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c71d0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68247552->68251647] (req 68247552->68251647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.314599:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c71d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68247552->68251647] (req 68247552->68251647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.314607:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760796.314619:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c71d0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68247552->68251647] (req 68247552->68251647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.314628:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c71d0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68247552->68251647] (req 68247552->68251647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.314646:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c71d0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68247552->68251647] (req 68247552->68251647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.314658:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2812cb1/0x1f99687d854c71d0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68247552->68251647] (req 68247552->68251647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.314753:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554900736:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5968us (6284us total) trans 4294983971 rc 0/0 00000100:00100000:1.0:1789760796.314764:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16673 00000100:00100000:1.0:1789760796.314773:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739534586574 time 739554388154 state 0x2 ref 0 00000100:00100000:1.0:1789760796.318613:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760796.318766:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554900864 00000100:00100000:2.0:1789760796.318776:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554900864 00000100:00100000:2.0:1789760796.318830:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739534586574 time 739554388154 state 0x0 ref 1 00000100:00100000:2.0:1789760796.318845:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739554648570 time 739554388154 state 0x1 ref 1 00000100:00100000:2.0:1789760796.318877:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739554648570 time 739564495495 state 0x0 ref 1 00000100:00100000:2.0:1789760796.318889:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554900864:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760796.318936:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68251648, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x208d0 00010000:00010000:2.0:1789760796.318946:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71d7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.319466:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760796.319471:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.319479:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68251648->68255743] (req 68251648->68255743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.319494:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68251648->68255743] (req 68251648->68255743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.326767:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983972 is committed 00002000:00010000:2.0:1789760796.327287:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68251648 -> 68255744 00002000:00010000:2.0:1789760796.327293:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133328 -> 133336 00010000:00010000:2.0:1789760796.327305:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71d7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68251648->68255743] (req 68251648->68255743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.327317:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71d7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68251648->68255743] (req 68251648->68255743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.327327:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68251648->68255743] (req 68251648->68255743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.327335:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760796.327340:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71d7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68251648->68255743] (req 68251648->68255743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.327349:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71d7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68251648->68255743] (req 68251648->68255743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.327370:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71d7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68251648->68255743] (req 68251648->68255743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.327421:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71d7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68251648->68255743] (req 68251648->68255743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.327763:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554900864:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8870us (9153us total) trans 4294983972 rc 0/0 00000100:00100000:2.0:1789760796.327773:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16674 00000100:00100000:2.0:1789760796.327783:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739554648570 time 739564495495 state 0x2 ref 0 00000100:00100000:3.0:1789760796.331614:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760796.331746:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554900992 00000100:00100000:2.0:1789760796.331756:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554900992 00000100:00100000:2.0:1789760796.331818:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739554648570 time 739564495495 state 0x0 ref 1 00000100:00100000:2.0:1789760796.331831:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739564755911 time 739564495495 state 0x1 ref 1 00000100:00100000:2.0:1789760796.331861:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739564755911 time 739577480138 state 0x0 ref 1 00000100:00100000:2.0:1789760796.331872:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554900992:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760796.331919:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68255744, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x208d8 00010000:00010000:2.0:1789760796.331927:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c71de lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.331943:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760796.331947:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.331954:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c71de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68255744->68259839] (req 68255744->68259839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.331969:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c71de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68255744->68259839] (req 68255744->68259839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.338223:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983973 is committed 00002000:00010000:2.0:1789760796.338413:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68255744 -> 68259840 00002000:00010000:2.0:1789760796.338418:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133336 -> 133344 00010000:00010000:2.0:1789760796.338431:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c71de lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68255744->68259839] (req 68255744->68259839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.338443:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c71de lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68255744->68259839] (req 68255744->68259839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.338451:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c71de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68255744->68259839] (req 68255744->68259839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.338459:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760796.338463:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c71de lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68255744->68259839] (req 68255744->68259839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.338471:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c71de lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68255744->68259839] (req 68255744->68259839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.338491:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c71de lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68255744->68259839] (req 68255744->68259839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.338504:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c71de lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68255744->68259839] (req 68255744->68259839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.338704:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554900992:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6827us (7092us total) trans 4294983973 rc 0/0 00000100:00100000:2.0:1789760796.338714:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16675 00000100:00100000:2.0:1789760796.338723:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739564755911 time 739577480138 state 0x2 ref 0 00000100:00100000:1.0:1789760796.344829:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760796.345273:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554901120 00000100:00100000:3.0:1789760796.345297:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554901120 00000100:00100000:3.0:1789760796.345352:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739564755911 time 739577480138 state 0x0 ref 1 00000100:00100000:3.0:1789760796.345368:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739577740554 time 739577480138 state 0x1 ref 1 00000100:00100000:3.0:1789760796.345435:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739577740554 time 739591052804 state 0x0 ref 1 00000100:00100000:3.0:1789760796.345449:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb031b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554901120:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760796.345494:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68259840, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x208e0 00010000:00010000:3.0:1789760796.345503:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c71e5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.345524:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760796.345529:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.345536:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c71e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68259840->68263935] (req 68259840->68263935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.345551:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c71e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68259840->68263935] (req 68259840->68263935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.354422:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983974 is committed 00002000:00010000:2.0:1789760796.354604:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68259840 -> 68263936 00002000:00010000:2.0:1789760796.354610:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133344 -> 133352 00010000:00010000:2.0:1789760796.354623:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c71e5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68259840->68263935] (req 68259840->68263935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.354636:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c71e5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68259840->68263935] (req 68259840->68263935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.354646:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c71e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68259840->68263935] (req 68259840->68263935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.354653:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760796.354658:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c71e5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68259840->68263935] (req 68259840->68263935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.354779:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c71e5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68259840->68263935] (req 68259840->68263935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.354805:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c71e5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68259840->68263935] (req 68259840->68263935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.354813:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a322c172/0x1f99687d854c71e5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68259840->68263935] (req 68259840->68263935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.354907:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb031b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554901120:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9456us (10083us total) trans 4294983974 rc 0/0 00000100:00100000:2.0:1789760796.354917:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16676 00000100:00100000:2.0:1789760796.354926:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739577740554 time 739591052804 state 0x2 ref 0 00000100:00100000:1.0:1789760796.364488:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760796.364609:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554901248 00000100:00100000:0.0:1789760796.364629:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554901248 00000100:00100000:0.0:1789760796.364692:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739577740554 time 739591052804 state 0x0 ref 1 00000100:00100000:0.0:1789760796.364707:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739591313220 time 739591052804 state 0x1 ref 1 00000100:00100000:0.0:1789760796.364739:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739591313220 time 739610357957 state 0x0 ref 1 00000100:00100000:0.0:1789760796.364752:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554901248:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760796.364812:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68263936, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x208e8 00010000:00010000:0.0:1789760796.364823:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c71ec lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.364839:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760796.364844:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.364851:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c71ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68263936->68268031] (req 68263936->68268031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.364865:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c71ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68263936->68268031] (req 68263936->68268031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.371129:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983975 is committed 00002000:00010000:0.0:1789760796.374946:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68263936 -> 68268032 00002000:00010000:0.0:1789760796.375015:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133352 -> 133360 00010000:00010000:0.0:1789760796.375029:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c71ec lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68263936->68268031] (req 68263936->68268031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.375042:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c71ec lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68263936->68268031] (req 68263936->68268031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.375051:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c71ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68263936->68268031] (req 68263936->68268031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.375058:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760796.375064:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c71ec lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68263936->68268031] (req 68263936->68268031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.375073:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c71ec lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68263936->68268031] (req 68263936->68268031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.375094:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c71ec lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68263936->68268031] (req 68263936->68268031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.375102:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c71ec lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68263936->68268031] (req 68263936->68268031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.375497:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554901248:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10741us (11013us total) trans 4294983975 rc 0/0 00000100:00100000:0.0:1789760796.375509:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16677 00000100:00100000:0.0:1789760796.375519:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739591313220 time 739610357957 state 0x2 ref 0 00000100:00100000:2.0:1789760796.379292:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760796.379366:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554901376 00000100:00100000:1.0:1789760796.379375:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554901376 00000100:00100000:1.0:1789760796.379464:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739591313220 time 739610357957 state 0x0 ref 1 00000100:00100000:1.0:1789760796.379486:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739610618373 time 739610357957 state 0x1 ref 1 00000100:00100000:1.0:1789760796.379516:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739610618373 time 739625134121 state 0x0 ref 1 00000100:00100000:1.0:1789760796.379529:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554901376:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760796.379589:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68268032, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x208f0 00010000:00010000:1.0:1789760796.379598:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c71f3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.379614:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760796.379619:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.379625:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c71f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68268032->68272127] (req 68268032->68272127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.379638:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c71f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68268032->68272127] (req 68268032->68272127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.384560:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983976 is committed 00002000:00010000:1.0:1789760796.384702:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68268032 -> 68272128 00002000:00010000:1.0:1789760796.384708:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133360 -> 133368 00010000:00010000:1.0:1789760796.384720:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c71f3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68268032->68272127] (req 68268032->68272127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.384733:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c71f3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68268032->68272127] (req 68268032->68272127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.384741:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c71f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68268032->68272127] (req 68268032->68272127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.384748:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655de00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760796.384753:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c71f3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68268032->68272127] (req 68268032->68272127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.384760:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c71f3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68268032->68272127] (req 68268032->68272127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.384780:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c71f3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68268032->68272127] (req 68268032->68272127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.384917:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000519d19cf/0x1f99687d854c71f3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68268032->68272127] (req 68268032->68272127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.385075:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554901376:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5544us (5789us total) trans 4294983976 rc 0/0 00000100:00100000:1.0:1789760796.385086:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16678 00000100:00100000:1.0:1789760796.385097:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739610618373 time 739625134121 state 0x2 ref 0 00000100:00100000:1.0:1789760796.390610:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760796.390880:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554901504 00000100:00100000:2.0:1789760796.390892:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554901504 00000100:00100000:2.0:1789760796.390944:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739610618373 time 739625134121 state 0x0 ref 1 00000100:00100000:2.0:1789760796.391026:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739625394537 time 739625134121 state 0x1 ref 1 00000100:00100000:2.0:1789760796.391099:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739625394537 time 739636716576 state 0x0 ref 1 00000100:00100000:2.0:1789760796.391114:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554901504:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760796.391178:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68272128, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x208f8 00010000:00010000:2.0:1789760796.391187:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71fa lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.391209:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760796.391214:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.391222:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68272128->68276223] (req 68272128->68276223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.391235:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68272128->68276223] (req 68272128->68276223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.396439:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983977 is committed 00002000:00010000:2.0:1789760796.396870:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68272128 -> 68276224 00002000:00010000:2.0:1789760796.396876:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133368 -> 133376 00010000:00010000:2.0:1789760796.396889:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71fa lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68272128->68276223] (req 68272128->68276223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.396902:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71fa lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68272128->68276223] (req 68272128->68276223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.396911:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68272128->68276223] (req 68272128->68276223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.396918:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760796.396923:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71fa lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68272128->68276223] (req 68272128->68276223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.396932:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71fa lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68272128->68276223] (req 68272128->68276223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.397015:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71fa lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68272128->68276223] (req 68272128->68276223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.397025:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c71fa lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68272128->68276223] (req 68272128->68276223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.397163:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554901504:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6047us (6557us total) trans 4294983977 rc 0/0 00000100:00100000:2.0:1789760796.397174:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16679 00000100:00100000:2.0:1789760796.397184:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739625394537 time 739636716576 state 0x2 ref 0 00000100:00100000:2.0:1789760796.401553:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760796.401926:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554901632 00000100:00100000:3.0:1789760796.401936:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554901632 00000100:00100000:3.0:1789760796.401997:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739625394537 time 739636716576 state 0x0 ref 1 00000100:00100000:3.0:1789760796.402014:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739636976992 time 739636716576 state 0x1 ref 1 00000100:00100000:3.0:1789760796.402046:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739636976992 time 739647665166 state 0x0 ref 1 00000100:00100000:3.0:1789760796.402059:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554901632:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760796.402105:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68276224, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20900 00010000:00010000:3.0:1789760796.402114:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7201 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.402127:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760796.402132:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.402137:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7201 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68276224->68280319] (req 68276224->68280319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.402150:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7201 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68276224->68280319] (req 68276224->68280319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.407106:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983978 is committed 00002000:00010000:3.0:1789760796.407210:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68276224 -> 68280320 00002000:00010000:3.0:1789760796.407215:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133376 -> 133384 00010000:00010000:3.0:1789760796.407228:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7201 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68276224->68280319] (req 68276224->68280319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.407241:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7201 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68276224->68280319] (req 68276224->68280319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.407249:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7201 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68276224->68280319] (req 68276224->68280319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.407261:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760796.407266:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7201 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68276224->68280319] (req 68276224->68280319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.407280:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7201 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68276224->68280319] (req 68276224->68280319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.407301:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7201 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68276224->68280319] (req 68276224->68280319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.407308:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7201 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68276224->68280319] (req 68276224->68280319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.407477:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554901632:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5416us (5934us total) trans 4294983978 rc 0/0 00000100:00100000:3.0:1789760796.407489:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16680 00000100:00100000:3.0:1789760796.407499:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739636976992 time 739647665166 state 0x2 ref 0 00000100:00100000:0.0:1789760796.423168:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760796.423264:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554901760 00000100:00100000:1.0:1789760796.423273:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554901760 00000100:00100000:1.0:1789760796.423325:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739636976992 time 739647665166 state 0x0 ref 1 00000100:00100000:1.0:1789760796.423341:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739647925582 time 739647665166 state 0x1 ref 1 00000100:00100000:1.0:1789760796.423419:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739647925582 time 739669035724 state 0x0 ref 1 00000100:00100000:1.0:1789760796.423432:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554901760:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760796.423485:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68280320, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20908 00010000:00010000:1.0:1789760796.423495:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c7208 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.423511:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760796.423516:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.423521:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c7208 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68280320->68284415] (req 68280320->68284415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.423537:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c7208 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68280320->68284415] (req 68280320->68284415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.430298:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983979 is committed 00002000:00010000:0.0:1789760796.430609:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68280320 -> 68284416 00002000:00010000:0.0:1789760796.430615:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133384 -> 133392 00010000:00010000:0.0:1789760796.430629:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c7208 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68280320->68284415] (req 68280320->68284415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.430654:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c7208 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68280320->68284415] (req 68280320->68284415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.430664:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c7208 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68280320->68284415] (req 68280320->68284415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.431589:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760796.431597:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c7208 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68280320->68284415] (req 68280320->68284415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.431610:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c7208 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68280320->68284415] (req 68280320->68284415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.431631:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c7208 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68280320->68284415] (req 68280320->68284415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.431639:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009a27eca7/0x1f99687d854c7208 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68280320->68284415] (req 68280320->68284415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.431956:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554901760:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8520us (8792us total) trans 4294983979 rc 0/0 00000100:00100000:0.0:1789760796.431968:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16681 00000100:00100000:0.0:1789760796.431977:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739647925582 time 739669035724 state 0x2 ref 0 00000100:00100000:1.0:1789760796.436893:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760796.437256:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554901888 00000100:00100000:2.0:1789760796.437265:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554901888 00000100:00100000:2.0:1789760796.437320:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739647925582 time 739669035724 state 0x0 ref 1 00000100:00100000:2.0:1789760796.437337:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739669296140 time 739669035724 state 0x1 ref 1 00000100:00100000:2.0:1789760796.437548:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739669296140 time 739683165037 state 0x0 ref 1 00000100:00100000:2.0:1789760796.437563:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554901888:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760796.437620:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68284416, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20910 00010000:00010000:2.0:1789760796.437629:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c720f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.437660:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760796.437664:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.437670:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c720f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68284416->68288511] (req 68284416->68288511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.437694:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c720f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68284416->68288511] (req 68284416->68288511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.448356:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983980 is committed 00002000:00010000:2.0:1789760796.449036:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68284416 -> 68288512 00002000:00010000:2.0:1789760796.449042:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133392 -> 133400 00010000:00010000:2.0:1789760796.449054:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c720f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68284416->68288511] (req 68284416->68288511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.449068:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c720f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68284416->68288511] (req 68284416->68288511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.449078:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c720f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68284416->68288511] (req 68284416->68288511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.449086:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760796.449091:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c720f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68284416->68288511] (req 68284416->68288511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.449102:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c720f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68284416->68288511] (req 68284416->68288511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.449126:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c720f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68284416->68288511] (req 68284416->68288511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.449135:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c720f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68284416->68288511] (req 68284416->68288511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.449232:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554901888:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11666us (12341us total) trans 4294983980 rc 0/0 00000100:00100000:2.0:1789760796.449244:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16682 00000100:00100000:2.0:1789760796.449264:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739669296140 time 739683165037 state 0x2 ref 0 00000100:00100000:2.0:1789760796.454090:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760796.454300:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554902016 00000100:00100000:1.0:1789760796.454310:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554902016 00000100:00100000:1.0:1789760796.455224:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739669296140 time 739683165037 state 0x0 ref 1 00000100:00100000:1.0:1789760796.455240:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739683425453 time 739683165037 state 0x1 ref 1 00000100:00100000:1.0:1789760796.455274:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739683425453 time 739700892537 state 0x0 ref 1 00000100:00100000:1.0:1789760796.455289:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554902016:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760796.455333:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68288512, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20918 00010000:00010000:1.0:1789760796.455342:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c7216 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.455356:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760796.455359:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.455365:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c7216 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68288512->68292607] (req 68288512->68292607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.455390:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c7216 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68288512->68292607] (req 68288512->68292607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.466435:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983981 is committed 00002000:00010000:1.0:1789760796.467221:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68288512 -> 68292608 00002000:00010000:1.0:1789760796.467228:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133400 -> 133408 00010000:00010000:1.0:1789760796.467241:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c7216 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68288512->68292607] (req 68288512->68292607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.467698:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c7216 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68288512->68292607] (req 68288512->68292607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.467709:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c7216 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68288512->68292607] (req 68288512->68292607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.467717:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760796.467722:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c7216 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68288512->68292607] (req 68288512->68292607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.467730:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c7216 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68288512->68292607] (req 68288512->68292607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.467751:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c7216 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68288512->68292607] (req 68288512->68292607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.467759:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c7216 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68288512->68292607] (req 68288512->68292607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.469296:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554902016:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14001us (15205us total) trans 4294983981 rc 0/0 00000100:00100000:1.0:1789760796.469311:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16683 00000100:00100000:1.0:1789760796.469321:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739683425453 time 739700892537 state 0x2 ref 0 00000100:00100000:0.0:1789760796.473499:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760796.473957:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554902144 00000100:00100000:1.0:1789760796.473968:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554902144 00000100:00100000:1.0:1789760796.474024:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739683425453 time 739700892537 state 0x0 ref 1 00000100:00100000:1.0:1789760796.474041:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739701152953 time 739700892537 state 0x1 ref 1 00000100:00100000:1.0:1789760796.474073:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739701152953 time 739719691468 state 0x0 ref 1 00000100:00100000:1.0:1789760796.474362:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e54a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554902144:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760796.474487:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68292608, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20920 00010000:00010000:0.0:1789760796.474496:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c721d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.474513:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760796.474517:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.474524:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c721d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68292608->68296703] (req 68292608->68296703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.474537:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c721d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68292608->68296703] (req 68292608->68296703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.480842:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983982 is committed 00002000:00010000:0.0:1789760796.481063:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68292608 -> 68296704 00002000:00010000:0.0:1789760796.481068:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133408 -> 133416 00010000:00010000:0.0:1789760796.481081:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c721d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68292608->68296703] (req 68292608->68296703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.481176:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c721d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68292608->68296703] (req 68292608->68296703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.481186:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c721d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68292608->68296703] (req 68292608->68296703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.481194:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760796.481199:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c721d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68292608->68296703] (req 68292608->68296703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.481209:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c721d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68292608->68296703] (req 68292608->68296703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.481227:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c721d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68292608->68296703] (req 68292608->68296703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.481593:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c721d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68292608->68296703] (req 68292608->68296703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.481736:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554902144:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7648us (8241us total) trans 4294983982 rc 0/0 00000100:00100000:0.0:1789760796.481746:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16684 00000100:00100000:0.0:1789760796.481755:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739701152953 time 739719691468 state 0x2 ref 0 00000100:00100000:3.0:1789760796.494265:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760796.494399:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554902272 00000100:00100000:0.0:1789760796.494408:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554902272 00000100:00100000:0.0:1789760796.494466:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739701152953 time 739719691468 state 0x0 ref 1 00000100:00100000:0.0:1789760796.494480:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739719951884 time 739719691468 state 0x1 ref 1 00000100:00100000:0.0:1789760796.494508:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739719951884 time 739740126439 state 0x0 ref 1 00000100:00100000:0.0:1789760796.494519:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554902272:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760796.494561:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68296704, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20928 00010000:00010000:0.0:1789760796.494570:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7224 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.494594:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760796.494598:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.494604:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7224 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68296704->68300799] (req 68296704->68300799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.494623:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7224 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68296704->68300799] (req 68296704->68300799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.503596:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983983 is committed 00002000:00010000:0.0:1789760796.504194:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68296704 -> 68300800 00002000:00010000:0.0:1789760796.504200:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133416 -> 133424 00010000:00010000:0.0:1789760796.504215:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7224 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68296704->68300799] (req 68296704->68300799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.504231:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7224 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68296704->68300799] (req 68296704->68300799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.504239:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7224 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68296704->68300799] (req 68296704->68300799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.504247:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760796.504251:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7224 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68296704->68300799] (req 68296704->68300799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.504261:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7224 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68296704->68300799] (req 68296704->68300799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.504283:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7224 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68296704->68300799] (req 68296704->68300799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.504292:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7224 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68296704->68300799] (req 68296704->68300799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.505214:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554902272:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10687us (10950us total) trans 4294983983 rc 0/0 00000100:00100000:0.0:1789760796.505226:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16685 00000100:00100000:0.0:1789760796.505237:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739719951884 time 739740126439 state 0x2 ref 0 00000100:00100000:2.0:1789760796.515816:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760796.515915:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554902400 00000100:00100000:1.0:1789760796.515926:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554902400 00000100:00100000:1.0:1789760796.515981:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739719951884 time 739740126439 state 0x0 ref 1 00000100:00100000:1.0:1789760796.515998:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739740386855 time 739740126439 state 0x1 ref 1 00000100:00100000:1.0:1789760796.516029:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739740386855 time 739761647770 state 0x0 ref 1 00000100:00100000:1.0:1789760796.516041:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554902400:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760796.516085:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68300800, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20930 00010000:00010000:1.0:1789760796.516095:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c722b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.516112:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760796.516116:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.516122:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c722b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68300800->68304895] (req 68300800->68304895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.516135:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c722b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68300800->68304895] (req 68300800->68304895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.524999:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983984 is committed 00002000:00010000:1.0:1789760796.525173:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68300800 -> 68304896 00002000:00010000:1.0:1789760796.525179:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133424 -> 133432 00010000:00010000:1.0:1789760796.525196:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c722b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68300800->68304895] (req 68300800->68304895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.525210:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c722b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68300800->68304895] (req 68300800->68304895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.525220:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c722b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68300800->68304895] (req 68300800->68304895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.525229:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760796.525233:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c722b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68300800->68304895] (req 68300800->68304895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.525243:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c722b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68300800->68304895] (req 68300800->68304895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.525269:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c722b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68300800->68304895] (req 68300800->68304895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.525277:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c722b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68300800->68304895] (req 68300800->68304895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.525415:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554902400:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9336us (9571us total) trans 4294983984 rc 0/0 00000100:00100000:1.0:1789760796.525425:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16686 00000100:00100000:1.0:1789760796.525435:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739740386855 time 739761647770 state 0x2 ref 0 00000100:00100000:2.0:1789760796.528935:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760796.529862:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554902528 00000100:00100000:1.0:1789760796.529875:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554902528 00000100:00100000:1.0:1789760796.529933:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739740386855 time 739761647770 state 0x0 ref 1 00000100:00100000:1.0:1789760796.530012:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739761908186 time 739761647770 state 0x1 ref 1 00000100:00100000:1.0:1789760796.530102:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739761908186 time 739775719071 state 0x0 ref 1 00000100:00100000:1.0:1789760796.530115:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554902528:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760796.530160:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68304896, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20938 00010000:00010000:1.0:1789760796.530168:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c7232 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.530184:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760796.530189:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.530196:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c7232 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68304896->68308991] (req 68304896->68308991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.530210:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c7232 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68304896->68308991] (req 68304896->68308991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.538943:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983985 is committed 00002000:00010000:1.0:1789760796.539084:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68304896 -> 68308992 00002000:00010000:1.0:1789760796.539089:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133432 -> 133440 00010000:00010000:1.0:1789760796.539102:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c7232 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68304896->68308991] (req 68304896->68308991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.539114:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c7232 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68304896->68308991] (req 68304896->68308991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.539122:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c7232 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68304896->68308991] (req 68304896->68308991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.539129:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655c200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760796.539134:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c7232 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68304896->68308991] (req 68304896->68308991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.539144:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c7232 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68304896->68308991] (req 68304896->68308991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.539162:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c7232 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68304896->68308991] (req 68304896->68308991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.539174:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000000d67a9b/0x1f99687d854c7232 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68304896->68308991] (req 68304896->68308991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.539270:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554902528:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9153us (10338us total) trans 4294983985 rc 0/0 00000100:00100000:1.0:1789760796.539281:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16687 00000100:00100000:1.0:1789760796.539290:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739761908186 time 739775719071 state 0x2 ref 0 00000100:00100000:1.0:1789760796.545821:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760796.545906:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554902656 00000100:00100000:0.0:1789760796.545915:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554902656 00000100:00100000:0.0:1789760796.546025:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739761908186 time 739775719071 state 0x0 ref 1 00000100:00100000:0.0:1789760796.546039:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739775979487 time 739775719071 state 0x1 ref 1 00000100:00100000:0.0:1789760796.546207:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739775979487 time 739791823761 state 0x0 ref 1 00000100:00100000:0.0:1789760796.546221:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554902656:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760796.546266:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68308992, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20940 00010000:00010000:0.0:1789760796.546275:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7239 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.546290:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760796.546294:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.546301:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7239 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68308992->68313087] (req 68308992->68313087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.546314:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7239 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68308992->68313087] (req 68308992->68313087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.555809:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983986 is committed 00002000:00010000:0.0:1789760796.555914:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68308992 -> 68313088 00002000:00010000:0.0:1789760796.555918:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133440 -> 133448 00010000:00010000:0.0:1789760796.555928:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7239 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68308992->68313087] (req 68308992->68313087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.555937:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7239 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68308992->68313087] (req 68308992->68313087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.555943:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7239 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68308992->68313087] (req 68308992->68313087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.556967:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760796.556973:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7239 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68308992->68313087] (req 68308992->68313087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.556985:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7239 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68308992->68313087] (req 68308992->68313087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.557007:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7239 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68308992->68313087] (req 68308992->68313087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.557016:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7239 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68308992->68313087] (req 68308992->68313087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.557124:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554902656:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10901us (11307us total) trans 4294983986 rc 0/0 00000100:00100000:0.0:1789760796.557136:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16688 00000100:00100000:0.0:1789760796.557145:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739775979487 time 739791823761 state 0x2 ref 0 00000100:00100000:3.0:1789760796.563227:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760796.565045:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554902784 00000100:00100000:0.0:1789760796.565060:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554902784 00000100:00100000:0.0:1789760796.565117:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739775979487 time 739791823761 state 0x0 ref 1 00000100:00100000:0.0:1789760796.565134:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739792084177 time 739791823761 state 0x1 ref 1 00000100:00100000:0.0:1789760796.565612:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739792084177 time 739811227566 state 0x0 ref 1 00000100:00100000:0.0:1789760796.565628:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554902784:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760796.565676:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68313088, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20948 00010000:00010000:0.0:1789760796.565685:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7240 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.565701:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760796.565705:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.565712:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7240 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68313088->68317183] (req 68313088->68317183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.565727:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7240 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68313088->68317183] (req 68313088->68317183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.571725:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983987 is committed 00002000:00010000:0.0:1789760796.571878:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68313088 -> 68317184 00002000:00010000:0.0:1789760796.571883:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133448 -> 133456 00010000:00010000:0.0:1789760796.571896:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7240 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68313088->68317183] (req 68313088->68317183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.571910:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7240 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68313088->68317183] (req 68313088->68317183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.571919:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7240 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68313088->68317183] (req 68313088->68317183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.571927:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760796.571933:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7240 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68313088->68317183] (req 68313088->68317183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.571943:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7240 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68313088->68317183] (req 68313088->68317183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.571977:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7240 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68313088->68317183] (req 68313088->68317183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.571985:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7240 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68313088->68317183] (req 68313088->68317183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.572074:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554902784:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6446us (8851us total) trans 4294983987 rc 0/0 00000100:00100000:0.0:1789760796.572085:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16689 00000100:00100000:0.0:1789760796.572094:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739792084177 time 739811227566 state 0x2 ref 0 00000100:00100000:2.0:1789760796.580496:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760796.580562:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554902912 00000100:00100000:1.0:1789760796.580582:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554902912 00000100:00100000:1.0:1789760796.580647:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739792084177 time 739811227566 state 0x0 ref 1 00000100:00100000:1.0:1789760796.580670:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739811487982 time 739811227566 state 0x1 ref 1 00000100:00100000:1.0:1789760796.580813:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739811487982 time 739826430750 state 0x0 ref 1 00000100:00100000:1.0:1789760796.580826:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054cb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554902912:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760796.580873:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68317184, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20950 00010000:00010000:1.0:1789760796.580884:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c7247 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.580899:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760796.580904:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.580910:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c7247 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68317184->68321279] (req 68317184->68321279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.580923:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c7247 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68317184->68321279] (req 68317184->68321279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.587301:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983988 is committed 00002000:00010000:1.0:1789760796.587529:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68317184 -> 68321280 00002000:00010000:1.0:1789760796.587535:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133456 -> 133464 00010000:00010000:1.0:1789760796.587552:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c7247 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68317184->68321279] (req 68317184->68321279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.587564:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c7247 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68317184->68321279] (req 68317184->68321279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.587583:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c7247 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68317184->68321279] (req 68317184->68321279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.587590:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760796.587595:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c7247 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68317184->68321279] (req 68317184->68321279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.587604:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c7247 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68317184->68321279] (req 68317184->68321279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.587623:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c7247 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68317184->68321279] (req 68317184->68321279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.587630:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000061c81877/0x1f99687d854c7247 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68317184->68321279] (req 68317184->68321279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.587883:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054cb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554902912:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7054us (7392us total) trans 4294983988 rc 0/0 00000100:00100000:1.0:1789760796.587893:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16690 00000100:00100000:1.0:1789760796.587903:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739811487982 time 739826430750 state 0x2 ref 0 00000100:00100000:1.0:1789760796.593549:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760796.593755:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554903040 00000100:00100000:2.0:1789760796.593765:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554903040 00000100:00100000:2.0:1789760796.593816:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739811487982 time 739826430750 state 0x0 ref 1 00000100:00100000:2.0:1789760796.593842:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739826691166 time 739826430750 state 0x1 ref 1 00000100:00100000:2.0:1789760796.593863:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739826691166 time 739839481996 state 0x0 ref 1 00000100:00100000:2.0:1789760796.593876:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554903040:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760796.593920:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68321280, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20958 00010000:00010000:2.0:1789760796.593930:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c724e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.593945:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760796.593960:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.593966:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c724e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68321280->68325375] (req 68321280->68325375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.593979:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c724e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68321280->68325375] (req 68321280->68325375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.600217:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983989 is committed 00002000:00010000:2.0:1789760796.600332:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68321280 -> 68325376 00002000:00010000:2.0:1789760796.600337:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133464 -> 133472 00010000:00010000:2.0:1789760796.600349:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c724e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68321280->68325375] (req 68321280->68325375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.600371:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c724e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68321280->68325375] (req 68321280->68325375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.600407:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c724e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68321280->68325375] (req 68321280->68325375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.600415:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760796.600420:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c724e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68321280->68325375] (req 68321280->68325375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.600429:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c724e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68321280->68325375] (req 68321280->68325375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.600450:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c724e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68321280->68325375] (req 68321280->68325375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.600458:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c724e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68321280->68325375] (req 68321280->68325375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.600541:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554903040:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6661us (6996us total) trans 4294983989 rc 0/0 00000100:00100000:2.0:1789760796.600551:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16691 00000100:00100000:2.0:1789760796.601045:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739826691166 time 739839481996 state 0x2 ref 0 00000100:00100000:1.0:1789760796.604878:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760796.604969:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554903168 00000100:00100000:0.0:1789760796.604980:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554903168 00000100:00100000:0.0:1789760796.605031:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739826691166 time 739839481996 state 0x0 ref 1 00000100:00100000:0.0:1789760796.605046:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739839742412 time 739839481996 state 0x1 ref 1 00000100:00100000:0.0:1789760796.605075:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739839742412 time 739850694059 state 0x0 ref 1 00000100:00100000:0.0:1789760796.605088:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb044b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554903168:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760796.605132:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68325376, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20960 00010000:00010000:0.0:1789760796.605141:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7255 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.605156:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760796.605160:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.605167:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7255 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68325376->68329471] (req 68325376->68329471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.605181:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7255 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68325376->68329471] (req 68325376->68329471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.611324:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983990 is committed 00002000:00010000:3.0:1789760796.611489:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68325376 -> 68329472 00002000:00010000:3.0:1789760796.611494:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133472 -> 133480 00010000:00010000:3.0:1789760796.611507:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7255 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68325376->68329471] (req 68325376->68329471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.611521:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7255 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68325376->68329471] (req 68325376->68329471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.611531:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7255 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68325376->68329471] (req 68325376->68329471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.611540:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760796.611547:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7255 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68325376->68329471] (req 68325376->68329471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.611557:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7255 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68325376->68329471] (req 68325376->68329471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.611588:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7255 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68325376->68329471] (req 68325376->68329471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.611595:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7255 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68325376->68329471] (req 68325376->68329471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.612033:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb044b3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554903168:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6941us (7157us total) trans 4294983990 rc 0/0 00000100:00100000:3.0:1789760796.612044:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16692 00000100:00100000:3.0:1789760796.612063:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739839742412 time 739850694059 state 0x2 ref 0 00000100:00100000:0.0:1789760796.619684:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760796.619860:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554903296 00000100:00100000:1.0:1789760796.619869:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554903296 00000100:00100000:1.0:1789760796.619922:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739839742412 time 739850694059 state 0x0 ref 1 00000100:00100000:1.0:1789760796.619936:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739850954475 time 739850694059 state 0x1 ref 1 00000100:00100000:1.0:1789760796.619975:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739850954475 time 739865593349 state 0x0 ref 1 00000100:00100000:1.0:1789760796.619987:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554903296:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760796.620035:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68329472, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20968 00010000:00010000:1.0:1789760796.620044:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c725c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.620060:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760796.620065:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.620071:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c725c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68329472->68333567] (req 68329472->68333567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.620090:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c725c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68329472->68333567] (req 68329472->68333567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.627372:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983991 is committed 00002000:00010000:1.0:1789760796.627562:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68329472 -> 68333568 00002000:00010000:1.0:1789760796.627567:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133480 -> 133488 00010000:00010000:1.0:1789760796.627592:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c725c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68329472->68333567] (req 68329472->68333567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.627615:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c725c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68329472->68333567] (req 68329472->68333567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.627624:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c725c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68329472->68333567] (req 68329472->68333567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.627631:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760796.627637:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c725c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68329472->68333567] (req 68329472->68333567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.627645:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c725c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68329472->68333567] (req 68329472->68333567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.627670:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c725c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68329472->68333567] (req 68329472->68333567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.627677:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c725c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68329472->68333567] (req 68329472->68333567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.627776:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554903296:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7785us (8095us total) trans 4294983991 rc 0/0 00000100:00100000:1.0:1789760796.627786:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16693 00000100:00100000:1.0:1789760796.627795:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739850954475 time 739865593349 state 0x2 ref 0 00000100:00100000:1.0:1789760796.639057:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760796.639184:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554903424 00000100:00100000:2.0:1789760796.639194:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554903424 00000100:00100000:2.0:1789760796.639246:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739850954475 time 739865593349 state 0x0 ref 1 00000100:00100000:2.0:1789760796.639261:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739865853765 time 739865593349 state 0x1 ref 1 00000100:00100000:2.0:1789760796.639290:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739865853765 time 739884908580 state 0x0 ref 1 00000100:00100000:2.0:1789760796.639306:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554903424:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760796.639358:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68333568, mtime: 0x6aad951b, atime: 0x0, ctime: 0x6aad951b, blocks: 0x20970 00010000:00010000:2.0:1789760796.639368:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7263 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.639423:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760796.639430:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.639439:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7263 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68333568->68337663] (req 68333568->68337663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.639454:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7263 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68333568->68337663] (req 68333568->68337663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.644598:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983992 is committed 00002000:00010000:1.0:1789760796.645092:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68333568 -> 68337664 00002000:00010000:1.0:1789760796.645097:0:8441:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760795 -> 1789760796 00002000:00010000:1.0:1789760796.645100:0:8441:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760795 -> 1789760796 00002000:00010000:1.0:1789760796.645102:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133488 -> 133496 00010000:00010000:1.0:1789760796.645114:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7263 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68333568->68337663] (req 68333568->68337663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.645127:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7263 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68333568->68337663] (req 68333568->68337663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.645137:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7263 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68333568->68337663] (req 68333568->68337663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.645145:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760796.645150:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7263 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68333568->68337663] (req 68333568->68337663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.645158:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7263 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68333568->68337663] (req 68333568->68337663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.645182:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7263 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68333568->68337663] (req 68333568->68337663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.645192:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7263 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68333568->68337663] (req 68333568->68337663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.645527:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554903424:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6218us (6471us total) trans 4294983992 rc 0/0 00000100:00100000:1.0:1789760796.645542:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16694 00000100:00100000:1.0:1789760796.645552:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739865853765 time 739884908580 state 0x2 ref 0 00000100:00100000:2.0:1789760796.653093:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760796.655621:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554903552 00000100:00100000:1.0:1789760796.655636:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554903552 00000100:00100000:1.0:1789760796.655699:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739865853765 time 739884908580 state 0x0 ref 1 00000100:00100000:1.0:1789760796.655722:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739885168996 time 739884908580 state 0x1 ref 1 00000100:00100000:1.0:1789760796.655761:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739885168996 time 739901380108 state 0x0 ref 1 00000100:00100000:1.0:1789760796.655774:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb054c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554903552:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760796.655822:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68337664, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20978 00010000:00010000:1.0:1789760796.655830:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c726a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.655846:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:1.0:1789760796.655852:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.655858:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c726a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68337664->68341759] (req 68337664->68341759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.655870:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c726a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68337664->68341759] (req 68337664->68341759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.661252:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983993 is committed 00002000:00010000:1.0:1789760796.663787:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68337664 -> 68341760 00002000:00010000:1.0:1789760796.663796:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133496 -> 133504 00010000:00010000:1.0:1789760796.663811:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c726a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68337664->68341759] (req 68337664->68341759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.663822:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c726a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68337664->68341759] (req 68337664->68341759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.663831:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c726a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68337664->68341759] (req 68337664->68341759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.663837:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655ca00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760796.663842:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c726a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68337664->68341759] (req 68337664->68341759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.663850:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c726a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68337664->68341759] (req 68337664->68341759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.663868:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c726a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68337664->68341759] (req 68337664->68341759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.663880:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e0ec8b23/0x1f99687d854c726a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68337664->68341759] (req 68337664->68341759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.663979:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb054c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554903552:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8203us (10891us total) trans 4294983993 rc 0/0 00000100:00100000:1.0:1789760796.663990:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16695 00000100:00100000:1.0:1789760796.664000:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739885168996 time 739901380108 state 0x2 ref 0 00000100:00100000:0.0:1789760796.669890:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760796.670304:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554903680 00000100:00100000:3.0:1789760796.670315:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554903680 00000100:00100000:3.0:1789760796.670366:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739885168996 time 739901380108 state 0x0 ref 1 00000100:00100000:3.0:1789760796.670412:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739901640524 time 739901380108 state 0x1 ref 1 00000100:00100000:3.0:1789760796.670499:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739901640524 time 739916116278 state 0x0 ref 1 00000100:00100000:3.0:1789760796.670514:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e56680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554903680:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760796.670559:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68341760, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20980 00010000:00010000:3.0:1789760796.670568:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7271 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.670593:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:3.0:1789760796.670598:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.670605:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7271 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68341760->68345855] (req 68341760->68345855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.670619:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7271 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68341760->68345855] (req 68341760->68345855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.688163:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983994 is committed 00002000:00010000:2.0:1789760796.689596:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68341760 -> 68345856 00002000:00010000:2.0:1789760796.689604:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133504 -> 133512 00010000:00010000:2.0:1789760796.689617:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7271 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68341760->68345855] (req 68341760->68345855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.689630:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7271 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68341760->68345855] (req 68341760->68345855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.689639:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7271 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68341760->68345855] (req 68341760->68345855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.689646:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760796.689651:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7271 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68341760->68345855] (req 68341760->68345855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.689661:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7271 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68341760->68345855] (req 68341760->68345855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.689680:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7271 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68341760->68345855] (req 68341760->68345855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.689697:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000109178a6/0x1f99687d854c7271 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68341760->68345855] (req 68341760->68345855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.689804:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e56680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554903680:12345-192.168.203.22@tcp:4:dd.0 Request processed in 19286us (19918us total) trans 4294983994 rc 0/0 00000100:00100000:2.0:1789760796.689815:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16696 00000100:00100000:2.0:1789760796.689824:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739901640524 time 739916116278 state 0x2 ref 0 00000100:00100000:2.0:1789760796.696402:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760796.696801:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554903808 00000100:00100000:0.0:1789760796.696811:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554903808 00000100:00100000:0.0:1789760796.696866:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739901640524 time 739916116278 state 0x0 ref 1 00000100:00100000:0.0:1789760796.696883:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739916376694 time 739916116278 state 0x1 ref 1 00000100:00100000:0.0:1789760796.696915:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739916376694 time 739942533478 state 0x0 ref 1 00000100:00100000:0.0:1789760796.696927:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554903808:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760796.696974:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68345856, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20988 00010000:00010000:0.0:1789760796.696984:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c7278 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.697000:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760796.697005:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.697011:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c7278 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68345856->68349951] (req 68345856->68349951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.697024:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c7278 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68345856->68349951] (req 68345856->68349951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.701633:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983995 is committed 00002000:00010000:0.0:1789760796.701811:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68345856 -> 68349952 00002000:00010000:0.0:1789760796.701816:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133512 -> 133520 00010000:00010000:0.0:1789760796.701830:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c7278 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68345856->68349951] (req 68345856->68349951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.701849:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c7278 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68345856->68349951] (req 68345856->68349951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.701859:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c7278 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68345856->68349951] (req 68345856->68349951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.701867:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760796.701871:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c7278 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68345856->68349951] (req 68345856->68349951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.701880:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c7278 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68345856->68349951] (req 68345856->68349951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.701897:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c7278 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68345856->68349951] (req 68345856->68349951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.701904:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c7278 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68345856->68349951] (req 68345856->68349951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.702045:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554903808:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5114us (5662us total) trans 4294983995 rc 0/0 00000100:00100000:0.0:1789760796.702055:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16697 00000100:00100000:0.0:1789760796.702065:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739916376694 time 739942533478 state 0x2 ref 0 00000100:00100000:1.0:1789760796.714201:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760796.714314:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554903936 00000100:00100000:2.0:1789760796.714324:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554903936 00000100:00100000:2.0:1789760796.714425:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739916376694 time 739942533478 state 0x0 ref 1 00000100:00100000:2.0:1789760796.714442:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739942793894 time 739942533478 state 0x1 ref 1 00000100:00100000:2.0:1789760796.714464:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739942793894 time 739960082683 state 0x0 ref 1 00000100:00100000:2.0:1789760796.714475:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554903936:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760796.714523:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68349952, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20990 00010000:00010000:2.0:1789760796.714532:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c727f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.714547:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760796.714552:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.714558:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c727f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68349952->68354047] (req 68349952->68354047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.714580:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c727f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68349952->68354047] (req 68349952->68354047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760796.720917:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983996 is committed 00002000:00010000:3.0:1789760796.721201:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68349952 -> 68354048 00002000:00010000:3.0:1789760796.721206:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133520 -> 133528 00010000:00010000:3.0:1789760796.721219:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c727f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68349952->68354047] (req 68349952->68354047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.721232:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c727f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68349952->68354047] (req 68349952->68354047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.721241:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c727f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68349952->68354047] (req 68349952->68354047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.721248:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760796.721253:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c727f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68349952->68354047] (req 68349952->68354047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.721262:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c727f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68349952->68354047] (req 68349952->68354047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.721280:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c727f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68349952->68354047] (req 68349952->68354047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.721287:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c727f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68349952->68354047] (req 68349952->68354047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.721487:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554903936:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7008us (7290us total) trans 4294983996 rc 0/0 00000100:00100000:3.0:1789760796.721498:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16698 00000100:00100000:3.0:1789760796.721507:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739942793894 time 739960082683 state 0x2 ref 0 00000100:00100000:0.0:1789760796.725458:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760796.725899:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554904064 00000100:00100000:2.0:1789760796.725909:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554904064 00000100:00100000:2.0:1789760796.726027:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739942793894 time 739960082683 state 0x0 ref 1 00000100:00100000:2.0:1789760796.726044:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739960343099 time 739960082683 state 0x1 ref 1 00000100:00100000:2.0:1789760796.726211:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739960343099 time 739971827587 state 0x0 ref 1 00000100:00100000:2.0:1789760796.726226:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e54380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554904064:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760796.726276:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68354048, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20998 00010000:00010000:2.0:1789760796.726285:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7286 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.726306:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:2.0:1789760796.726311:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.726317:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7286 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68354048->68358143] (req 68354048->68358143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.726329:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7286 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68354048->68358143] (req 68354048->68358143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760796.733547:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983997 is committed 00002000:00010000:1.0:1789760796.733856:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68354048 -> 68358144 00002000:00010000:1.0:1789760796.733862:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133528 -> 133536 00010000:00010000:1.0:1789760796.733875:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7286 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68354048->68358143] (req 68354048->68358143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.733887:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7286 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68354048->68358143] (req 68354048->68358143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.733897:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7286 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68354048->68358143] (req 68354048->68358143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.733904:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760796.733909:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7286 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68354048->68358143] (req 68354048->68358143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.733918:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7286 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68354048->68358143] (req 68354048->68358143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.733938:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7286 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68354048->68358143] (req 68354048->68358143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.733946:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7286 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68354048->68358143] (req 68354048->68358143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.734860:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554904064:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8079us (8853us total) trans 4294983997 rc 0/0 00000100:00100000:1.0:1789760796.734873:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16699 00000100:00100000:1.0:1789760796.734883:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739960343099 time 739971827587 state 0x2 ref 0 00000100:00100000:0.0:1789760796.742347:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760796.743641:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554904192 00000100:00100000:2.0:1789760796.743653:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554904192 00000100:00100000:0.0:1789760796.743837:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739960343099 time 739971827587 state 0x0 ref 1 00000100:00100000:0.0:1789760796.743853:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739972088003 time 739971827587 state 0x1 ref 1 00000100:00100000:0.0:1789760796.743885:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739972088003 time 739989503780 state 0x0 ref 1 00000100:00100000:0.0:1789760796.743898:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e57100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554904192:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760796.743958:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68358144, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x209a0 00010000:00010000:0.0:1789760796.743968:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c728d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.743983:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=739s v=(0 0) 00010000:00010000:0.0:1789760796.743987:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.743994:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c728d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68358144->68362239] (req 68358144->68362239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.744007:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c728d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68358144->68362239] (req 68358144->68362239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.751620:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983998 is committed 00002000:00010000:0.0:1789760796.751873:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68358144 -> 68362240 00002000:00010000:0.0:1789760796.751878:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133536 -> 133544 00010000:00010000:0.0:1789760796.751893:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c728d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68358144->68362239] (req 68358144->68362239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.751907:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c728d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68358144->68362239] (req 68358144->68362239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.751917:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c728d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68358144->68362239] (req 68358144->68362239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.751924:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760796.751929:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c728d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68358144->68362239] (req 68358144->68362239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.751937:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c728d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68358144->68362239] (req 68358144->68362239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.751960:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c728d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68358144->68362239] (req 68358144->68362239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.751968:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c728d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68358144->68362239] (req 68358144->68362239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.752276:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e57100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554904192:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8374us (9934us total) trans 4294983998 rc 0/0 00000100:00100000:0.0:1789760796.752286:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16700 00000100:00100000:0.0:1789760796.752296:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739972088003 time 739989503780 state 0x2 ref 0 00000100:00100000:2.0:1789760796.759777:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760796.760020:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554904320 00000100:00100000:0.0:1789760796.760031:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554904320 00000100:00100000:0.0:1789760796.760096:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739972088003 time 739989503780 state 0x0 ref 1 00000100:00100000:0.0:1789760796.760112:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739989764196 time 739989503780 state 0x1 ref 1 00000100:00100000:0.0:1789760796.760146:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739989764196 time 740005764562 state 0x0 ref 1 00000100:00100000:0.0:1789760796.760159:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554904320:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760796.760207:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68362240, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x209a8 00010000:00010000:0.0:1789760796.760216:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7294 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.760232:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760796.760237:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.760244:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7294 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68362240->68366335] (req 68362240->68366335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.760259:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7294 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68362240->68366335] (req 68362240->68366335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.768423:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294983999 is committed 00002000:00010000:0.0:1789760796.768898:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68362240 -> 68366336 00002000:00010000:0.0:1789760796.768904:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133544 -> 133552 00010000:00010000:0.0:1789760796.768917:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7294 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68362240->68366335] (req 68362240->68366335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.768930:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7294 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68362240->68366335] (req 68362240->68366335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.768939:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7294 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68362240->68366335] (req 68362240->68366335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.768951:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760796.768956:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7294 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68362240->68366335] (req 68362240->68366335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.768965:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7294 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68362240->68366335] (req 68362240->68366335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.768984:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7294 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68362240->68366335] (req 68362240->68366335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.768992:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7294 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68362240->68366335] (req 68362240->68366335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.769455:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554904320:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9292us (9681us total) trans 4294983999 rc 0/0 00000100:00100000:0.0:1789760796.769466:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16701 00000100:00100000:0.0:1789760796.769476:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739989764196 time 740005764562 state 0x2 ref 0 00000100:00100000:1.0:1789760796.779280:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760796.779642:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554904448 00000100:00100000:3.0:1789760796.779651:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554904448 00000100:00100000:1.0:1789760796.780069:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 739989764196 time 740005764562 state 0x0 ref 1 00000100:00100000:1.0:1789760796.780093:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740006024978 time 740005764562 state 0x1 ref 1 00000100:00100000:1.0:1789760796.780122:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740006024978 time 740025741086 state 0x0 ref 1 00000100:00100000:1.0:1789760796.780133:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554904448:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760796.780175:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68366336, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x209b0 00010000:00010000:1.0:1789760796.780184:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c729b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.780199:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760796.780204:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760796.780210:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c729b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68366336->68370431] (req 68366336->68370431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.780222:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c729b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68366336->68370431] (req 68366336->68370431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760796.786558:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984000 is committed 00002000:00010000:1.0:1789760796.786862:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68366336 -> 68370432 00002000:00010000:1.0:1789760796.786868:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133552 -> 133560 00010000:00010000:1.0:1789760796.786881:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c729b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68366336->68370431] (req 68366336->68370431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.786893:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c729b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68366336->68370431] (req 68366336->68370431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.786902:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c729b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68366336->68370431] (req 68366336->68370431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.786909:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655cc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760796.786914:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c729b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68366336->68370431] (req 68366336->68370431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.786923:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c729b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68366336->68370431] (req 68366336->68370431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.786942:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c729b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68366336->68370431] (req 68366336->68370431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760796.786966:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf9337f4/0x1f99687d854c729b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68366336->68370431] (req 68366336->68370431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760796.787114:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554904448:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6977us (7836us total) trans 4294984000 rc 0/0 00000100:00100000:1.0:1789760796.787124:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16702 00000100:00100000:1.0:1789760796.787133:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740006024978 time 740025741086 state 0x2 ref 0 00000100:00100000:1.0:1789760796.791498:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760796.791590:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554904576 00000100:00100000:3.0:1789760796.791600:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554904576 00000100:00100000:3.0:1789760796.792183:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740006024978 time 740025741086 state 0x0 ref 1 00000100:00100000:3.0:1789760796.792199:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740026001502 time 740025741086 state 0x1 ref 1 00000100:00100000:3.0:1789760796.792277:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740026001502 time 740037895117 state 0x0 ref 1 00000100:00100000:3.0:1789760796.792290:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554904576:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760796.792338:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68370432, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x209b8 00010000:00010000:3.0:1789760796.792347:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c72a2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.792363:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760796.792367:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.792374:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c72a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68370432->68374527] (req 68370432->68374527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.792411:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c72a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68370432->68374527] (req 68370432->68374527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.798506:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984001 is committed 00002000:00010000:3.0:1789760796.798928:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68370432 -> 68374528 00002000:00010000:3.0:1789760796.798934:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133560 -> 133568 00010000:00010000:3.0:1789760796.798954:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c72a2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68370432->68374527] (req 68370432->68374527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.798969:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c72a2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68370432->68374527] (req 68370432->68374527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.798978:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c72a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68370432->68374527] (req 68370432->68374527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.798985:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760796.798991:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c72a2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68370432->68374527] (req 68370432->68374527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.799001:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c72a2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68370432->68374527] (req 68370432->68374527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.799022:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c72a2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68370432->68374527] (req 68370432->68374527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.799030:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c72a2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68370432->68374527] (req 68370432->68374527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.799123:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554904576:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6830us (7630us total) trans 4294984001 rc 0/0 00000100:00100000:3.0:1789760796.799135:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16703 00000100:00100000:3.0:1789760796.799144:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740026001502 time 740037895117 state 0x2 ref 0 00000100:00100000:1.0:1789760796.808338:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760796.808471:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554904704 00000100:00100000:0.0:1789760796.808482:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554904704 00000100:00100000:0.0:1789760796.808532:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740026001502 time 740037895117 state 0x0 ref 1 00000100:00100000:0.0:1789760796.808548:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740038155533 time 740037895117 state 0x1 ref 1 00000100:00100000:0.0:1789760796.808587:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740038155533 time 740054205252 state 0x0 ref 1 00000100:00100000:0.0:1789760796.808603:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554904704:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760796.808649:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68374528, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x209c0 00010000:00010000:0.0:1789760796.808657:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72a9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.808674:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760796.808678:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.808684:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68374528->68378623] (req 68374528->68378623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.808709:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68374528->68378623] (req 68374528->68378623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.814839:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984002 is committed 00002000:00010000:0.0:1789760796.815215:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68374528 -> 68378624 00002000:00010000:0.0:1789760796.815220:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133568 -> 133576 00010000:00010000:0.0:1789760796.815233:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72a9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68374528->68378623] (req 68374528->68378623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.815246:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72a9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68374528->68378623] (req 68374528->68378623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.815255:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68374528->68378623] (req 68374528->68378623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.815263:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760796.815268:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72a9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68374528->68378623] (req 68374528->68378623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.815277:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72a9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68374528->68378623] (req 68374528->68378623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.815297:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72a9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68374528->68378623] (req 68374528->68378623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.815304:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72a9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68374528->68378623] (req 68374528->68378623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.815900:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554904704:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7293us (7565us total) trans 4294984002 rc 0/0 00000100:00100000:0.0:1789760796.815912:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16704 00000100:00100000:0.0:1789760796.815921:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740038155533 time 740054205252 state 0x2 ref 0 00000100:00100000:1.0:1789760796.821845:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760796.822312:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554904832 00000100:00100000:0.0:1789760796.822322:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554904832 00000100:00100000:0.0:1789760796.822516:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740038155533 time 740054205252 state 0x0 ref 1 00000100:00100000:0.0:1789760796.822531:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740054465668 time 740054205252 state 0x1 ref 1 00000100:00100000:3.0:1789760796.822725:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740054465668 time 740068340823 state 0x0 ref 1 00000100:00100000:3.0:1789760796.822746:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc07bcf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554904832:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760796.822796:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68378624, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x209c8 00010000:00010000:3.0:1789760796.822805:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c72b0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.822822:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760796.822827:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760796.822834:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c72b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68378624->68382719] (req 68378624->68382719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.822849:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c72b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68378624->68382719] (req 68378624->68382719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.828133:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984003 is committed 00002000:00010000:3.0:1789760796.828307:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68378624 -> 68382720 00002000:00010000:3.0:1789760796.828312:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133576 -> 133584 00010000:00010000:3.0:1789760796.828325:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c72b0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68378624->68382719] (req 68378624->68382719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.828338:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c72b0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68378624->68382719] (req 68378624->68382719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.828346:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c72b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68378624->68382719] (req 68378624->68382719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.828354:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760796.828359:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c72b0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68378624->68382719] (req 68378624->68382719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.828368:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c72b0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68378624->68382719] (req 68378624->68382719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.828414:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c72b0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68378624->68382719] (req 68378624->68382719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.828428:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c72b0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68378624->68382719] (req 68378624->68382719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.828653:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc07bcf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554904832:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5910us (6812us total) trans 4294984003 rc 0/0 00000100:00100000:2.0:1789760796.828665:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16705 00000100:00100000:2.0:1789760796.828674:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740054465668 time 740068340823 state 0x2 ref 0 00000100:00100000:1.0:1789760796.832534:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760796.832694:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554904960 00000100:00100000:2.0:1789760796.832703:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554904960 00000100:00100000:2.0:1789760796.832755:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740054465668 time 740068340823 state 0x0 ref 1 00000100:00100000:2.0:1789760796.832770:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740068601239 time 740068340823 state 0x1 ref 1 00000100:00100000:2.0:1789760796.832805:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740068601239 time 740078423425 state 0x0 ref 1 00000100:00100000:2.0:1789760796.832818:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554904960:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760796.832862:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68382720, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x209d0 00010000:00010000:2.0:1789760796.832872:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72b7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.832892:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760796.832897:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.832903:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68382720->68386815] (req 68382720->68386815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.832916:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68382720->68386815] (req 68382720->68386815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.837654:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984004 is committed 00002000:00010000:2.0:1789760796.837815:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68382720 -> 68386816 00002000:00010000:2.0:1789760796.837821:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133584 -> 133592 00010000:00010000:2.0:1789760796.837834:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72b7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68382720->68386815] (req 68382720->68386815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.837846:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72b7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68382720->68386815] (req 68382720->68386815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.837856:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68382720->68386815] (req 68382720->68386815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.837863:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760796.837868:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72b7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68382720->68386815] (req 68382720->68386815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.837876:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72b7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68382720->68386815] (req 68382720->68386815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.837898:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72b7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68382720->68386815] (req 68382720->68386815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.837905:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72b7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68382720->68386815] (req 68382720->68386815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.838006:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554904960:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5186us (5476us total) trans 4294984004 rc 0/0 00000100:00100000:2.0:1789760796.838016:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16706 00000100:00100000:2.0:1789760796.838025:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740068601239 time 740078423425 state 0x2 ref 0 00000100:00100000:1.0:1789760796.845704:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760796.845796:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554905088 00000100:00100000:0.0:1789760796.845805:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554905088 00000100:00100000:0.0:1789760796.845854:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740068601239 time 740078423425 state 0x0 ref 1 00000100:00100000:0.0:1789760796.845868:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740078683841 time 740078423425 state 0x1 ref 1 00000100:00100000:0.0:1789760796.845898:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740078683841 time 740091515986 state 0x0 ref 1 00000100:00100000:0.0:1789760796.845910:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554905088:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760796.845958:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68386816, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x209d8 00010000:00010000:0.0:1789760796.845967:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72be lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.845982:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760796.845986:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.845992:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68386816->68390911] (req 68386816->68390911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.846006:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68386816->68390911] (req 68386816->68390911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.850686:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984005 is committed 00002000:00010000:0.0:1789760796.850918:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68386816 -> 68390912 00002000:00010000:0.0:1789760796.850923:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133592 -> 133600 00010000:00010000:0.0:1789760796.850936:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72be lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68386816->68390911] (req 68386816->68390911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.850958:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72be lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68386816->68390911] (req 68386816->68390911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.850967:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68386816->68390911] (req 68386816->68390911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.850979:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760796.850984:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72be lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68386816->68390911] (req 68386816->68390911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.850993:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72be lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68386816->68390911] (req 68386816->68390911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.851011:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72be lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68386816->68390911] (req 68386816->68390911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.851019:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72be lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68386816->68390911] (req 68386816->68390911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.851149:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554905088:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5235us (5447us total) trans 4294984005 rc 0/0 00000100:00100000:0.0:1789760796.851159:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16707 00000100:00100000:0.0:1789760796.851169:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740078683841 time 740091515986 state 0x2 ref 0 00000100:00100000:1.0:1789760796.858209:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760796.858981:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554905216 00000100:00100000:0.0:1789760796.858993:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554905216 00000100:00100000:0.0:1789760796.859045:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740078683841 time 740091515986 state 0x0 ref 1 00000100:00100000:0.0:1789760796.859060:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740091776402 time 740091515986 state 0x1 ref 1 00000100:00100000:0.0:1789760796.859099:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740091776402 time 740104717930 state 0x0 ref 1 00000100:00100000:0.0:1789760796.859115:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554905216:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760796.859160:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68390912, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x209e0 00010000:00010000:0.0:1789760796.859170:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72c5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.859185:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760796.859190:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.859196:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68390912->68395007] (req 68390912->68395007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.859211:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68390912->68395007] (req 68390912->68395007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.865182:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984006 is committed 00002000:00010000:0.0:1789760796.865287:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68390912 -> 68395008 00002000:00010000:0.0:1789760796.865292:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133600 -> 133608 00010000:00010000:0.0:1789760796.865305:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72c5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68390912->68395007] (req 68390912->68395007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.865318:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72c5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68390912->68395007] (req 68390912->68395007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.865326:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68390912->68395007] (req 68390912->68395007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.865334:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760796.865345:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72c5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68390912->68395007] (req 68390912->68395007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.865354:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72c5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68390912->68395007] (req 68390912->68395007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.865372:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72c5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68390912->68395007] (req 68390912->68395007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.865557:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72c5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68390912->68395007] (req 68390912->68395007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.865659:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554905216:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6541us (7455us total) trans 4294984006 rc 0/0 00000100:00100000:0.0:1789760796.865669:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16708 00000100:00100000:0.0:1789760796.865678:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740091776402 time 740104717930 state 0x2 ref 0 00000100:00100000:1.0:1789760796.870726:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760796.870824:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554905344 00000100:00100000:2.0:1789760796.870834:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554905344 00000100:00100000:2.0:1789760796.870887:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740091776402 time 740104717930 state 0x0 ref 1 00000100:00100000:2.0:1789760796.870901:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740104978346 time 740104717930 state 0x1 ref 1 00000100:00100000:2.0:1789760796.870958:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740104978346 time 740116575980 state 0x0 ref 1 00000100:00100000:2.0:1789760796.870970:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554905344:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760796.871018:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68395008, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x209e8 00010000:00010000:2.0:1789760796.871027:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72cc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.871043:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760796.871049:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.871055:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68395008->68399103] (req 68395008->68399103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.871069:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68395008->68399103] (req 68395008->68399103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.875446:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984007 is committed 00002000:00010000:2.0:1789760796.875653:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68395008 -> 68399104 00002000:00010000:2.0:1789760796.875658:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133608 -> 133616 00010000:00010000:2.0:1789760796.875670:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72cc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68395008->68399103] (req 68395008->68399103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.875683:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72cc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68395008->68399103] (req 68395008->68399103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.875692:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68395008->68399103] (req 68395008->68399103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.875699:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760796.875704:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72cc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68395008->68399103] (req 68395008->68399103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.875713:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72cc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68395008->68399103] (req 68395008->68399103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.875733:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72cc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68395008->68399103] (req 68395008->68399103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.875745:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72cc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68395008->68399103] (req 68395008->68399103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.875833:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554905344:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4860us (5113us total) trans 4294984007 rc 0/0 00000100:00100000:2.0:1789760796.875843:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16709 00000100:00100000:2.0:1789760796.875852:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740104978346 time 740116575980 state 0x2 ref 0 00000100:00100000:1.0:1789760796.887802:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760796.887930:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554905472 00000100:00100000:2.0:1789760796.887942:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554905472 00000100:00100000:2.0:1789760796.888000:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740104978346 time 740116575980 state 0x0 ref 1 00000100:00100000:2.0:1789760796.888016:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740116836396 time 740116575980 state 0x1 ref 1 00000100:00100000:2.0:1789760796.888088:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740116836396 time 740133705865 state 0x0 ref 1 00000100:00100000:2.0:1789760796.888112:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3786c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554905472:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760796.888160:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68399104, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x209f0 00010000:00010000:2.0:1789760796.888170:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72d3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.888188:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760796.888192:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.888199:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68399104->68403199] (req 68399104->68403199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.888218:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68399104->68403199] (req 68399104->68403199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.895100:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984008 is committed 00002000:00010000:2.0:1789760796.895774:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68399104 -> 68403200 00002000:00010000:2.0:1789760796.895780:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133616 -> 133624 00010000:00010000:2.0:1789760796.895792:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72d3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68399104->68403199] (req 68399104->68403199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.895804:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72d3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68399104->68403199] (req 68399104->68403199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.895811:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68399104->68403199] (req 68399104->68403199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.895818:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760796.895822:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72d3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68399104->68403199] (req 68399104->68403199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.895832:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72d3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68399104->68403199] (req 68399104->68403199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.895852:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72d3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68399104->68403199] (req 68399104->68403199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.895859:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72d3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68399104->68403199] (req 68399104->68403199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.895959:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3786c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554905472:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7855us (8161us total) trans 4294984008 rc 0/0 00000100:00100000:2.0:1789760796.895968:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16710 00000100:00100000:2.0:1789760796.895977:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740116836396 time 740133705865 state 0x2 ref 0 00000100:00100000:1.0:1789760796.900305:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760796.900674:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554905600 00000100:00100000:0.0:1789760796.900683:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554905600 00000100:00100000:0.0:1789760796.900732:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740116836396 time 740133705865 state 0x0 ref 1 00000100:00100000:0.0:1789760796.900745:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740133966281 time 740133705865 state 0x1 ref 1 00000100:00100000:0.0:1789760796.900799:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740133966281 time 740146417048 state 0x0 ref 1 00000100:00100000:0.0:1789760796.900811:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554905600:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760796.900853:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68403200, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x209f8 00010000:00010000:0.0:1789760796.900862:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72da lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.900877:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760796.900881:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.900887:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68403200->68407295] (req 68403200->68407295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.900900:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68403200->68407295] (req 68403200->68407295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.907272:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984009 is committed 00002000:00010000:0.0:1789760796.907429:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68403200 -> 68407296 00002000:00010000:0.0:1789760796.907434:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133624 -> 133632 00010000:00010000:0.0:1789760796.907449:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72da lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68403200->68407295] (req 68403200->68407295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.907462:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72da lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68403200->68407295] (req 68403200->68407295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.907481:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68403200->68407295] (req 68403200->68407295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.907489:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760796.907494:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72da lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68403200->68407295] (req 68403200->68407295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.907504:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72da lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68403200->68407295] (req 68403200->68407295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.907524:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72da lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68403200->68407295] (req 68403200->68407295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.907531:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72da lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68403200->68407295] (req 68403200->68407295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.907627:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554905600:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6813us (7327us total) trans 4294984009 rc 0/0 00000100:00100000:0.0:1789760796.907638:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16711 00000100:00100000:0.0:1789760796.907647:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740133966281 time 740146417048 state 0x2 ref 0 00000100:00100000:1.0:1789760796.910995:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760796.911083:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554905728 00000100:00100000:0.0:1789760796.911092:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554905728 00000100:00100000:0.0:1789760796.911143:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740133966281 time 740146417048 state 0x0 ref 1 00000100:00100000:0.0:1789760796.911158:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740146677464 time 740146417048 state 0x1 ref 1 00000100:00100000:0.0:1789760796.911192:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740146677464 time 740156810727 state 0x0 ref 1 00000100:00100000:0.0:1789760796.911208:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554905728:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760796.911256:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68407296, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20a00 00010000:00010000:0.0:1789760796.911265:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c72e1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.911280:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760796.911285:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.911292:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c72e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68407296->68411391] (req 68407296->68411391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.911306:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c72e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68407296->68411391] (req 68407296->68411391) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.919724:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984010 is committed 00002000:00010000:0.0:1789760796.919840:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68407296 -> 68411392 00002000:00010000:0.0:1789760796.919846:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133632 -> 133640 00010000:00010000:0.0:1789760796.919858:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c72e1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68407296->68411391] (req 68407296->68411391) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.919871:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c72e1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68407296->68411391] (req 68407296->68411391) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.919879:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c72e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68407296->68411391] (req 68407296->68411391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.919886:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760796.919891:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c72e1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68407296->68411391] (req 68407296->68411391) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.919901:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c72e1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68407296->68411391] (req 68407296->68411391) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.919921:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c72e1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68407296->68411391] (req 68407296->68411391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.919933:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c72e1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68407296->68411391] (req 68407296->68411391) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.920074:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554905728:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8865us (9087us total) trans 4294984010 rc 0/0 00000100:00100000:0.0:1789760796.920085:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16712 00000100:00100000:0.0:1789760796.920094:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740146677464 time 740156810727 state 0x2 ref 0 00000100:00100000:1.0:1789760796.930487:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760796.930919:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554905856 00000100:00100000:2.0:1789760796.930929:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554905856 00000100:00100000:2.0:1789760796.930994:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740146677464 time 740156810727 state 0x0 ref 1 00000100:00100000:2.0:1789760796.931009:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740157071143 time 740156810727 state 0x1 ref 1 00000100:00100000:2.0:1789760796.931042:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740157071143 time 740176657698 state 0x0 ref 1 00000100:00100000:2.0:1789760796.931053:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554905856:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760796.931099:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68411392, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20a08 00010000:00010000:2.0:1789760796.931107:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72e8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.931121:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760796.931126:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.931132:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68411392->68415487] (req 68411392->68415487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.931144:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68411392->68415487] (req 68411392->68415487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.936109:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984011 is committed 00002000:00010000:2.0:1789760796.943661:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68411392 -> 68415488 00002000:00010000:2.0:1789760796.943671:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133640 -> 133648 00010000:00010000:2.0:1789760796.943693:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72e8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68411392->68415487] (req 68411392->68415487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.943954:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72e8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68411392->68415487] (req 68411392->68415487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.944024:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68411392->68415487] (req 68411392->68415487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.944031:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760796.944036:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72e8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68411392->68415487] (req 68411392->68415487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.944045:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72e8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68411392->68415487] (req 68411392->68415487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.944064:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72e8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68411392->68415487] (req 68411392->68415487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.944071:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72e8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68411392->68415487] (req 68411392->68415487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760796.944176:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554905856:12345-192.168.203.22@tcp:4:dd.0 Request processed in 13119us (13695us total) trans 4294984011 rc 0/0 00000100:00100000:2.0:1789760796.944188:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16713 00000100:00100000:2.0:1789760796.944198:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740157071143 time 740176657698 state 0x2 ref 0 00000100:00100000:1.0:1789760796.954321:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760796.954443:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554905984 00000100:00100000:2.0:1789760796.954453:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554905984 00000100:00100000:2.0:1789760796.954504:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740157071143 time 740176657698 state 0x0 ref 1 00000100:00100000:2.0:1789760796.954521:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740176918114 time 740176657698 state 0x1 ref 1 00000100:00100000:2.0:1789760796.954553:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740176918114 time 740200171707 state 0x0 ref 1 00000100:00100000:2.0:1789760796.954565:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554905984:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760796.954620:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68415488, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20a10 00010000:00010000:2.0:1789760796.954629:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72ef lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.954644:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760796.954648:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760796.954654:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68415488->68419583] (req 68415488->68419583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760796.954668:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68415488->68419583] (req 68415488->68419583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.959860:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984012 is committed 00002000:00010000:3.0:1789760796.961618:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68415488 -> 68419584 00002000:00010000:3.0:1789760796.961626:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133648 -> 133656 00010000:00010000:3.0:1789760796.961641:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72ef lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68415488->68419583] (req 68415488->68419583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.961653:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72ef lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68415488->68419583] (req 68415488->68419583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.961661:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68415488->68419583] (req 68415488->68419583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.961668:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760796.961672:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72ef lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68415488->68419583] (req 68415488->68419583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.961681:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72ef lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68415488->68419583] (req 68415488->68419583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.962109:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72ef lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68415488->68419583] (req 68415488->68419583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760796.962119:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c72ef lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68415488->68419583] (req 68415488->68419583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760796.962221:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554905984:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7653us (7904us total) trans 4294984012 rc 0/0 00000100:00100000:3.0:1789760796.962232:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16714 00000100:00100000:3.0:1789760796.962242:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740176918114 time 740200171707 state 0x2 ref 0 00000100:00100000:1.0:1789760796.972858:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760796.972974:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554906112 00000100:00100000:0.0:1789760796.972988:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554906112 00000100:00100000:0.0:1789760796.973040:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740176918114 time 740200171707 state 0x0 ref 1 00000100:00100000:0.0:1789760796.973056:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740200432123 time 740200171707 state 0x1 ref 1 00000100:00100000:0.0:1789760796.973089:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740200432123 time 740218707173 state 0x0 ref 1 00000100:00100000:0.0:1789760796.973101:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554906112:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760796.973145:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68419584, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20a18 00010000:00010000:0.0:1789760796.973153:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c72f6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.973169:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760796.973174:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.973180:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c72f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68419584->68423679] (req 68419584->68423679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.973199:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c72f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68419584->68423679] (req 68419584->68423679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.981254:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984013 is committed 00002000:00010000:0.0:1789760796.983580:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68419584 -> 68423680 00002000:00010000:0.0:1789760796.983590:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133656 -> 133664 00010000:00010000:0.0:1789760796.983604:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c72f6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68419584->68423679] (req 68419584->68423679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.983615:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c72f6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68419584->68423679] (req 68419584->68423679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.983623:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c72f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68419584->68423679] (req 68419584->68423679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.983631:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760796.983635:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c72f6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68419584->68423679] (req 68419584->68423679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.983643:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c72f6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68419584->68423679] (req 68419584->68423679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.983910:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c72f6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68419584->68423679] (req 68419584->68423679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.983922:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c72f6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68419584->68423679] (req 68419584->68423679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.984228:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554906112:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11124us (11375us total) trans 4294984013 rc 0/0 00000100:00100000:0.0:1789760796.984240:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16715 00000100:00100000:0.0:1789760796.984250:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740200432123 time 740218707173 state 0x2 ref 0 00000100:00100000:1.0:1789760796.989794:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760796.989904:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554906240 00000100:00100000:0.0:1789760796.989915:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554906240 00000100:00100000:0.0:1789760796.990071:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740200432123 time 740218707173 state 0x0 ref 1 00000100:00100000:0.0:1789760796.990085:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740218967589 time 740218707173 state 0x1 ref 1 00000100:00100000:0.0:1789760796.990149:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740218967589 time 740235767443 state 0x0 ref 1 00000100:00100000:0.0:1789760796.990162:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554906240:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760796.990208:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68423680, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20a20 00010000:00010000:0.0:1789760796.990218:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72fd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.990234:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760796.990238:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760796.990244:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68423680->68427775] (req 68423680->68427775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.990258:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68423680->68427775] (req 68423680->68427775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760796.996614:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984014 is committed 00002000:00010000:0.0:1789760796.996795:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68423680 -> 68427776 00002000:00010000:0.0:1789760796.996801:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133664 -> 133672 00010000:00010000:0.0:1789760796.996813:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72fd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68423680->68427775] (req 68423680->68427775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.996825:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72fd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68423680->68427775] (req 68423680->68427775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.996834:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68423680->68427775] (req 68423680->68427775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.996847:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760796.996852:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72fd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68423680->68427775] (req 68423680->68427775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.996861:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72fd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68423680->68427775] (req 68423680->68427775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.996881:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72fd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68423680->68427775] (req 68423680->68427775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760796.996890:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c72fd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68423680->68427775] (req 68423680->68427775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760796.997047:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554906240:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6882us (7256us total) trans 4294984014 rc 0/0 00000100:00100000:0.0:1789760796.997058:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16716 00000100:00100000:0.0:1789760796.997068:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740218967589 time 740235767443 state 0x2 ref 0 00000100:00100000:1.0:1789760797.000781:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760797.000883:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554906368 00000100:00100000:2.0:1789760797.000893:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554906368 00000100:00100000:2.0:1789760797.000986:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740218967589 time 740235767443 state 0x0 ref 1 00000100:00100000:2.0:1789760797.000998:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740236027859 time 740235767443 state 0x1 ref 1 00000100:00100000:2.0:1789760797.001020:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740236027859 time 740246640040 state 0x0 ref 1 00000100:00100000:2.0:1789760797.001030:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554906368:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760797.001075:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68427776, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20a28 00010000:00010000:2.0:1789760797.001084:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7304 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.001099:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760797.001103:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.001110:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7304 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68427776->68431871] (req 68427776->68431871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.001129:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7304 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68427776->68431871] (req 68427776->68431871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789760797.001429:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:1.0:1789760797.007447:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984015 is committed 00002000:00010000:2.0:1789760797.007623:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68427776 -> 68431872 00002000:00010000:2.0:1789760797.007750:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133672 -> 133680 00010000:00010000:2.0:1789760797.007765:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7304 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68427776->68431871] (req 68427776->68431871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.007778:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7304 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68427776->68431871] (req 68427776->68431871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.007792:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7304 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68427776->68431871] (req 68427776->68431871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.007798:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.007803:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7304 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68427776->68431871] (req 68427776->68431871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.007812:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7304 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68427776->68431871] (req 68427776->68431871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.007834:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7304 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68427776->68431871] (req 68427776->68431871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.007841:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7304 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68427776->68431871] (req 68427776->68431871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.007992:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554906368:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6958us (7216us total) trans 4294984015 rc 0/0 00000100:00100000:2.0:1789760797.008003:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16717 00000100:00100000:2.0:1789760797.008013:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740236027859 time 740246640040 state 0x2 ref 0 00000100:00100000:1.0:1789760797.013615:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760797.013810:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554906496 00000100:00100000:3.0:1789760797.013820:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554906496 00000100:00100000:3.0:1789760797.013875:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740236027859 time 740246640040 state 0x0 ref 1 00000100:00100000:3.0:1789760797.013890:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740246900456 time 740246640040 state 0x1 ref 1 00000100:00100000:3.0:1789760797.013920:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740246900456 time 740259538811 state 0x0 ref 1 00000100:00100000:3.0:1789760797.013933:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554906496:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760797.014034:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68431872, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20a30 00010000:00010000:3.0:1789760797.014504:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c730b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.014525:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760797.014529:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.014537:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c730b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68431872->68435967] (req 68431872->68435967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.014552:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c730b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68431872->68435967] (req 68431872->68435967) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.021520:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984016 is committed 00002000:00010000:3.0:1789760797.021781:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68431872 -> 68435968 00002000:00010000:3.0:1789760797.021786:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133680 -> 133688 00010000:00010000:3.0:1789760797.021800:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c730b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68431872->68435967] (req 68431872->68435967) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.021817:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c730b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68431872->68435967] (req 68431872->68435967) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.021826:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c730b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68431872->68435967] (req 68431872->68435967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.021835:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760797.021839:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c730b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68431872->68435967] (req 68431872->68435967) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.021849:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c730b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68431872->68435967] (req 68431872->68435967) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.021868:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c730b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68431872->68435967] (req 68431872->68435967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.021876:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c730b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68431872->68435967] (req 68431872->68435967) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.022125:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554906496:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8190us (8514us total) trans 4294984016 rc 0/0 00000100:00100000:3.0:1789760797.022137:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16718 00000100:00100000:3.0:1789760797.022146:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740246900456 time 740259538811 state 0x2 ref 0 00000100:00100000:1.0:1789760797.027132:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760797.027341:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554906624 00000100:00100000:0.0:1789760797.027351:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554906624 00000100:00100000:0.0:1789760797.027706:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740246900456 time 740259538811 state 0x0 ref 1 00000100:00100000:0.0:1789760797.027722:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740259799227 time 740259538811 state 0x1 ref 1 00000100:00100000:0.0:1789760797.027790:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740259799227 time 740273407559 state 0x0 ref 1 00000100:00100000:0.0:1789760797.027805:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb067b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554906624:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760797.027858:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68435968, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20a38 00010000:00010000:0.0:1789760797.027868:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7312 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.027885:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760797.027890:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.027897:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7312 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68435968->68440063] (req 68435968->68440063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.027911:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7312 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68435968->68440063] (req 68435968->68440063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.033840:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984017 is committed 00002000:00010000:0.0:1789760797.034732:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68435968 -> 68440064 00002000:00010000:0.0:1789760797.034738:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133688 -> 133696 00010000:00010000:0.0:1789760797.034753:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7312 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68435968->68440063] (req 68435968->68440063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.034766:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7312 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68435968->68440063] (req 68435968->68440063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.034775:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7312 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68435968->68440063] (req 68435968->68440063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.034782:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760797.034786:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7312 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68435968->68440063] (req 68435968->68440063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.034795:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7312 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68435968->68440063] (req 68435968->68440063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.034817:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7312 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68435968->68440063] (req 68435968->68440063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.034824:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7312 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68435968->68440063] (req 68435968->68440063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.035031:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb067b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554906624:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7223us (7901us total) trans 4294984017 rc 0/0 00000100:00100000:0.0:1789760797.035101:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16719 00000100:00100000:0.0:1789760797.035113:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740259799227 time 740273407559 state 0x2 ref 0 00000100:00100000:0.0:1789760797.043161:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760797.043259:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554906752 00000100:00100000:2.0:1789760797.043268:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554906752 00000100:00100000:2.0:1789760797.043317:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740259799227 time 740273407559 state 0x0 ref 1 00000100:00100000:2.0:1789760797.043331:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740273667975 time 740273407559 state 0x1 ref 1 00000100:00100000:2.0:1789760797.043359:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740273667975 time 740288977728 state 0x0 ref 1 00000100:00100000:2.0:1789760797.043371:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554906752:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760797.043445:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68440064, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20a40 00010000:00010000:2.0:1789760797.043454:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7319 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.043469:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760797.043473:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.043479:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7319 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68440064->68444159] (req 68440064->68444159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.043491:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7319 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68440064->68444159] (req 68440064->68444159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.048436:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984018 is committed 00002000:00010000:2.0:1789760797.048627:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68440064 -> 68444160 00002000:00010000:2.0:1789760797.048633:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133696 -> 133704 00010000:00010000:2.0:1789760797.048650:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7319 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68440064->68444159] (req 68440064->68444159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.048662:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7319 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68440064->68444159] (req 68440064->68444159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.048671:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7319 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68440064->68444159] (req 68440064->68444159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.048683:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.048687:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7319 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68440064->68444159] (req 68440064->68444159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.048696:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7319 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68440064->68444159] (req 68440064->68444159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.048716:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7319 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68440064->68444159] (req 68440064->68444159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.048724:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7319 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68440064->68444159] (req 68440064->68444159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.049034:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554906752:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5660us (5881us total) trans 4294984018 rc 0/0 00000100:00100000:2.0:1789760797.049053:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16720 00000100:00100000:2.0:1789760797.049062:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740273667975 time 740288977728 state 0x2 ref 0 00000100:00100000:1.0:1789760797.053935:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760797.054624:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554906880 00000100:00100000:2.0:1789760797.054635:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554906880 00000100:00100000:2.0:1789760797.054693:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740273667975 time 740288977728 state 0x0 ref 1 00000100:00100000:2.0:1789760797.054708:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740289238144 time 740288977728 state 0x1 ref 1 00000100:00100000:2.0:1789760797.054784:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740289238144 time 740300401182 state 0x0 ref 1 00000100:00100000:2.0:1789760797.054801:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06401c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554906880:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760797.054855:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68444160, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20a48 00010000:00010000:2.0:1789760797.054864:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c7320 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.054878:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760797.054882:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.054888:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c7320 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68444160->68448255] (req 68444160->68448255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.054903:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c7320 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68444160->68448255] (req 68444160->68448255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.061056:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984019 is committed 00002000:00010000:2.0:1789760797.061177:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68444160 -> 68448256 00002000:00010000:2.0:1789760797.061182:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133704 -> 133712 00010000:00010000:2.0:1789760797.061195:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c7320 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68444160->68448255] (req 68444160->68448255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.061208:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c7320 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68444160->68448255] (req 68444160->68448255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.061217:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c7320 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68444160->68448255] (req 68444160->68448255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.061224:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.061229:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c7320 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68444160->68448255] (req 68444160->68448255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.061238:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c7320 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68444160->68448255] (req 68444160->68448255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.061257:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c7320 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68444160->68448255] (req 68444160->68448255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.061263:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c7320 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68444160->68448255] (req 68444160->68448255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.061362:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06401c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554906880:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6559us (7432us total) trans 4294984019 rc 0/0 00000100:00100000:2.0:1789760797.061373:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16721 00000100:00100000:2.0:1789760797.061676:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740289238144 time 740300401182 state 0x2 ref 0 00000100:00100000:1.0:1789760797.069484:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760797.070280:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554907008 00000100:00100000:3.0:1789760797.070290:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554907008 00000100:00100000:3.0:1789760797.071025:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740289238144 time 740300401182 state 0x0 ref 1 00000100:00100000:3.0:1789760797.071041:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740300661598 time 740300401182 state 0x1 ref 1 00000100:00100000:3.0:1789760797.071223:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740300661598 time 740316838831 state 0x0 ref 1 00000100:00100000:3.0:1789760797.071238:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06402d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554907008:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760797.071286:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68448256, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20a50 00010000:00010000:3.0:1789760797.071296:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7327 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.071312:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760797.071317:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.071323:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7327 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68448256->68452351] (req 68448256->68452351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.071350:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7327 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68448256->68452351] (req 68448256->68452351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.076732:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984020 is committed 00002000:00010000:3.0:1789760797.076904:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68448256 -> 68452352 00002000:00010000:3.0:1789760797.076909:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133712 -> 133720 00010000:00010000:3.0:1789760797.076923:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7327 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68448256->68452351] (req 68448256->68452351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.076936:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7327 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68448256->68452351] (req 68448256->68452351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.076955:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7327 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68448256->68452351] (req 68448256->68452351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.076964:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760797.076968:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7327 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68448256->68452351] (req 68448256->68452351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.076978:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7327 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68448256->68452351] (req 68448256->68452351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.076997:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7327 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68448256->68452351] (req 68448256->68452351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.077005:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7327 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68448256->68452351] (req 68448256->68452351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.078039:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06402d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554907008:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6795us (8555us total) trans 4294984020 rc 0/0 00000100:00100000:3.0:1789760797.078052:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16722 00000100:00100000:3.0:1789760797.078061:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740300661598 time 740316838831 state 0x2 ref 0 00000100:00100000:0.0:1789760797.083990:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760797.084122:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554907136 00000100:00100000:1.0:1789760797.084132:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554907136 00000100:00100000:1.0:1789760797.084187:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740300661598 time 740316838831 state 0x0 ref 1 00000100:00100000:1.0:1789760797.084202:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740317099247 time 740316838831 state 0x1 ref 1 00000100:00100000:1.0:1789760797.084238:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740317099247 time 740329856209 state 0x0 ref 1 00000100:00100000:1.0:1789760797.084251:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e54e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554907136:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760797.084298:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68452352, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20a58 00010000:00010000:1.0:1789760797.084308:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c732e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.084334:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760797.084339:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.084345:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c732e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68452352->68456447] (req 68452352->68456447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.084359:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c732e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68452352->68456447] (req 68452352->68456447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.092794:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984021 is committed 00002000:00010000:1.0:1789760797.093117:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68452352 -> 68456448 00002000:00010000:1.0:1789760797.093123:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133720 -> 133728 00010000:00010000:1.0:1789760797.093137:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c732e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68452352->68456447] (req 68452352->68456447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.093150:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c732e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68452352->68456447] (req 68452352->68456447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.093158:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c732e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68452352->68456447] (req 68452352->68456447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.093164:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760797.093170:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c732e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68452352->68456447] (req 68452352->68456447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.093183:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c732e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68452352->68456447] (req 68452352->68456447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.093201:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c732e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68452352->68456447] (req 68452352->68456447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.093208:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca03f093/0x1f99687d854c732e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68452352->68456447] (req 68452352->68456447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.093370:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554907136:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9116us (9386us total) trans 4294984021 rc 0/0 00000100:00100000:1.0:1789760797.093422:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16723 00000100:00100000:1.0:1789760797.093432:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740317099247 time 740329856209 state 0x2 ref 0 00000100:00100000:0.0:1789760797.100061:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760797.100158:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554907264 00000100:00100000:2.0:1789760797.100168:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554907264 00000100:00100000:2.0:1789760797.100227:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740317099247 time 740329856209 state 0x0 ref 1 00000100:00100000:2.0:1789760797.100242:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740330116625 time 740329856209 state 0x1 ref 1 00000100:00100000:2.0:1789760797.100271:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740330116625 time 740345889403 state 0x0 ref 1 00000100:00100000:2.0:1789760797.100348:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554907264:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760797.100501:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68456448, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20a60 00010000:00010000:2.0:1789760797.100511:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7335 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.100528:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760797.100532:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.100538:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7335 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68456448->68460543] (req 68456448->68460543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.100551:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7335 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68456448->68460543] (req 68456448->68460543) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.111356:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984022 is committed 00002000:00010000:2.0:1789760797.111654:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68456448 -> 68460544 00002000:00010000:2.0:1789760797.111660:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133728 -> 133736 00010000:00010000:2.0:1789760797.111672:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7335 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68456448->68460543] (req 68456448->68460543) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.111685:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7335 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68456448->68460543] (req 68456448->68460543) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.111694:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7335 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68456448->68460543] (req 68456448->68460543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.111701:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.111706:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7335 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68456448->68460543] (req 68456448->68460543) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.111714:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7335 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68456448->68460543] (req 68456448->68460543) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.111734:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7335 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68456448->68460543] (req 68456448->68460543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.111742:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c7335 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68456448->68460543] (req 68456448->68460543) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.111878:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554907264:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11590us (11821us total) trans 4294984022 rc 0/0 00000100:00100000:2.0:1789760797.111889:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16724 00000100:00100000:2.0:1789760797.111899:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740330116625 time 740345889403 state 0x2 ref 0 00000100:00100000:2.0:1789760797.115748:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760797.115851:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554907392 00000100:00100000:0.0:1789760797.115862:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554907392 00000100:00100000:0.0:1789760797.115919:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740330116625 time 740345889403 state 0x0 ref 1 00000100:00100000:0.0:1789760797.115935:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740346149819 time 740345889403 state 0x1 ref 1 00000100:00100000:0.0:1789760797.116075:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740346149819 time 740361692896 state 0x0 ref 1 00000100:00100000:0.0:1789760797.116089:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554907392:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760797.116139:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68460544, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20a68 00010000:00010000:0.0:1789760797.116148:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c733c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.116164:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760797.116169:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.116175:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c733c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68460544->68464639] (req 68460544->68464639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.116190:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c733c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68460544->68464639] (req 68460544->68464639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.122373:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984023 is committed 00002000:00010000:0.0:1789760797.122512:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68460544 -> 68464640 00002000:00010000:0.0:1789760797.122518:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133736 -> 133744 00010000:00010000:0.0:1789760797.122533:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c733c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68460544->68464639] (req 68460544->68464639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.122546:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c733c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68460544->68464639] (req 68460544->68464639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.122556:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c733c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68460544->68464639] (req 68460544->68464639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.122564:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760797.122581:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c733c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68460544->68464639] (req 68460544->68464639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.122591:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c733c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68460544->68464639] (req 68460544->68464639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.122610:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c733c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68460544->68464639] (req 68460544->68464639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.122618:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c733c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68460544->68464639] (req 68460544->68464639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.122720:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554907392:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6629us (6978us total) trans 4294984023 rc 0/0 00000100:00100000:0.0:1789760797.122731:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16725 00000100:00100000:0.0:1789760797.122740:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740346149819 time 740361692896 state 0x2 ref 0 00000100:00100000:0.0:1789760797.134577:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760797.134662:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554907520 00000100:00100000:3.0:1789760797.134672:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554907520 00000100:00100000:3.0:1789760797.134725:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740346149819 time 740361692896 state 0x0 ref 1 00000100:00100000:3.0:1789760797.134781:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740361953312 time 740361692896 state 0x1 ref 1 00000100:00100000:3.0:1789760797.134817:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740361953312 time 740380435648 state 0x0 ref 1 00000100:00100000:3.0:1789760797.134829:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e56d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554907520:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760797.134880:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68464640, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20a70 00010000:00010000:3.0:1789760797.134890:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7343 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.134905:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760797.134909:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.134915:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7343 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68464640->68468735] (req 68464640->68468735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.134929:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7343 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68464640->68468735] (req 68464640->68468735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.141581:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984024 is committed 00002000:00010000:3.0:1789760797.141757:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68464640 -> 68468736 00002000:00010000:3.0:1789760797.141762:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133744 -> 133752 00010000:00010000:3.0:1789760797.141776:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7343 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68464640->68468735] (req 68464640->68468735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.141789:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7343 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68464640->68468735] (req 68464640->68468735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.141797:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7343 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68464640->68468735] (req 68464640->68468735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.141805:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760797.141810:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7343 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68464640->68468735] (req 68464640->68468735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.141819:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7343 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68464640->68468735] (req 68464640->68468735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.141839:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7343 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68464640->68468735] (req 68464640->68468735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.141847:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7343 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68464640->68468735] (req 68464640->68468735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.141998:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e56d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554907520:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7165us (7433us total) trans 4294984024 rc 0/0 00000100:00100000:3.0:1789760797.142012:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16726 00000100:00100000:3.0:1789760797.142022:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740361953312 time 740380435648 state 0x2 ref 0 00000100:00100000:1.0:1789760797.147727:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760797.148455:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554907648 00000100:00100000:0.0:1789760797.148465:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554907648 00000100:00100000:0.0:1789760797.148518:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740361953312 time 740380435648 state 0x0 ref 1 00000100:00100000:0.0:1789760797.148533:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740380696064 time 740380435648 state 0x1 ref 1 00000100:00100000:0.0:1789760797.148576:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740380696064 time 740394186468 state 0x0 ref 1 00000100:00100000:0.0:1789760797.148589:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06400000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554907648:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760797.148644:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68468736, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20a78 00010000:00010000:0.0:1789760797.148653:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c734a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.148669:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760797.148673:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.148679:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c734a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68468736->68472831] (req 68468736->68472831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.148693:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c734a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68468736->68472831] (req 68468736->68472831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.154926:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984025 is committed 00002000:00010000:1.0:1789760797.155042:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68468736 -> 68472832 00002000:00010000:1.0:1789760797.155048:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133752 -> 133760 00010000:00010000:1.0:1789760797.155061:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c734a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68468736->68472831] (req 68468736->68472831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.155073:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c734a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68468736->68472831] (req 68468736->68472831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.155082:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c734a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68468736->68472831] (req 68468736->68472831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.155095:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760797.155101:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c734a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68468736->68472831] (req 68468736->68472831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.155110:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c734a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68468736->68472831] (req 68468736->68472831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.155129:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c734a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68468736->68472831] (req 68468736->68472831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.155137:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7ec1f7b/0x1f99687d854c734a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68468736->68472831] (req 68468736->68472831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.155272:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06400000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554907648:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6680us (7548us total) trans 4294984025 rc 0/0 00000100:00100000:1.0:1789760797.155282:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16727 00000100:00100000:1.0:1789760797.155292:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740380696064 time 740394186468 state 0x2 ref 0 00000100:00100000:2.0:1789760797.158918:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760797.159292:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554907776 00000100:00100000:0.0:1789760797.159301:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554907776 00000100:00100000:0.0:1789760797.159431:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740380696064 time 740394186468 state 0x0 ref 1 00000100:00100000:0.0:1789760797.159448:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740394446884 time 740394186468 state 0x1 ref 1 00000100:00100000:0.0:1789760797.159520:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740394446884 time 740405137248 state 0x0 ref 1 00000100:00100000:0.0:1789760797.159533:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554907776:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760797.159590:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68472832, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20a80 00010000:00010000:0.0:1789760797.159599:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c7351 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.159614:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760797.159619:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.159625:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c7351 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68472832->68476927] (req 68472832->68476927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.159637:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c7351 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68472832->68476927] (req 68472832->68476927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.164126:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984026 is committed 00002000:00010000:0.0:1789760797.164315:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68472832 -> 68476928 00002000:00010000:0.0:1789760797.164320:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133760 -> 133768 00010000:00010000:0.0:1789760797.164333:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c7351 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68472832->68476927] (req 68472832->68476927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.164345:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c7351 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68472832->68476927] (req 68472832->68476927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.164354:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c7351 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68472832->68476927] (req 68472832->68476927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.164362:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760797.164367:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c7351 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68472832->68476927] (req 68472832->68476927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.164395:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c7351 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68472832->68476927] (req 68472832->68476927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.164481:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c7351 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68472832->68476927] (req 68472832->68476927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.164491:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c7351 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68472832->68476927] (req 68472832->68476927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.164786:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554907776:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5248us (5870us total) trans 4294984026 rc 0/0 00000100:00100000:0.0:1789760797.164797:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16728 00000100:00100000:0.0:1789760797.164807:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740394446884 time 740405137248 state 0x2 ref 0 00000100:00100000:1.0:1789760797.168527:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760797.168611:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554907904 00000100:00100000:2.0:1789760797.168681:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554907904 00000100:00100000:2.0:1789760797.168738:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740394446884 time 740405137248 state 0x0 ref 1 00000100:00100000:2.0:1789760797.168753:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740405397664 time 740405137248 state 0x1 ref 1 00000100:00100000:2.0:1789760797.168809:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740405397664 time 740414427367 state 0x0 ref 1 00000100:00100000:2.0:1789760797.168821:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06403100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554907904:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760797.168865:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68476928, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20a88 00010000:00010000:2.0:1789760797.168874:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7358 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.168890:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760797.168894:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.168900:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7358 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68476928->68481023] (req 68476928->68481023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.168913:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7358 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68476928->68481023] (req 68476928->68481023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.174816:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984027 is committed 00002000:00010000:2.0:1789760797.174917:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68476928 -> 68481024 00002000:00010000:2.0:1789760797.174922:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133768 -> 133776 00010000:00010000:2.0:1789760797.174937:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7358 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68476928->68481023] (req 68476928->68481023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.174948:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7358 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68476928->68481023] (req 68476928->68481023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.174958:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7358 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68476928->68481023] (req 68476928->68481023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.174965:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.174969:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7358 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68476928->68481023] (req 68476928->68481023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.174977:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7358 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68476928->68481023] (req 68476928->68481023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.175000:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7358 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68476928->68481023] (req 68476928->68481023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.175008:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7358 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68476928->68481023] (req 68476928->68481023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.175113:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06403100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554907904:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6281us (6581us total) trans 4294984027 rc 0/0 00000100:00100000:2.0:1789760797.175127:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16729 00000100:00100000:2.0:1789760797.175136:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740405397664 time 740414427367 state 0x2 ref 0 00000100:00100000:1.0:1789760797.178582:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760797.179134:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554908032 00000100:00100000:3.0:1789760797.179144:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554908032 00000100:00100000:3.0:1789760797.179203:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740405397664 time 740414427367 state 0x0 ref 1 00000100:00100000:3.0:1789760797.179221:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740414687783 time 740414427367 state 0x1 ref 1 00000100:00100000:3.0:1789760797.179257:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740414687783 time 740424874866 state 0x0 ref 1 00000100:00100000:3.0:1789760797.179272:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06400a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554908032:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760797.179329:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68481024, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20a90 00010000:00010000:3.0:1789760797.179339:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c735f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.179357:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760797.179362:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.179369:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c735f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68481024->68485119] (req 68481024->68485119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.179420:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c735f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68481024->68485119] (req 68481024->68485119) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.184683:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984028 is committed 00002000:00010000:3.0:1789760797.184953:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68481024 -> 68485120 00002000:00010000:3.0:1789760797.184958:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133776 -> 133784 00010000:00010000:3.0:1789760797.184971:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c735f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68481024->68485119] (req 68481024->68485119) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.184984:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c735f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68481024->68485119] (req 68481024->68485119) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.184993:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c735f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68481024->68485119] (req 68481024->68485119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.185002:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760797.185006:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c735f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68481024->68485119] (req 68481024->68485119) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.185016:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c735f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68481024->68485119] (req 68481024->68485119) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.185035:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c735f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68481024->68485119] (req 68481024->68485119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.185043:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c735f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68481024->68485119] (req 68481024->68485119) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.185148:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06400a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554908032:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5869us (6575us total) trans 4294984028 rc 0/0 00000100:00100000:3.0:1789760797.185159:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16730 00000100:00100000:3.0:1789760797.185169:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740414687783 time 740424874866 state 0x2 ref 0 00000100:00100000:1.0:1789760797.191136:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760797.191351:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554908160 00000100:00100000:2.0:1789760797.191361:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554908160 00000100:00100000:2.0:1789760797.191452:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740414687783 time 740424874866 state 0x0 ref 1 00000100:00100000:2.0:1789760797.191467:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740425135282 time 740424874866 state 0x1 ref 1 00000100:00100000:2.0:1789760797.191490:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740425135282 time 740437109196 state 0x0 ref 1 00000100:00100000:2.0:1789760797.191503:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06402680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554908160:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760797.191551:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68485120, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20a98 00010000:00010000:2.0:1789760797.191560:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7366 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.191587:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760797.191593:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.191600:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7366 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68485120->68489215] (req 68485120->68489215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.191613:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7366 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68485120->68489215] (req 68485120->68489215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.196059:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984029 is committed 00002000:00010000:2.0:1789760797.196773:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68485120 -> 68489216 00002000:00010000:2.0:1789760797.196779:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133784 -> 133792 00010000:00010000:2.0:1789760797.196793:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7366 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68485120->68489215] (req 68485120->68489215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.196806:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7366 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68485120->68489215] (req 68485120->68489215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.196815:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7366 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68485120->68489215] (req 68485120->68489215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.196823:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.196827:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7366 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68485120->68489215] (req 68485120->68489215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.196836:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7366 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68485120->68489215] (req 68485120->68489215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.196856:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7366 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68485120->68489215] (req 68485120->68489215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.196869:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7366 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68485120->68489215] (req 68485120->68489215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.196976:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06402680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554908160:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5472us (5846us total) trans 4294984029 rc 0/0 00000100:00100000:2.0:1789760797.196987:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16731 00000100:00100000:2.0:1789760797.196996:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740425135282 time 740437109196 state 0x2 ref 0 00000100:00100000:1.0:1789760797.208159:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760797.208424:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554908288 00000100:00100000:0.0:1789760797.208435:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554908288 00000100:00100000:0.0:1789760797.208490:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740425135282 time 740437109196 state 0x0 ref 1 00000100:00100000:0.0:1789760797.208505:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740437369612 time 740437109196 state 0x1 ref 1 00000100:00100000:0.0:1789760797.208540:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740437369612 time 740454158348 state 0x0 ref 1 00000100:00100000:0.0:1789760797.208552:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06401500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554908288:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760797.208611:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68489216, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20aa0 00010000:00010000:0.0:1789760797.208621:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c736d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.208636:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760797.208641:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.208647:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c736d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68489216->68493311] (req 68489216->68493311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.208661:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c736d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68489216->68493311] (req 68489216->68493311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.215498:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984030 is committed 00002000:00010000:2.0:1789760797.215673:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68489216 -> 68493312 00002000:00010000:2.0:1789760797.215678:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133792 -> 133800 00010000:00010000:2.0:1789760797.215691:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c736d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68489216->68493311] (req 68489216->68493311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.215702:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c736d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68489216->68493311] (req 68489216->68493311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.215711:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c736d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68489216->68493311] (req 68489216->68493311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.215717:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.215722:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c736d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68489216->68493311] (req 68489216->68493311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.215730:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c736d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68489216->68493311] (req 68489216->68493311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.215755:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c736d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68489216->68493311] (req 68489216->68493311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.215763:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c736d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68489216->68493311] (req 68489216->68493311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.215856:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06401500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554908288:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7302us (7703us total) trans 4294984030 rc 0/0 00000100:00100000:2.0:1789760797.215865:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16732 00000100:00100000:2.0:1789760797.215874:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740437369612 time 740454158348 state 0x2 ref 0 00000100:00100000:1.0:1789760797.224015:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760797.224280:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554908416 00000100:00100000:0.0:1789760797.224289:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554908416 00000100:00100000:0.0:1789760797.224342:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740437369612 time 740454158348 state 0x0 ref 1 00000100:00100000:0.0:1789760797.224359:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740454418764 time 740454158348 state 0x1 ref 1 00000100:00100000:0.0:1789760797.224425:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740454418764 time 740470043297 state 0x0 ref 1 00000100:00100000:0.0:1789760797.224439:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb06401880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554908416:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760797.224486:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68493312, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20aa8 00010000:00010000:0.0:1789760797.224496:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c7374 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.224512:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760797.224516:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.224522:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c7374 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68493312->68497407] (req 68493312->68497407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.224534:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c7374 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68493312->68497407] (req 68493312->68497407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.233369:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984031 is committed 00002000:00010000:2.0:1789760797.233532:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68493312 -> 68497408 00002000:00010000:2.0:1789760797.233537:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133800 -> 133808 00010000:00010000:2.0:1789760797.233550:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c7374 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68493312->68497407] (req 68493312->68497407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.233563:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c7374 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68493312->68497407] (req 68493312->68497407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.233583:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c7374 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68493312->68497407] (req 68493312->68497407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.233590:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.233595:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c7374 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68493312->68497407] (req 68493312->68497407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.233618:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c7374 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68493312->68497407] (req 68493312->68497407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.233639:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c7374 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68493312->68497407] (req 68493312->68497407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.233646:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c7374 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68493312->68497407] (req 68493312->68497407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.233728:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb06401880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554908416:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9287us (9719us total) trans 4294984031 rc 0/0 00000100:00100000:2.0:1789760797.233738:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16733 00000100:00100000:2.0:1789760797.233746:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740454418764 time 740470043297 state 0x2 ref 0 00000100:00100000:1.0:1789760797.240065:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760797.240196:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554908544 00000100:00100000:3.0:1789760797.240205:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554908544 00000100:00100000:3.0:1789760797.240259:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740454418764 time 740470043297 state 0x0 ref 1 00000100:00100000:3.0:1789760797.240274:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740470303713 time 740470043297 state 0x1 ref 1 00000100:00100000:3.0:1789760797.240306:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740470303713 time 740485924029 state 0x0 ref 1 00000100:00100000:3.0:1789760797.240317:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554908544:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760797.240365:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68497408, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20ab0 00010000:00010000:3.0:1789760797.240375:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c737b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.240430:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760797.240435:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.240442:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c737b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68497408->68501503] (req 68497408->68501503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.240455:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c737b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68497408->68501503] (req 68497408->68501503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.248072:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984032 is committed 00002000:00010000:3.0:1789760797.248243:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68497408 -> 68501504 00002000:00010000:3.0:1789760797.248248:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133808 -> 133816 00010000:00010000:3.0:1789760797.248261:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c737b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68497408->68501503] (req 68497408->68501503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.248273:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c737b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68497408->68501503] (req 68497408->68501503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.248283:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c737b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68497408->68501503] (req 68497408->68501503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.248290:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760797.248294:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c737b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68497408->68501503] (req 68497408->68501503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.248303:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c737b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68497408->68501503] (req 68497408->68501503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.248323:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c737b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68497408->68501503] (req 68497408->68501503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.248331:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c737b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68497408->68501503] (req 68497408->68501503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.248466:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554908544:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8145us (8414us total) trans 4294984032 rc 0/0 00000100:00100000:3.0:1789760797.248477:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16734 00000100:00100000:3.0:1789760797.248486:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740470303713 time 740485924029 state 0x2 ref 0 00000100:00100000:1.0:1789760797.254993:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760797.255093:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554908672 00000100:00100000:2.0:1789760797.255103:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554908672 00000100:00100000:2.0:1789760797.255155:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740470303713 time 740485924029 state 0x0 ref 1 00000100:00100000:2.0:1789760797.255170:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740486184445 time 740485924029 state 0x1 ref 1 00000100:00100000:2.0:1789760797.255206:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740486184445 time 740500825141 state 0x0 ref 1 00000100:00100000:2.0:1789760797.255219:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554908672:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760797.255265:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68501504, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20ab8 00010000:00010000:2.0:1789760797.255275:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7382 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.255291:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760797.255295:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.255301:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7382 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68501504->68505599] (req 68501504->68505599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.255315:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7382 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68501504->68505599] (req 68501504->68505599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.264423:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984033 is committed 00010000:00010000:3.0:1789760797.265763:0:6509:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789760797.274346:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68501504 -> 68505600 00002000:00010000:2.0:1789760797.274356:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133816 -> 133824 00010000:00010000:2.0:1789760797.274369:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7382 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68501504->68505599] (req 68501504->68505599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.274410:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7382 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68501504->68505599] (req 68501504->68505599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.274418:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7382 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68501504->68505599] (req 68501504->68505599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.274425:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.274429:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7382 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68501504->68505599] (req 68501504->68505599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.274439:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7382 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68501504->68505599] (req 68501504->68505599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.274464:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7382 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68501504->68505599] (req 68501504->68505599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.274473:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c7382 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68501504->68505599] (req 68501504->68505599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.274576:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554908672:12345-192.168.203.22@tcp:4:dd.0 Request processed in 19346us (19578us total) trans 4294984033 rc 0/0 00000100:00100000:2.0:1789760797.274586:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16735 00000100:00100000:2.0:1789760797.274596:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740486184445 time 740500825141 state 0x2 ref 0 00000100:00100000:1.0:1789760797.279691:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760797.279795:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554908800 00000100:00100000:2.0:1789760797.279804:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554908800 00000100:00100000:2.0:1789760797.279855:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740486184445 time 740500825141 state 0x0 ref 1 00000100:00100000:2.0:1789760797.279869:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740501085557 time 740500825141 state 0x1 ref 1 00000100:00100000:2.0:1789760797.279898:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740501085557 time 740525516713 state 0x0 ref 1 00000100:00100000:2.0:1789760797.279910:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554908800:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760797.279954:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68505600, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20ac0 00010000:00010000:2.0:1789760797.279964:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7389 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.279979:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760797.279985:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.279991:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7389 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68505600->68509695] (req 68505600->68509695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.280005:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7389 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68505600->68509695] (req 68505600->68509695) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.287927:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984034 is committed 00002000:00010000:2.0:1789760797.288049:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68505600 -> 68509696 00002000:00010000:2.0:1789760797.288055:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133824 -> 133832 00010000:00010000:2.0:1789760797.288069:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7389 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68505600->68509695] (req 68505600->68509695) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.288081:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7389 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68505600->68509695] (req 68505600->68509695) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.288089:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7389 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68505600->68509695] (req 68505600->68509695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.288096:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.288102:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7389 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68505600->68509695] (req 68505600->68509695) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.288112:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7389 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68505600->68509695] (req 68505600->68509695) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.288130:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7389 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68505600->68509695] (req 68505600->68509695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.288144:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7389 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68505600->68509695] (req 68505600->68509695) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.288234:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554908800:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8322us (8547us total) trans 4294984034 rc 0/0 00000100:00100000:2.0:1789760797.288244:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16736 00000100:00100000:2.0:1789760797.288253:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740501085557 time 740525516713 state 0x2 ref 0 00000100:00100000:1.0:1789760797.292079:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760797.292172:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554908928 00000100:00100000:0.0:1789760797.292182:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554908928 00000100:00100000:0.0:1789760797.292235:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740501085557 time 740525516713 state 0x0 ref 1 00000100:00100000:0.0:1789760797.292250:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740525777129 time 740525516713 state 0x1 ref 1 00000100:00100000:0.0:1789760797.292289:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740525777129 time 740537907091 state 0x0 ref 1 00000100:00100000:0.0:1789760797.292300:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e92a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554908928:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760797.292348:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68509696, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20ac8 00010000:00010000:0.0:1789760797.292358:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7390 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.292416:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760797.292421:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.292428:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7390 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68509696->68513791] (req 68509696->68513791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.292442:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7390 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68509696->68513791] (req 68509696->68513791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.297931:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984035 is committed 00002000:00010000:0.0:1789760797.298050:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68509696 -> 68513792 00002000:00010000:0.0:1789760797.298056:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133832 -> 133840 00010000:00010000:0.0:1789760797.298069:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7390 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68509696->68513791] (req 68509696->68513791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.298081:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7390 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68509696->68513791] (req 68509696->68513791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.298091:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7390 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68509696->68513791] (req 68509696->68513791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.298098:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760797.298103:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7390 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68509696->68513791] (req 68509696->68513791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.298112:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7390 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68509696->68513791] (req 68509696->68513791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.298137:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7390 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68509696->68513791] (req 68509696->68513791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.298144:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7390 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68509696->68513791] (req 68509696->68513791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.298230:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e92a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554908928:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5927us (6154us total) trans 4294984035 rc 0/0 00000100:00100000:0.0:1789760797.298240:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16737 00000100:00100000:0.0:1789760797.298249:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740525777129 time 740537907091 state 0x2 ref 0 00000100:00100000:1.0:1789760797.301615:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760797.302144:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554909056 00000100:00100000:3.0:1789760797.302154:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554909056 00000100:00100000:3.0:1789760797.302207:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740525777129 time 740537907091 state 0x0 ref 1 00000100:00100000:3.0:1789760797.302222:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740538167507 time 740537907091 state 0x1 ref 1 00000100:00100000:3.0:1789760797.302302:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740538167507 time 740547919760 state 0x0 ref 1 00000100:00100000:3.0:1789760797.302317:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554909056:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760797.302364:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68513792, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20ad0 00010000:00010000:3.0:1789760797.302373:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7397 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.302423:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760797.302428:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.302434:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7397 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68513792->68517887] (req 68513792->68517887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.302448:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7397 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68513792->68517887] (req 68513792->68517887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.310460:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984036 is committed 00002000:00010000:3.0:1789760797.310885:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68513792 -> 68517888 00002000:00010000:3.0:1789760797.310891:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133840 -> 133848 00010000:00010000:3.0:1789760797.310903:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7397 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68513792->68517887] (req 68513792->68517887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.310916:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7397 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68513792->68517887] (req 68513792->68517887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.310924:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7397 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68513792->68517887] (req 68513792->68517887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.310931:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760797.310936:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7397 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68513792->68517887] (req 68513792->68517887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.311003:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7397 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68513792->68517887] (req 68513792->68517887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.311023:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7397 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68513792->68517887] (req 68513792->68517887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.311031:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7397 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68513792->68517887] (req 68513792->68517887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.311162:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554909056:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8842us (9550us total) trans 4294984036 rc 0/0 00000100:00100000:3.0:1789760797.311172:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16738 00000100:00100000:3.0:1789760797.311182:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740538167507 time 740547919760 state 0x2 ref 0 00000100:00100000:1.0:1789760797.316305:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760797.316489:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554909184 00000100:00100000:2.0:1789760797.316498:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554909184 00000100:00100000:2.0:1789760797.316549:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740538167507 time 740547919760 state 0x0 ref 1 00000100:00100000:2.0:1789760797.316563:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740548180176 time 740547919760 state 0x1 ref 1 00000100:00100000:2.0:1789760797.316605:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740548180176 time 740562222956 state 0x0 ref 1 00000100:00100000:2.0:1789760797.316616:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e93800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554909184:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760797.316663:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68517888, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20ad8 00010000:00010000:2.0:1789760797.316671:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c739e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.316685:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760797.316689:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.316695:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c739e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68517888->68521983] (req 68517888->68521983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.316711:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c739e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68517888->68521983] (req 68517888->68521983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.327600:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984037 is committed 00002000:00010000:2.0:1789760797.327771:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68517888 -> 68521984 00002000:00010000:2.0:1789760797.327776:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133848 -> 133856 00010000:00010000:2.0:1789760797.327788:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c739e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68517888->68521983] (req 68517888->68521983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.327801:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c739e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68517888->68521983] (req 68517888->68521983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.327811:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c739e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68517888->68521983] (req 68517888->68521983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.327818:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.327822:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c739e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68517888->68521983] (req 68517888->68521983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.327831:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c739e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68517888->68521983] (req 68517888->68521983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.327850:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c739e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68517888->68521983] (req 68517888->68521983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.327858:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c739e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68517888->68521983] (req 68517888->68521983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.327955:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e93800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554909184:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11329us (11648us total) trans 4294984037 rc 0/0 00000100:00100000:2.0:1789760797.327966:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16739 00000100:00100000:2.0:1789760797.327975:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740548180176 time 740562222956 state 0x2 ref 0 00000100:00100000:1.0:1789760797.335345:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760797.335492:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554909312 00000100:00100000:2.0:1789760797.335503:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554909312 00000100:00100000:2.0:1789760797.335556:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740548180176 time 740562222956 state 0x0 ref 1 00000100:00100000:2.0:1789760797.335581:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740562483372 time 740562222956 state 0x1 ref 1 00000100:00100000:2.0:1789760797.335622:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740562483372 time 740581240140 state 0x0 ref 1 00000100:00100000:2.0:1789760797.335634:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a387b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554909312:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760797.335682:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68521984, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20ae0 00010000:00010000:2.0:1789760797.335692:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73a5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.335707:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760797.335712:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.335718:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68521984->68526079] (req 68521984->68526079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.335732:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68521984->68526079] (req 68521984->68526079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.341916:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984038 is committed 00002000:00010000:2.0:1789760797.342049:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68521984 -> 68526080 00002000:00010000:2.0:1789760797.342054:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133856 -> 133864 00010000:00010000:2.0:1789760797.342067:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73a5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68521984->68526079] (req 68521984->68526079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.342080:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73a5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68521984->68526079] (req 68521984->68526079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.342089:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68521984->68526079] (req 68521984->68526079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.342097:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.342102:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73a5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68521984->68526079] (req 68521984->68526079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.342112:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73a5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68521984->68526079] (req 68521984->68526079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.342131:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73a5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68521984->68526079] (req 68521984->68526079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.342139:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73a5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68521984->68526079] (req 68521984->68526079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.342234:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a387b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554909312:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6597us (6903us total) trans 4294984038 rc 0/0 00000100:00100000:2.0:1789760797.342245:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16740 00000100:00100000:2.0:1789760797.342255:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740562483372 time 740581240140 state 0x2 ref 0 00000100:00100000:1.0:1789760797.349294:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760797.351532:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554909440 00000100:00100000:0.0:1789760797.351546:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554909440 00000100:00100000:0.0:1789760797.351622:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740562483372 time 740581240140 state 0x0 ref 1 00000100:00100000:0.0:1789760797.351639:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740581500556 time 740581240140 state 0x1 ref 1 00000100:00100000:0.0:1789760797.351718:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740581500556 time 740597334850 state 0x0 ref 1 00000100:00100000:0.0:1789760797.351732:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a384a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554909440:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760797.351782:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68526080, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20ae8 00010000:00010000:0.0:1789760797.351791:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c73ac lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.351807:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760797.351812:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.351818:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c73ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68526080->68530175] (req 68526080->68530175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.351832:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c73ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68526080->68530175] (req 68526080->68530175) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.356760:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984039 is committed 00002000:00010000:0.0:1789760797.356883:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68526080 -> 68530176 00002000:00010000:0.0:1789760797.356889:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133864 -> 133872 00010000:00010000:0.0:1789760797.356902:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c73ac lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68526080->68530175] (req 68526080->68530175) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.356916:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c73ac lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68526080->68530175] (req 68526080->68530175) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.356925:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c73ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68526080->68530175] (req 68526080->68530175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.356933:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760797.356938:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c73ac lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68526080->68530175] (req 68526080->68530175) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.356957:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c73ac lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68526080->68530175] (req 68526080->68530175) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.356977:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c73ac lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68526080->68530175] (req 68526080->68530175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.356991:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c73ac lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68526080->68530175] (req 68526080->68530175) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.357090:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a384a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554909440:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5356us (7801us total) trans 4294984039 rc 0/0 00000100:00100000:0.0:1789760797.357100:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16741 00000100:00100000:0.0:1789760797.357109:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740581500556 time 740597334850 state 0x2 ref 0 00000100:00100000:0.0:1789760797.367704:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760797.370270:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554909568 00000100:00100000:3.0:1789760797.370284:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554909568 00000100:00100000:3.0:1789760797.370363:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740581500556 time 740597334850 state 0x0 ref 1 00000100:00100000:3.0:1789760797.370422:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740597595266 time 740597334850 state 0x1 ref 1 00000100:00100000:3.0:1789760797.370460:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740597595266 time 740616078593 state 0x0 ref 1 00000100:00100000:3.0:1789760797.370474:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e54000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554909568:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760797.370520:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68530176, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20af0 00010000:00010000:3.0:1789760797.370530:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c73b3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.370545:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760797.370550:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.370556:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c73b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68530176->68534271] (req 68530176->68534271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.370580:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c73b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68530176->68534271] (req 68530176->68534271) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.376591:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984040 is committed 00002000:00010000:3.0:1789760797.377177:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68530176 -> 68534272 00002000:00010000:3.0:1789760797.377185:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133872 -> 133880 00010000:00010000:3.0:1789760797.377199:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c73b3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68530176->68534271] (req 68530176->68534271) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.377212:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c73b3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68530176->68534271] (req 68530176->68534271) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.377222:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c73b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68530176->68534271] (req 68530176->68534271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.377236:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760797.377241:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c73b3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68530176->68534271] (req 68530176->68534271) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.377251:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c73b3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68530176->68534271] (req 68530176->68534271) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.377271:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c73b3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68530176->68534271] (req 68530176->68534271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.377279:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c73b3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68530176->68534271] (req 68530176->68534271) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.377670:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554909568:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7193us (9967us total) trans 4294984040 rc 0/0 00000100:00100000:2.0:1789760797.377682:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16742 00000100:00100000:2.0:1789760797.377691:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740597595266 time 740616078593 state 0x2 ref 0 00000100:00100000:0.0:1789760797.383106:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760797.383278:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554909696 00000100:00100000:2.0:1789760797.383288:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554909696 00000100:00100000:2.0:1789760797.383446:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740597595266 time 740616078593 state 0x0 ref 1 00000100:00100000:2.0:1789760797.383462:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740616339009 time 740616078593 state 0x1 ref 1 00000100:00100000:2.0:1789760797.383529:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740616339009 time 740629147077 state 0x0 ref 1 00000100:00100000:2.0:1789760797.383544:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e56300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554909696:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760797.383599:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68534272, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20af8 00010000:00010000:2.0:1789760797.383609:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73ba lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.383629:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760797.383634:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.383639:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68534272->68538367] (req 68534272->68538367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.383659:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68534272->68538367] (req 68534272->68538367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.390278:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984041 is committed 00002000:00010000:2.0:1789760797.390617:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68534272 -> 68538368 00002000:00010000:2.0:1789760797.390674:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133880 -> 133888 00010000:00010000:2.0:1789760797.390689:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73ba lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68534272->68538367] (req 68534272->68538367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.390702:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73ba lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68534272->68538367] (req 68534272->68538367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.390712:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68534272->68538367] (req 68534272->68538367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.390721:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.390726:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73ba lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68534272->68538367] (req 68534272->68538367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.390737:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73ba lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68534272->68538367] (req 68534272->68538367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.390758:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73ba lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68534272->68538367] (req 68534272->68538367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.390766:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73ba lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68534272->68538367] (req 68534272->68538367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.391015:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e56300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554909696:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7469us (7912us total) trans 4294984041 rc 0/0 00000100:00100000:2.0:1789760797.391027:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16743 00000100:00100000:2.0:1789760797.391037:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740616339009 time 740629147077 state 0x2 ref 0 00000100:00100000:0.0:1789760797.395255:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760797.395990:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554909824 00000100:00100000:2.0:1789760797.396001:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554909824 00000100:00100000:2.0:1789760797.396222:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740616339009 time 740629147077 state 0x0 ref 1 00000100:00100000:2.0:1789760797.396240:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740629407493 time 740629147077 state 0x1 ref 1 00000100:00100000:2.0:1789760797.396449:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740629407493 time 740642066219 state 0x0 ref 1 00000100:00100000:2.0:1789760797.396473:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554909824:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760797.396521:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68538368, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20b00 00010000:00010000:2.0:1789760797.396530:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73c1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.396545:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760797.396548:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.396554:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68538368->68542463] (req 68538368->68542463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.396576:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68538368->68542463] (req 68538368->68542463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.402413:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984042 is committed 00002000:00010000:2.0:1789760797.402581:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68538368 -> 68542464 00002000:00010000:2.0:1789760797.402585:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133888 -> 133896 00010000:00010000:2.0:1789760797.402597:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73c1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68538368->68542463] (req 68538368->68542463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.402609:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73c1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68538368->68542463] (req 68538368->68542463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.402625:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68538368->68542463] (req 68538368->68542463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.402633:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.402638:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73c1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68538368->68542463] (req 68538368->68542463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.402648:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73c1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68538368->68542463] (req 68538368->68542463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.402667:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73c1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68538368->68542463] (req 68538368->68542463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.402676:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73c1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68538368->68542463] (req 68538368->68542463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.402843:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554909824:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6375us (7590us total) trans 4294984042 rc 0/0 00000100:00100000:2.0:1789760797.402857:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16744 00000100:00100000:2.0:1789760797.402868:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740629407493 time 740642066219 state 0x2 ref 0 00000100:00100000:0.0:1789760797.408534:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760797.408659:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554909952 00000100:00100000:2.0:1789760797.408668:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554909952 00000100:00100000:2.0:1789760797.408716:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740629407493 time 740642066219 state 0x0 ref 1 00000100:00100000:2.0:1789760797.408731:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740642326635 time 740642066219 state 0x1 ref 1 00000100:00100000:2.0:1789760797.408765:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740642326635 time 740654384347 state 0x0 ref 1 00000100:00100000:2.0:1789760797.408778:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e56a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554909952:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760797.408820:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68542464, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20b08 00010000:00010000:2.0:1789760797.408830:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73c8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.408846:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760797.408850:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.408856:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68542464->68546559] (req 68542464->68546559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.408870:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68542464->68546559] (req 68542464->68546559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760797.415040:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984043 is committed 00002000:00010000:2.0:1789760797.417441:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68542464 -> 68546560 00002000:00010000:2.0:1789760797.417450:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133896 -> 133904 00010000:00010000:2.0:1789760797.417465:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73c8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68542464->68546559] (req 68542464->68546559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.417479:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73c8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68542464->68546559] (req 68542464->68546559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.417487:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68542464->68546559] (req 68542464->68546559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.417495:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.417500:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73c8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68542464->68546559] (req 68542464->68546559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.417508:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73c8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68542464->68546559] (req 68542464->68546559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.417528:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73c8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68542464->68546559] (req 68542464->68546559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.417536:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73c8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68542464->68546559] (req 68542464->68546559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.417699:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e56a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554909952:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8918us (9168us total) trans 4294984043 rc 0/0 00000100:00100000:2.0:1789760797.417710:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16745 00000100:00100000:2.0:1789760797.417719:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740642326635 time 740654384347 state 0x2 ref 0 00000100:00100000:1.0:1789760797.421837:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760797.422547:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554910080 00000100:00100000:2.0:1789760797.422558:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554910080 00000100:00100000:2.0:1789760797.422621:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740642326635 time 740654384347 state 0x0 ref 1 00000100:00100000:2.0:1789760797.422633:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740654644763 time 740654384347 state 0x1 ref 1 00000100:00100000:2.0:1789760797.422662:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740654644763 time 740668280995 state 0x0 ref 1 00000100:00100000:2.0:1789760797.422675:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a386a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554910080:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760797.422719:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68546560, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20b10 00010000:00010000:2.0:1789760797.422727:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73cf lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.422739:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760797.422742:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.422747:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68546560->68550655] (req 68546560->68550655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.422761:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68546560->68550655] (req 68546560->68550655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.429179:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984044 is committed 00002000:00010000:2.0:1789760797.429599:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68546560 -> 68550656 00002000:00010000:2.0:1789760797.429605:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133904 -> 133912 00010000:00010000:2.0:1789760797.429617:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73cf lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68546560->68550655] (req 68546560->68550655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.429635:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73cf lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68546560->68550655] (req 68546560->68550655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.429644:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68546560->68550655] (req 68546560->68550655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.429652:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.429656:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73cf lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68546560->68550655] (req 68546560->68550655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.429666:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73cf lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68546560->68550655] (req 68546560->68550655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.429685:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73cf lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68546560->68550655] (req 68546560->68550655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.429694:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73cf lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68546560->68550655] (req 68546560->68550655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.429788:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a386a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554910080:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7111us (7956us total) trans 4294984044 rc 0/0 00000100:00100000:2.0:1789760797.429800:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16746 00000100:00100000:2.0:1789760797.429810:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740654644763 time 740668280995 state 0x2 ref 0 00000100:00100000:1.0:1789760797.438202:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760797.438907:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554910208 00000100:00100000:2.0:1789760797.438918:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554910208 00000100:00100000:2.0:1789760797.438974:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740654644763 time 740668280995 state 0x0 ref 1 00000100:00100000:2.0:1789760797.438989:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740668541411 time 740668280995 state 0x1 ref 1 00000100:00100000:2.0:1789760797.439023:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740668541411 time 740684641373 state 0x0 ref 1 00000100:00100000:2.0:1789760797.439036:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a386300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554910208:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760797.439081:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68550656, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20b18 00010000:00010000:2.0:1789760797.439091:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73d6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.439106:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760797.439111:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.439117:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68550656->68554751] (req 68550656->68554751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.439130:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68550656->68554751] (req 68550656->68554751) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.444146:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984045 is committed 00002000:00010000:2.0:1789760797.444341:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68550656 -> 68554752 00002000:00010000:2.0:1789760797.444347:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133912 -> 133920 00010000:00010000:2.0:1789760797.444362:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73d6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68550656->68554751] (req 68550656->68554751) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.444374:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73d6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68550656->68554751] (req 68550656->68554751) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.444418:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68550656->68554751] (req 68550656->68554751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.444426:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.444430:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73d6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68550656->68554751] (req 68550656->68554751) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.444439:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73d6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68550656->68554751] (req 68550656->68554751) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.444459:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73d6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68550656->68554751] (req 68550656->68554751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.444467:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73d6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68550656->68554751] (req 68550656->68554751) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.444553:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a386300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554910208:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5515us (6356us total) trans 4294984045 rc 0/0 00000100:00100000:2.0:1789760797.444564:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16747 00000100:00100000:2.0:1789760797.444583:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740668541411 time 740684641373 state 0x2 ref 0 00000100:00100000:0.0:1789760797.449993:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760797.450068:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554910336 00000100:00100000:2.0:1789760797.450078:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554910336 00000100:00100000:2.0:1789760797.450131:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740668541411 time 740684641373 state 0x0 ref 1 00000100:00100000:2.0:1789760797.450145:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740684901789 time 740684641373 state 0x1 ref 1 00000100:00100000:2.0:1789760797.450177:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740684901789 time 740695795736 state 0x0 ref 1 00000100:00100000:2.0:1789760797.450190:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e57b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554910336:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760797.450236:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68554752, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20b20 00010000:00010000:2.0:1789760797.450245:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73dd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.450260:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760797.450264:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.450270:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68554752->68558847] (req 68554752->68558847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.450283:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68554752->68558847] (req 68554752->68558847) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.458908:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984046 is committed 00002000:00010000:2.0:1789760797.459036:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68554752 -> 68558848 00002000:00010000:2.0:1789760797.459042:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133920 -> 133928 00010000:00010000:2.0:1789760797.459057:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73dd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68554752->68558847] (req 68554752->68558847) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.459070:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73dd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68554752->68558847] (req 68554752->68558847) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.459079:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68554752->68558847] (req 68554752->68558847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.459092:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.459098:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73dd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68554752->68558847] (req 68554752->68558847) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.459108:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73dd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68554752->68558847] (req 68554752->68558847) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.459141:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73dd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68554752->68558847] (req 68554752->68558847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.459148:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c73dd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68554752->68558847] (req 68554752->68558847) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.459246:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e57b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554910336:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9054us (9258us total) trans 4294984046 rc 0/0 00000100:00100000:2.0:1789760797.459256:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16748 00000100:00100000:2.0:1789760797.459265:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740684901789 time 740695795736 state 0x2 ref 0 00000100:00100000:1.0:1789760797.464827:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760797.465684:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554910464 00000100:00100000:0.0:1789760797.465695:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554910464 00000100:00100000:0.0:1789760797.465751:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740684901789 time 740695795736 state 0x0 ref 1 00000100:00100000:0.0:1789760797.465766:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740696056152 time 740695795736 state 0x1 ref 1 00000100:00100000:0.0:1789760797.465800:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740696056152 time 740711418070 state 0x0 ref 1 00000100:00100000:0.0:1789760797.465811:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554910464:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760797.465856:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68558848, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20b28 00010000:00010000:0.0:1789760797.465864:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c73e4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.465880:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760797.465884:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.465890:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c73e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68558848->68562943] (req 68558848->68562943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.465902:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c73e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68558848->68562943] (req 68558848->68562943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.469770:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984047 is committed 00002000:00010000:0.0:1789760797.470045:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68558848 -> 68562944 00002000:00010000:0.0:1789760797.470051:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133928 -> 133936 00010000:00010000:0.0:1789760797.470066:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c73e4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68558848->68562943] (req 68558848->68562943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.470077:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c73e4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68558848->68562943] (req 68558848->68562943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.470085:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c73e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68558848->68562943] (req 68558848->68562943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.470092:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760797.470097:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c73e4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68558848->68562943] (req 68558848->68562943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.470170:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c73e4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68558848->68562943] (req 68558848->68562943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.470192:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c73e4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68558848->68562943] (req 68558848->68562943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.470199:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c73e4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68558848->68562943] (req 68558848->68562943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.470294:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554910464:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4480us (5474us total) trans 4294984047 rc 0/0 00000100:00100000:0.0:1789760797.470313:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16749 00000100:00100000:0.0:1789760797.471262:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740696056152 time 740711418070 state 0x2 ref 0 00000100:00100000:1.0:1789760797.474183:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760797.474313:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554910592 00000100:00100000:2.0:1789760797.474323:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554910592 00000100:00100000:2.0:1789760797.474419:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740696056152 time 740711418070 state 0x0 ref 1 00000100:00100000:2.0:1789760797.474436:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740711678486 time 740711418070 state 0x1 ref 1 00000100:00100000:2.0:1789760797.474464:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740711678486 time 740720082675 state 0x0 ref 1 00000100:00100000:2.0:1789760797.474479:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a386680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554910592:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760797.474529:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68562944, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20b30 00010000:00010000:2.0:1789760797.474538:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73eb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.474553:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760797.474557:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.474564:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68562944->68567039] (req 68562944->68567039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.474587:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68562944->68567039] (req 68562944->68567039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.483685:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984048 is committed 00002000:00010000:2.0:1789760797.483800:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68562944 -> 68567040 00002000:00010000:2.0:1789760797.483805:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133936 -> 133944 00010000:00010000:2.0:1789760797.483818:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73eb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68562944->68567039] (req 68562944->68567039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.483831:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73eb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68562944->68567039] (req 68562944->68567039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.483840:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68562944->68567039] (req 68562944->68567039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.483848:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.483854:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73eb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68562944->68567039] (req 68562944->68567039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.483863:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73eb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68562944->68567039] (req 68562944->68567039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.483883:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73eb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68562944->68567039] (req 68562944->68567039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.483891:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c73eb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68562944->68567039] (req 68562944->68567039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.483997:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a386680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554910592:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9515us (9819us total) trans 4294984048 rc 0/0 00000100:00100000:2.0:1789760797.484008:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16750 00000100:00100000:2.0:1789760797.484017:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740711678486 time 740720082675 state 0x2 ref 0 00000100:00100000:2.0:1789760797.488754:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760797.488838:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554910720 00000100:00100000:1.0:1789760797.488847:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554910720 00000100:00100000:1.0:1789760797.488903:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740711678486 time 740720082675 state 0x0 ref 1 00000100:00100000:1.0:1789760797.488923:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740720343091 time 740720082675 state 0x1 ref 1 00000100:00100000:1.0:1789760797.488961:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740720343091 time 740734579075 state 0x0 ref 1 00000100:00100000:1.0:1789760797.488972:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554910720:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760797.489020:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68567040, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20b38 00010000:00010000:1.0:1789760797.489029:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c73f2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.489045:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760797.489050:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.489056:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c73f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68567040->68571135] (req 68567040->68571135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.489069:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c73f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68567040->68571135] (req 68567040->68571135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.497925:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984049 is committed 00002000:00010000:1.0:1789760797.498300:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68567040 -> 68571136 00002000:00010000:1.0:1789760797.498306:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133944 -> 133952 00010000:00010000:1.0:1789760797.498318:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c73f2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68567040->68571135] (req 68567040->68571135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.498454:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c73f2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68567040->68571135] (req 68567040->68571135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.498463:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c73f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68567040->68571135] (req 68567040->68571135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.498470:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655d200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760797.498474:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c73f2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68567040->68571135] (req 68567040->68571135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.498484:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c73f2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68567040->68571135] (req 68567040->68571135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.498503:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c73f2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68567040->68571135] (req 68567040->68571135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.498515:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071a6109d/0x1f99687d854c73f2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68567040->68571135] (req 68567040->68571135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.498660:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554910720:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9684us (9911us total) trans 4294984049 rc 0/0 00000100:00100000:1.0:1789760797.498671:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16751 00000100:00100000:1.0:1789760797.498679:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740720343091 time 740734579075 state 0x2 ref 0 00000100:00100000:1.0:1789760797.502789:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760797.502934:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554910848 00000100:00100000:0.0:1789760797.503003:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554910848 00000100:00100000:0.0:1789760797.503059:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740720343091 time 740734579075 state 0x0 ref 1 00000100:00100000:0.0:1789760797.503075:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740734839491 time 740734579075 state 0x1 ref 1 00000100:00100000:0.0:1789760797.503201:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740734839491 time 740748817997 state 0x0 ref 1 00000100:00100000:0.0:1789760797.503215:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554910848:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760797.503269:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68571136, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20b40 00010000:00010000:0.0:1789760797.503278:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c73f9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.503294:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760797.503298:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.503305:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c73f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68571136->68575231] (req 68571136->68575231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.503318:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c73f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68571136->68575231] (req 68571136->68575231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.507781:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984050 is committed 00002000:00010000:0.0:1789760797.514493:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68571136 -> 68575232 00002000:00010000:0.0:1789760797.514503:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133952 -> 133960 00010000:00010000:0.0:1789760797.514517:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c73f9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68571136->68575231] (req 68571136->68575231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.514530:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c73f9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68571136->68575231] (req 68571136->68575231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.514539:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c73f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68571136->68575231] (req 68571136->68575231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.514546:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760797.514552:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c73f9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68571136->68575231] (req 68571136->68575231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.514574:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c73f9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68571136->68575231] (req 68571136->68575231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.514596:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c73f9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68571136->68575231] (req 68571136->68575231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.514604:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c73f9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68571136->68575231] (req 68571136->68575231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.514751:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554910848:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11534us (11967us total) trans 4294984050 rc 0/0 00000100:00100000:0.0:1789760797.514762:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16752 00000100:00100000:0.0:1789760797.514770:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740734839491 time 740748817997 state 0x2 ref 0 00000100:00100000:2.0:1789760797.519695:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760797.519777:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554910976 00000100:00100000:0.0:1789760797.519786:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554910976 00000100:00100000:0.0:1789760797.519834:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740734839491 time 740748817997 state 0x0 ref 1 00000100:00100000:0.0:1789760797.519849:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740749078413 time 740748817997 state 0x1 ref 1 00000100:00100000:0.0:1789760797.519920:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740749078413 time 740765537291 state 0x0 ref 1 00000100:00100000:0.0:1789760797.519933:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554910976:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760797.519988:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68575232, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20b48 00010000:00010000:0.0:1789760797.519996:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7400 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.520011:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760797.520016:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.520022:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7400 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68575232->68579327] (req 68575232->68579327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.520035:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7400 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68575232->68579327] (req 68575232->68579327) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.525059:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984051 is committed 00002000:00010000:2.0:1789760797.525173:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68575232 -> 68579328 00002000:00010000:2.0:1789760797.525178:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133960 -> 133968 00010000:00010000:2.0:1789760797.525191:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7400 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68575232->68579327] (req 68575232->68579327) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.525203:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7400 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68575232->68579327] (req 68575232->68579327) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.525218:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7400 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68575232->68579327] (req 68575232->68579327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.525224:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.525230:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7400 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68575232->68579327] (req 68575232->68579327) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.525239:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7400 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68575232->68579327] (req 68575232->68579327) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.525257:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7400 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68575232->68579327] (req 68575232->68579327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.525265:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025b080ad/0x1f99687d854c7400 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68575232->68579327] (req 68575232->68579327) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.525363:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554910976:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5428us (5672us total) trans 4294984051 rc 0/0 00000100:00100000:2.0:1789760797.525373:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16753 00000100:00100000:2.0:1789760797.525416:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740749078413 time 740765537291 state 0x2 ref 0 00000100:00100000:2.0:1789760797.530148:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760797.530441:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554911104 00000100:00100000:1.0:1789760797.530451:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554911104 00000100:00100000:1.0:1789760797.530503:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740749078413 time 740765537291 state 0x0 ref 1 00000100:00100000:1.0:1789760797.530518:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740765797707 time 740765537291 state 0x1 ref 1 00000100:00100000:1.0:1789760797.530547:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740765797707 time 740776166161 state 0x0 ref 1 00000100:00100000:1.0:1789760797.530558:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554911104:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760797.530610:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68579328, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20b50 00010000:00010000:1.0:1789760797.530619:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c7407 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.530635:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:1.0:1789760797.530638:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760797.530645:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c7407 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68579328->68583423] (req 68579328->68583423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.530658:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c7407 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68579328->68583423] (req 68579328->68583423) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.534690:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984052 is committed 00002000:00010000:1.0:1789760797.535156:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68579328 -> 68583424 00002000:00010000:1.0:1789760797.535162:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133968 -> 133976 00010000:00010000:1.0:1789760797.535175:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c7407 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68579328->68583423] (req 68579328->68583423) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.535188:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c7407 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68579328->68583423] (req 68579328->68583423) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.535197:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c7407 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68579328->68583423] (req 68579328->68583423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.535204:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655da00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760797.535209:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c7407 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68579328->68583423] (req 68579328->68583423) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.535222:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c7407 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68579328->68583423] (req 68579328->68583423) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.535242:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c7407 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68579328->68583423] (req 68579328->68583423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.535256:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067260f45/0x1f99687d854c7407 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68579328->68583423] (req 68579328->68583423) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.535681:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554911104:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5117us (5535us total) trans 4294984052 rc 0/0 00000100:00100000:1.0:1789760797.535691:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16754 00000100:00100000:1.0:1789760797.535701:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740765797707 time 740776166161 state 0x2 ref 0 00000100:00100000:1.0:1789760797.539512:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760797.539610:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554911232 00000100:00100000:2.0:1789760797.539620:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554911232 00000100:00100000:2.0:1789760797.539671:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740765797707 time 740776166161 state 0x0 ref 1 00000100:00100000:2.0:1789760797.539688:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740776426577 time 740776166161 state 0x1 ref 1 00000100:00100000:2.0:1789760797.539754:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740776426577 time 740785371825 state 0x0 ref 1 00000100:00100000:2.0:1789760797.539766:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554911232:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760797.539811:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68583424, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20b58 00010000:00010000:2.0:1789760797.539820:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c740e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.539835:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760797.539839:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.539846:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c740e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68583424->68587519] (req 68583424->68587519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.539861:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c740e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68583424->68587519] (req 68583424->68587519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.544092:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984053 is committed 00002000:00010000:2.0:1789760797.544434:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68583424 -> 68587520 00002000:00010000:2.0:1789760797.544440:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133976 -> 133984 00010000:00010000:2.0:1789760797.544453:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c740e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68583424->68587519] (req 68583424->68587519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.544467:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c740e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68583424->68587519] (req 68583424->68587519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.544476:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c740e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68583424->68587519] (req 68583424->68587519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.544485:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.544489:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c740e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68583424->68587519] (req 68583424->68587519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.544499:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c740e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68583424->68587519] (req 68583424->68587519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.544519:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c740e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68583424->68587519] (req 68583424->68587519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.544527:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c740e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68583424->68587519] (req 68583424->68587519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.545606:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554911232:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5831us (6095us total) trans 4294984053 rc 0/0 00000100:00100000:2.0:1789760797.545620:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16755 00000100:00100000:2.0:1789760797.545630:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740776426577 time 740785371825 state 0x2 ref 0 00000100:00100000:2.0:1789760797.554369:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760797.554505:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554911360 00000100:00100000:0.0:1789760797.554514:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554911360 00000100:00100000:0.0:1789760797.554582:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740776426577 time 740785371825 state 0x0 ref 1 00000100:00100000:0.0:1789760797.554596:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740785632241 time 740785371825 state 0x1 ref 1 00000100:00100000:0.0:1789760797.554692:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740785632241 time 740800309740 state 0x0 ref 1 00000100:00100000:0.0:1789760797.554704:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554911360:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760797.554748:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68587520, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20b60 00010000:00010000:0.0:1789760797.554757:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c7415 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.554772:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760797.554776:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.554782:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c7415 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68587520->68591615] (req 68587520->68591615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.554796:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c7415 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68587520->68591615] (req 68587520->68591615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.563077:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984054 is committed 00002000:00010000:0.0:1789760797.563205:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68587520 -> 68591616 00002000:00010000:0.0:1789760797.563211:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133984 -> 133992 00010000:00010000:0.0:1789760797.563225:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c7415 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68587520->68591615] (req 68587520->68591615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.563240:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c7415 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68587520->68591615] (req 68587520->68591615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.563249:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c7415 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68587520->68591615] (req 68587520->68591615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.563257:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760797.563261:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c7415 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68587520->68591615] (req 68587520->68591615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.563271:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c7415 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68587520->68591615] (req 68587520->68591615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.563290:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c7415 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68587520->68591615] (req 68587520->68591615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.563304:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ee3096d/0x1f99687d854c7415 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68587520->68591615] (req 68587520->68591615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.563456:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554911360:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8748us (9090us total) trans 4294984054 rc 0/0 00000100:00100000:0.0:1789760797.563467:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16756 00000100:00100000:0.0:1789760797.563476:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740785632241 time 740800309740 state 0x2 ref 0 00000100:00100000:0.0:1789760797.571285:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760797.571450:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554911488 00000100:00100000:2.0:1789760797.571460:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554911488 00000100:00100000:2.0:1789760797.571515:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740785632241 time 740800309740 state 0x0 ref 1 00000100:00100000:2.0:1789760797.571530:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740800570156 time 740800309740 state 0x1 ref 1 00000100:00100000:2.0:1789760797.571634:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740800570156 time 740817251683 state 0x0 ref 1 00000100:00100000:2.0:1789760797.571649:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e57480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554911488:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760797.571694:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68591616, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20b68 00010000:00010000:2.0:1789760797.571705:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c741c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.571721:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760797.571726:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.571732:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c741c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68591616->68595711] (req 68591616->68595711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.571746:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c741c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68591616->68595711] (req 68591616->68595711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.581613:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984055 is committed 00002000:00010000:2.0:1789760797.581795:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68591616 -> 68595712 00002000:00010000:2.0:1789760797.581801:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 133992 -> 134000 00010000:00010000:2.0:1789760797.581813:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c741c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68591616->68595711] (req 68591616->68595711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.581827:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c741c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68591616->68595711] (req 68591616->68595711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.581836:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c741c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68591616->68595711] (req 68591616->68595711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.581843:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.581848:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c741c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68591616->68595711] (req 68591616->68595711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.581857:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c741c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68591616->68595711] (req 68591616->68595711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.581876:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c741c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68591616->68595711] (req 68591616->68595711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.581884:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c741c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68591616->68595711] (req 68591616->68595711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.582047:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e57480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554911488:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10396us (10768us total) trans 4294984055 rc 0/0 00000100:00100000:2.0:1789760797.582057:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16757 00000100:00100000:2.0:1789760797.582067:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740800570156 time 740817251683 state 0x2 ref 0 00000100:00100000:0.0:1789760797.586929:0:3646:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789760797.586966:0:3646:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb03e54700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573923584:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789760797.587037:0:3646:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760797.587215:0:11526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573923584 00000100:00100000:3.0:1789760797.587223:0:11526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573923584 00000100:00100000:3.0:1789760797.587285:0:11526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 374 00000100:00100000:3.0:1789760797.587297:0:11526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e57800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573923584:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789760797.587327:0:11526:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:3.0:1789760797.587403:0:11526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e57800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3646:x1876699573923584:12345-0@lo:13:osp-pre-1-0.0 Request processed in 108us (364us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760797.587410:0:11526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 374 00000100:00100000:3.0:1789760797.587458:0:3646:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb03e54700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3646:1876699573923584:0@lo:13:osp-pre-1-0.0 00000100:00080000:0.0:1789760797.587524:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789760797.587932:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760797.588052:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554911616 00000100:00100000:3.0:1789760797.588057:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554911616 00000100:00100000:3.0:1789760797.588094:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740800570156 time 740817251683 state 0x0 ref 1 00000100:00100000:3.0:1789760797.588107:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740817512099 time 740817251683 state 0x1 ref 1 00000100:00100000:3.0:1789760797.588133:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740817512099 time 740833751729 state 0x0 ref 1 00000100:00100000:3.0:1789760797.588144:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554911616:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760797.588184:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68595712, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20b70 00010000:00010000:3.0:1789760797.588193:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7423 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.588209:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760797.588215:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.588221:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7423 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68595712->68599807] (req 68595712->68599807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.588234:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7423 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68595712->68599807] (req 68595712->68599807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.588947:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cc3a384e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:5711db52-8702-4d6b-b42c-8543e9bf1cb2:3648:1876699573923712:0@lo:400:kworker.0 00000100:00100000:0.0:1789760797.588999:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760797.589042:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cc3a385880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3648:1876699573924224:0@lo:400:kworker.0 00000100:00100000:0.0:1789760797.589069:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760797.589089:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb049d6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3648:1876699573924736:0@lo:400:kworker.0 00000100:00100000:0.0:1789760797.589110:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760797.589128:0:3648:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:0.0:1789760797.589134:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cc3a386d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3648:1876699573923840:0@lo:400:kworker.0 00000100:00100000:0.0:1789760797.589157:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760797.589173:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb049d4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3648:1876699573924352:0@lo:400:kworker.0 00000100:00100000:0.0:1789760797.589193:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760797.589208:0:3648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb049d7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3648:1876699573924864:0@lo:400:kworker.0 00000100:00100000:0.0:1789760797.589226:0:3648:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760797.589254:0:7831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573924224 00000100:00100000:0.0:1789760797.589259:0:7831:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573924224 00000100:00100000:0.0:1789760797.589292:0:7831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 398 00000100:00100000:0.0:1789760797.589299:0:7831:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e54a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3648:x1876699573924224:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760797.589344:0:7831:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3648:x1876699573924224:12345-0@lo:400:kworker.0 Request processed in 45us (274us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760797.589351:0:7831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 398 00000100:00100000:0.0:1789760797.589372:0:6522:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573923840 00000100:00100000:0.0:1789760797.589393:0:6522:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573923840 00000100:00100000:0.0:1789760797.589417:0:6522:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573924352 00000100:00100000:0.0:1789760797.589420:0:6522:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573924352 00000100:00100000:0.0:1789760797.589440:0:6522:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573924864 00000100:00100000:0.0:1789760797.589443:0:6522:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573924864 00000100:00100000:0.0:1789760797.589458:0:6522:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 909 00000100:00100000:0.0:1789760797.589464:0:6522:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e56680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3648:x1876699573923840:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760797.589494:0:6522:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e56680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3648:x1876699573923840:12345-0@lo:400:kworker.0 Request processed in 30us (336us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760797.589500:0:6522:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 909 00000100:00100000:0.0:1789760797.589512:0:6522:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 910 00000100:00100000:0.0:1789760797.589518:0:6522:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e54380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3648:x1876699573924352:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760797.589542:0:6522:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3648:x1876699573924352:12345-0@lo:400:kworker.0 Request processed in 24us (348us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760797.589547:0:6522:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 910 00000100:00100000:0.0:1789760797.589558:0:6522:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 911 00000100:00100000:0.0:1789760797.589573:0:6522:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e57100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3648:x1876699573924864:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760797.589599:0:6522:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e57100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3648:x1876699573924864:12345-0@lo:400:kworker.0 Request processed in 35us (372us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760797.589605:0:6522:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 911 00000100:00080000:0.0:1789760797.589627:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789760797.589651:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cc3a386d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3648:1876699573923840:0@lo:400:kworker.0 00000100:00100000:0.0:1789760797.589661:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb049d4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3648:1876699573924352:0@lo:400:kworker.0 00000100:00100000:0.0:1789760797.589669:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb049d7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3648:1876699573924864:0@lo:400:kworker.0 00000100:00100000:0.0:1789760797.589677:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cc3a385880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3648:1876699573924224:0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.589827:0:6515:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573923712 00000100:00100000:1.0:1789760797.589832:0:6515:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573923712 00000100:00100000:1.0:1789760797.589867:0:6515:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 477 00000100:00100000:1.0:1789760797.589874:0:6515:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc320b4c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5711db52-8702-4d6b-b42c-8543e9bf1cb2+24:3648:x1876699573923712:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789760797.589914:0:6515:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc320b4c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5711db52-8702-4d6b-b42c-8543e9bf1cb2+24:3648:x1876699573923712:12345-0@lo:400:kworker.0 Request processed in 41us (915us total) trans 0 rc 0/0 00000100:00100000:1.0:1789760797.589920:0:6515:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 477 00000100:00100000:1.0:1789760797.589956:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cc3a384e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:5711db52-8702-4d6b-b42c-8543e9bf1cb2:3648:1876699573923712:0@lo:400:kworker.0 00000100:00100000:2.0:1789760797.591780:0:3649:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789760797.591952:0:3649:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cc1d94d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3649:1876699573925248:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1789760797.592008:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760797.592109:0:8439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573925248 00000100:00100000:2.0:1789760797.592115:0:8439:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573925248 00000100:00100000:2.0:1789760797.592244:0:8439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 375 00000100:00100000:2.0:1789760797.592253:0:8439:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1876699573925248:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1789760797.592275:0:8439:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3923529728 avail 3692007424 00000100:00100000:2.0:1789760797.592475:0:8439:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3649:x1876699573925248:12345-0@lo:13:osp-pre-1-1.0 Request processed in 221us (464us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760797.592486:0:8439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 375 00000100:00080000:2.0:1789760797.592515:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789760797.592539:0:3647:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cc3a387480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3647:1876699573924096:0@lo:400:kworker.0 00000100:00100000:2.0:1789760797.592546:0:3649:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb046fa300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3649:1876699573925376:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789760797.592600:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760797.592607:0:3649:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760797.592637:0:3647:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb049d5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3647:1876699573924608:0@lo:400:kworker.0 00000100:00100000:2.0:1789760797.592657:0:3649:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cc1d94d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3649:1876699573925248:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1789760797.592668:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760797.592688:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573925376 00000100:00100000:2.0:1789760797.592693:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573925376 00000100:00100000:3.0:1789760797.592694:0:3647:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb049d5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3647:1876699573925120:0@lo:400:kworker.0 00000100:00100000:3.0:1789760797.592725:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789760797.592729:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 399 00000100:00100000:2.0:1789760797.592738:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3649:x1876699573925376:12345-0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1789760797.592751:0:3647:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:3.0:1789760797.592758:0:3647:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cc3a385f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3647:1876699573923968:0@lo:400:kworker.0 00000004:00000024:2.0:1789760797.592764:0:6526:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446227968 avail 1299988480 00000100:00100000:3.0:1789760797.592784:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760797.592800:0:3647:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb049d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3647:1876699573924480:0@lo:400:kworker.0 00000100:00100000:0.0:1789760797.594691:0:8438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573924736 00000100:00100000:0.0:1789760797.594704:0:8438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573924736 00000100:00100000:0.0:1789760797.594773:0:8438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573924608 00000100:00100000:0.0:1789760797.594777:0:8438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573924608 00000100:00100000:0.0:1789760797.594794:0:8438:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573925120 00000100:00100000:0.0:1789760797.595469:0:8438:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573925120 00000100:00080000:1.0:1789760797.595486:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789760797.595503:0:8438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 305 00000100:00100000:0.0:1789760797.595513:0:8438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e55880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3648:x1876699573924736:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760797.597715:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3649:x1876699573925376:12345-0@lo:41:osp-pre-0-1.0 Request processed in 4970us (5108us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760797.597727:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 399 00000100:00100000:2.0:1789760797.597761:0:6526:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573923968 00000100:00100000:2.0:1789760797.597768:0:6526:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573923968 00000100:00100000:0.0:1789760797.597790:0:8438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3648:x1876699573924736:12345-0@lo:400:kworker.0 Request processed in 2276us (8675us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760797.597799:0:6526:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 400 00000100:00100000:0.0:1789760797.597802:0:8438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 305 00000100:00100000:2.0:1789760797.597811:0:6526:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3647:x1876699573923968:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760797.597827:0:8438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 306 00000100:00100000:0.0:1789760797.597839:0:8438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3647:x1876699573924608:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760797.597873:0:6526:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3647:x1876699573923968:12345-0@lo:400:kworker.0 Request processed in 64us (5088us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760797.597881:0:6526:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 400 00000100:00100000:0.0:1789760797.597886:0:8438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3647:x1876699573924608:12345-0@lo:400:kworker.0 Request processed in 49us (5217us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760797.597894:0:8438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 306 00000100:00100000:0.0:1789760797.597911:0:8438:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 307 00000100:00100000:0.0:1789760797.597918:0:8438:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3647:x1876699573925120:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760797.597947:0:3649:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb046fa300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3649:1876699573925376:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1789760797.597956:0:8438:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046faa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+5:3647:x1876699573925120:12345-0@lo:400:kworker.0 Request processed in 38us (5231us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760797.597963:0:8438:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 307 00000100:00100000:0.0:1789760797.597995:0:11089:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573924096 00000100:00100000:0.0:1789760797.598000:0:11089:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573924096 00000100:00100000:0.0:1789760797.598035:0:11089:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 912 00000100:00100000:0.0:1789760797.598042:0:11089:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3647:x1876699573924096:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789760797.598076:0:11089:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3647:x1876699573924096:12345-0@lo:400:kworker.0 Request processed in 36us (5482us total) trans 0 rc 0/0 00000100:00100000:0.0:1789760797.598083:0:11089:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 912 00000100:00100000:1.0:1789760797.598657:0:3648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb049d6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3648:1876699573924736:0@lo:400:kworker.0 00000100:00100000:3.0:1789760797.599865:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789760797.599917:0:3647:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff91cb049d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3647:1876699573924992:0@lo:400:kworker.0 00000100:00100000:0.0:1789760797.599918:0:7470:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573924480 00000100:00100000:0.0:1789760797.599928:0:7470:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573924480 00000100:00100000:3.0:1789760797.599960:0:3647:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789760797.599995:0:7470:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699573924992 00000100:00100000:0.0:1789760797.599999:0:7470:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699573924992 00000100:00080000:1.0:1789760797.600019:0:6516:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 00000100:00100000:0.0:1789760797.600034:0:7470:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 913 00000100:00100000:0.0:1789760797.600043:0:7470:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3647:x1876699573924480:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789760797.600082:0:3647:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cc3a387480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3647:1876699573924096:0@lo:400:kworker.0 00000100:00100000:0.0:1789760797.600097:0:7470:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3647:x1876699573924480:12345-0@lo:400:kworker.0 Request processed in 54us (235us total) trans 0 rc 0/0 00000100:00100000:3.0:1789760797.600105:0:3647:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb049d5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3647:1876699573924608:0@lo:400:kworker.0 00000100:00100000:3.0:1789760797.600144:0:3647:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb049d5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3647:1876699573925120:0@lo:400:kworker.0 00000100:00100000:3.0:1789760797.600168:0:3647:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cc3a385f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3647:1876699573923968:0@lo:400:kworker.0 00000100:00100000:3.0:1789760797.600178:0:3647:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb049d4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3647:1876699573924480:0@lo:400:kworker.0 00000100:00100000:2.0:1789760797.600256:0:6520:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 914 00000100:00100000:2.0:1789760797.600269:0:6520:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3647:x1876699573924992:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789760797.600319:0:6520:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3647:x1876699573924992:12345-0@lo:400:kworker.0 Request processed in 54us (359us total) trans 0 rc 0/0 00000100:00100000:2.0:1789760797.600405:0:6520:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 914 00000100:00100000:0.0:1789760797.600973:0:7470:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 913 00000100:00100000:2.0:1789760797.605811:0:3647:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff91cb049d4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3647:1876699573924992:0@lo:400:kworker.0 00000001:00080000:1.0:1789760797.607222:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984056 is committed 00002000:00010000:3.0:1789760797.607343:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68595712 -> 68599808 00002000:00010000:3.0:1789760797.607348:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134000 -> 134008 00010000:00010000:3.0:1789760797.607361:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7423 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68595712->68599807] (req 68595712->68599807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.607374:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7423 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68595712->68599807] (req 68595712->68599807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.607408:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7423 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68595712->68599807] (req 68595712->68599807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.607419:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760797.607424:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7423 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68595712->68599807] (req 68595712->68599807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.607441:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7423 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68595712->68599807] (req 68595712->68599807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.607460:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7423 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68595712->68599807] (req 68595712->68599807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.607468:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7423 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68595712->68599807] (req 68595712->68599807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.607560:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554911616:12345-192.168.203.22@tcp:4:dd.0 Request processed in 19412us (19625us total) trans 4294984056 rc 0/0 00000100:00100000:3.0:1789760797.607579:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16758 00000100:00100000:3.0:1789760797.607589:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740817512099 time 740833751729 state 0x2 ref 0 00000100:00100000:1.0:1789760797.614808:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760797.615012:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554911744 00000100:00100000:2.0:1789760797.615022:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554911744 00000100:00100000:2.0:1789760797.615081:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740817512099 time 740833751729 state 0x0 ref 1 00000100:00100000:2.0:1789760797.615096:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740834012145 time 740833751729 state 0x1 ref 1 00000100:00100000:2.0:1789760797.615133:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740834012145 time 740860751018 state 0x0 ref 1 00000100:00100000:2.0:1789760797.615145:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554911744:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760797.615193:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68599808, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20b78 00010000:00010000:2.0:1789760797.615203:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c742a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.615219:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760797.615224:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.615228:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c742a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68599808->68603903] (req 68599808->68603903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.615240:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c742a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68599808->68603903] (req 68599808->68603903) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.622920:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984057 is committed 00002000:00010000:2.0:1789760797.624111:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68599808 -> 68603904 00002000:00010000:2.0:1789760797.624119:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134008 -> 134016 00010000:00010000:2.0:1789760797.624133:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c742a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68599808->68603903] (req 68599808->68603903) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.624147:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c742a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68599808->68603903] (req 68599808->68603903) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.624157:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c742a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68599808->68603903] (req 68599808->68603903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.624165:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.624170:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c742a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68599808->68603903] (req 68599808->68603903) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.624179:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c742a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68599808->68603903] (req 68599808->68603903) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.624199:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c742a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68599808->68603903] (req 68599808->68603903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.624206:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c742a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68599808->68603903] (req 68599808->68603903) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.624909:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554911744:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9757us (10105us total) trans 4294984057 rc 0/0 00000100:00100000:2.0:1789760797.624922:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16759 00000100:00100000:2.0:1789760797.624931:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740834012145 time 740860751018 state 0x2 ref 0 00000100:00100000:0.0:1789760797.634311:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760797.634494:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554911872 00000100:00100000:2.0:1789760797.634503:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554911872 00000100:00100000:2.0:1789760797.634575:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740834012145 time 740860751018 state 0x0 ref 1 00000100:00100000:2.0:1789760797.634598:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740861011434 time 740860751018 state 0x1 ref 1 00000100:00100000:2.0:1789760797.634623:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740861011434 time 740880242363 state 0x0 ref 1 00000100:00100000:2.0:1789760797.634635:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03e54700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554911872:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760797.634682:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68603904, mtime: 0x6aad951c, atime: 0x0, ctime: 0x6aad951c, blocks: 0x20b80 00010000:00010000:2.0:1789760797.634691:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7431 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.634704:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760797.634709:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.634715:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7431 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68603904->68607999] (req 68603904->68607999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.634729:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7431 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68603904->68607999] (req 68603904->68607999) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.640746:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984058 is committed 00002000:00010000:2.0:1789760797.641086:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68603904 -> 68608000 00002000:00010000:2.0:1789760797.641092:0:8443:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760796 -> 1789760797 00002000:00010000:2.0:1789760797.641095:0:8443:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760796 -> 1789760797 00002000:00010000:2.0:1789760797.641097:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134016 -> 134024 00010000:00010000:2.0:1789760797.641110:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7431 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68603904->68607999] (req 68603904->68607999) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.641128:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7431 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68603904->68607999] (req 68603904->68607999) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.641136:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7431 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68603904->68607999] (req 68603904->68607999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.641142:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.641147:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7431 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68603904->68607999] (req 68603904->68607999) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.641156:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7431 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68603904->68607999] (req 68603904->68607999) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.641175:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7431 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68603904->68607999] (req 68603904->68607999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.641183:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7431 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68603904->68607999] (req 68603904->68607999) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.641278:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03e54700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554911872:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6640us (6971us total) trans 4294984058 rc 0/0 00000100:00100000:2.0:1789760797.641288:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16760 00000100:00100000:2.0:1789760797.641298:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740861011434 time 740880242363 state 0x2 ref 0 00000100:00100000:1.0:1789760797.655906:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760797.656350:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554912000 00000100:00100000:0.0:1789760797.656359:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554912000 00000100:00100000:0.0:1789760797.656674:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740861011434 time 740880242363 state 0x0 ref 1 00000100:00100000:0.0:1789760797.657023:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740880502779 time 740880242363 state 0x1 ref 1 00000100:00100000:0.0:1789760797.657078:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740880502779 time 740902695061 state 0x0 ref 1 00000100:00100000:0.0:1789760797.657093:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554912000:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760797.657141:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68608000, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20b88 00010000:00010000:0.0:1789760797.657151:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c7438 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.657166:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760797.657171:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.657177:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c7438 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68608000->68612095] (req 68608000->68612095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.657191:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c7438 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68608000->68612095] (req 68608000->68612095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.664051:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984059 is committed 00002000:00010000:2.0:1789760797.664295:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68608000 -> 68612096 00002000:00010000:2.0:1789760797.664302:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134024 -> 134032 00010000:00010000:2.0:1789760797.664315:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c7438 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68608000->68612095] (req 68608000->68612095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.664744:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c7438 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68608000->68612095] (req 68608000->68612095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.664754:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c7438 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68608000->68612095] (req 68608000->68612095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.664762:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.664767:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c7438 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68608000->68612095] (req 68608000->68612095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.664776:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c7438 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68608000->68612095] (req 68608000->68612095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.664797:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c7438 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68608000->68612095] (req 68608000->68612095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.664805:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006ba9ee6a/0x1f99687d854c7438 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68608000->68612095] (req 68608000->68612095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.664894:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554912000:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7800us (8993us total) trans 4294984059 rc 0/0 00000100:00100000:2.0:1789760797.664905:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16761 00000100:00100000:2.0:1789760797.664915:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740880502779 time 740902695061 state 0x2 ref 0 00000100:00100000:1.0:1789760797.679420:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760797.679636:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554912128 00000100:00100000:3.0:1789760797.679646:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554912128 00000100:00100000:3.0:1789760797.679703:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740880502779 time 740902695061 state 0x0 ref 1 00000100:00100000:3.0:1789760797.679719:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740902955477 time 740902695061 state 0x1 ref 1 00000100:00100000:3.0:1789760797.679752:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740902955477 time 740925370532 state 0x0 ref 1 00000100:00100000:3.0:1789760797.679768:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554912128:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760797.679817:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68612096, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20b90 00010000:00010000:3.0:1789760797.679827:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c743f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.679844:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760797.679848:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.679854:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c743f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68612096->68616191] (req 68612096->68616191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.679868:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c743f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68612096->68616191] (req 68612096->68616191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.686738:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984060 is committed 00002000:00010000:3.0:1789760797.686995:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68612096 -> 68616192 00002000:00010000:3.0:1789760797.687000:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134032 -> 134040 00010000:00010000:3.0:1789760797.687011:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c743f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68612096->68616191] (req 68612096->68616191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.687020:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c743f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68612096->68616191] (req 68612096->68616191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.687026:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c743f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68612096->68616191] (req 68612096->68616191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.687031:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760797.687035:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c743f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68612096->68616191] (req 68612096->68616191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.687041:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c743f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68612096->68616191] (req 68612096->68616191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.687125:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c743f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68612096->68616191] (req 68612096->68616191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.687134:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c743f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68612096->68616191] (req 68612096->68616191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.687257:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554912128:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7486us (7840us total) trans 4294984060 rc 0/0 00000100:00100000:3.0:1789760797.687268:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16762 00000100:00100000:3.0:1789760797.687278:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740902955477 time 740925370532 state 0x2 ref 0 00000100:00100000:0.0:1789760797.700136:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760797.700237:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554912256 00000100:00100000:2.0:1789760797.700258:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554912256 00000100:00100000:2.0:1789760797.700314:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740902955477 time 740925370532 state 0x0 ref 1 00000100:00100000:2.0:1789760797.700341:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740925630948 time 740925370532 state 0x1 ref 1 00000100:00100000:2.0:1789760797.700373:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740925630948 time 740945991299 state 0x0 ref 1 00000100:00100000:2.0:1789760797.700423:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554912256:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760797.700478:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68616192, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20b98 00010000:00010000:2.0:1789760797.700487:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7446 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.700504:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:2.0:1789760797.700508:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.700514:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7446 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68616192->68620287] (req 68616192->68620287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.700529:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7446 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68616192->68620287] (req 68616192->68620287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.713780:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984061 is committed 00002000:00010000:2.0:1789760797.714169:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68616192 -> 68620288 00002000:00010000:2.0:1789760797.714175:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134040 -> 134048 00010000:00010000:2.0:1789760797.714188:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7446 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68616192->68620287] (req 68616192->68620287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.714206:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7446 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68616192->68620287] (req 68616192->68620287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.714215:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7446 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68616192->68620287] (req 68616192->68620287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.714223:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.714228:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7446 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68616192->68620287] (req 68616192->68620287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.714237:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7446 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68616192->68620287] (req 68616192->68620287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.714256:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7446 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68616192->68620287] (req 68616192->68620287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.714264:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7446 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68616192->68620287] (req 68616192->68620287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.714757:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554912256:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14330us (14624us total) trans 4294984061 rc 0/0 00000100:00100000:2.0:1789760797.714770:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16763 00000100:00100000:2.0:1789760797.714779:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740925630948 time 740945991299 state 0x2 ref 0 00000100:00100000:1.0:1789760797.718280:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760797.718821:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554912384 00000100:00100000:0.0:1789760797.718832:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554912384 00000100:00100000:0.0:1789760797.718884:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740925630948 time 740945991299 state 0x0 ref 1 00000100:00100000:0.0:1789760797.718911:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740946251715 time 740945991299 state 0x1 ref 1 00000100:00100000:0.0:1789760797.719000:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740946251715 time 740964561431 state 0x0 ref 1 00000100:00100000:0.0:1789760797.719019:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554912384:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760797.719064:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68620288, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20ba0 00010000:00010000:0.0:1789760797.719073:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c744d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.719089:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760797.719094:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.719100:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c744d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68620288->68624383] (req 68620288->68624383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.719114:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c744d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68620288->68624383] (req 68620288->68624383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.727236:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984062 is committed 00002000:00010000:1.0:1789760797.727622:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68620288 -> 68624384 00002000:00010000:1.0:1789760797.727628:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134048 -> 134056 00010000:00010000:1.0:1789760797.727643:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c744d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68620288->68624383] (req 68620288->68624383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.727657:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c744d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68620288->68624383] (req 68620288->68624383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.727666:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c744d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68620288->68624383] (req 68620288->68624383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.727673:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760797.727679:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c744d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68620288->68624383] (req 68620288->68624383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.727688:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c744d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68620288->68624383] (req 68620288->68624383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.727707:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c744d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68620288->68624383] (req 68620288->68624383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760797.727715:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009e2423ce/0x1f99687d854c744d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68620288->68624383] (req 68620288->68624383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760797.727853:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554912384:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8832us (9576us total) trans 4294984062 rc 0/0 00000100:00100000:1.0:1789760797.727864:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16764 00000100:00100000:1.0:1789760797.727873:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740946251715 time 740964561431 state 0x2 ref 0 00000100:00100000:1.0:1789760797.732841:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760797.733068:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554912512 00000100:00100000:0.0:1789760797.733078:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554912512 00000100:00100000:0.0:1789760797.733131:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740946251715 time 740964561431 state 0x0 ref 1 00000100:00100000:0.0:1789760797.733146:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740964821847 time 740964561431 state 0x1 ref 1 00000100:00100000:0.0:1789760797.733223:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740964821847 time 740978839927 state 0x0 ref 1 00000100:00100000:0.0:1789760797.733237:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554912512:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760797.733287:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68624384, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20ba8 00010000:00010000:0.0:1789760797.733296:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c7454 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.733313:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:0.0:1789760797.733441:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.733450:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c7454 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68624384->68628479] (req 68624384->68628479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.733466:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c7454 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68624384->68628479] (req 68624384->68628479) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.744429:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984063 is committed 00002000:00010000:2.0:1789760797.745584:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68624384 -> 68628480 00002000:00010000:2.0:1789760797.745589:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134056 -> 134064 00010000:00010000:2.0:1789760797.745602:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c7454 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68624384->68628479] (req 68624384->68628479) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.745614:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c7454 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68624384->68628479] (req 68624384->68628479) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.745623:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c7454 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68624384->68628479] (req 68624384->68628479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.745629:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.745633:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c7454 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68624384->68628479] (req 68624384->68628479) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.745642:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c7454 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68624384->68628479] (req 68624384->68628479) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.745660:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c7454 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68624384->68628479] (req 68624384->68628479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.745667:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042ad1e9f/0x1f99687d854c7454 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68624384->68628479] (req 68624384->68628479) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.745765:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554912512:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12526us (12929us total) trans 4294984063 rc 0/0 00000100:00100000:2.0:1789760797.745776:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16765 00000100:00100000:2.0:1789760797.745785:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740964821847 time 740978839927 state 0x2 ref 0 00000100:00100000:1.0:1789760797.752842:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760797.754375:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554912640 00000100:00100000:3.0:1789760797.754430:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554912640 00000100:00100000:3.0:1789760797.754500:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740964821847 time 740978839927 state 0x0 ref 1 00000100:00100000:3.0:1789760797.754517:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740979100343 time 740978839927 state 0x1 ref 1 00000100:00100000:3.0:1789760797.754549:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740979100343 time 741000167193 state 0x0 ref 1 00000100:00100000:3.0:1789760797.754561:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554912640:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760797.754854:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68628480, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20bb0 00010000:00010000:3.0:1789760797.754867:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c745b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.754884:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=740s v=(0 0) 00010000:00010000:3.0:1789760797.754888:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.754894:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c745b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68628480->68632575] (req 68628480->68632575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.754908:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c745b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68628480->68632575] (req 68628480->68632575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.761978:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984064 is committed 00002000:00010000:3.0:1789760797.762086:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68628480 -> 68632576 00002000:00010000:3.0:1789760797.762091:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134064 -> 134072 00010000:00010000:3.0:1789760797.762104:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c745b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68628480->68632575] (req 68628480->68632575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.762117:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c745b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68628480->68632575] (req 68628480->68632575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.762126:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c745b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68628480->68632575] (req 68628480->68632575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.762134:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760797.762139:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c745b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68628480->68632575] (req 68628480->68632575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.762148:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c745b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68628480->68632575] (req 68628480->68632575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.762172:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c745b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68628480->68632575] (req 68628480->68632575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.762180:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c745b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68628480->68632575] (req 68628480->68632575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.762283:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554912640:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7719us (9444us total) trans 4294984064 rc 0/0 00000100:00100000:3.0:1789760797.762295:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16766 00000100:00100000:3.0:1789760797.762305:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740979100343 time 741000167193 state 0x2 ref 0 00000100:00100000:1.0:1789760797.781201:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760797.781334:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554912768 00000100:00100000:2.0:1789760797.781344:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554912768 00000100:00100000:2.0:1789760797.781431:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 740979100343 time 741000167193 state 0x0 ref 1 00000100:00100000:2.0:1789760797.781457:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741000427609 time 741000167193 state 0x1 ref 1 00000100:00100000:2.0:1789760797.781490:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741000427609 time 741027108575 state 0x0 ref 1 00000100:00100000:2.0:1789760797.781504:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3786e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554912768:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760797.781561:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68632576, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20bb8 00010000:00010000:2.0:1789760797.781577:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7462 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.781592:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760797.781596:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.781602:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7462 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68632576->68636671] (req 68632576->68636671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.781615:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7462 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68632576->68636671] (req 68632576->68636671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.789733:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984065 is committed 00002000:00010000:2.0:1789760797.804201:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68632576 -> 68636672 00002000:00010000:2.0:1789760797.804212:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134072 -> 134080 00010000:00010000:2.0:1789760797.804226:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7462 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68632576->68636671] (req 68632576->68636671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.804246:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7462 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68632576->68636671] (req 68632576->68636671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.804257:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7462 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68632576->68636671] (req 68632576->68636671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.804264:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.804270:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7462 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68632576->68636671] (req 68632576->68636671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.804279:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7462 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68632576->68636671] (req 68632576->68636671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.804300:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7462 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68632576->68636671] (req 68632576->68636671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.804307:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7462 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68632576->68636671] (req 68632576->68636671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.804528:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3786e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554912768:12345-192.168.203.22@tcp:4:dd.0 Request processed in 23022us (23331us total) trans 4294984065 rc 0/0 00000100:00100000:2.0:1789760797.804539:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16767 00000100:00100000:2.0:1789760797.804548:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741000427609 time 741027108575 state 0x2 ref 0 00000100:00100000:1.0:1789760797.808641:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760797.808747:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554912896 00000100:00100000:2.0:1789760797.808765:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554912896 00000100:00100000:3.0:1789760797.808868:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741000427609 time 741027108575 state 0x0 ref 1 00000100:00100000:3.0:1789760797.808883:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741027368991 time 741027108575 state 0x1 ref 1 00000100:00100000:3.0:1789760797.808916:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741027368991 time 741054534517 state 0x0 ref 1 00000100:00100000:3.0:1789760797.808928:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554912896:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760797.808982:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68636672, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20bc0 00010000:00010000:3.0:1789760797.808991:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7469 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.809007:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760797.809011:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.809024:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7469 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68636672->68640767] (req 68636672->68640767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.809038:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7469 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68636672->68640767] (req 68636672->68640767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.826991:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984066 is committed 00002000:00010000:3.0:1789760797.827154:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68636672 -> 68640768 00002000:00010000:3.0:1789760797.827160:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134080 -> 134088 00010000:00010000:3.0:1789760797.827173:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7469 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68636672->68640767] (req 68636672->68640767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.827187:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7469 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68636672->68640767] (req 68636672->68640767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.827196:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7469 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68636672->68640767] (req 68636672->68640767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.827204:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760797.827208:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7469 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68636672->68640767] (req 68636672->68640767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.827217:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7469 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68636672->68640767] (req 68636672->68640767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.827235:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7469 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68636672->68640767] (req 68636672->68640767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.827247:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7469 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68636672->68640767] (req 68636672->68640767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.827339:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554912896:12345-192.168.203.22@tcp:4:dd.0 Request processed in 18408us (18702us total) trans 4294984066 rc 0/0 00000100:00100000:3.0:1789760797.827349:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16768 00000100:00100000:3.0:1789760797.827358:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741027368991 time 741054534517 state 0x2 ref 0 00000100:00100000:3.0:1789760797.836150:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760797.836254:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554913024 00000100:00100000:0.0:1789760797.836264:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554913024 00000100:00100000:0.0:1789760797.836316:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741027368991 time 741054534517 state 0x0 ref 1 00000100:00100000:0.0:1789760797.836330:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741054794933 time 741054534517 state 0x1 ref 1 00000100:00100000:0.0:1789760797.836363:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741054794933 time 741081981728 state 0x0 ref 1 00000100:00100000:0.0:1789760797.836422:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046f9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554913024:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760797.836472:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68640768, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20bc8 00010000:00010000:0.0:1789760797.836481:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7470 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.836497:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760797.836502:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.836508:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7470 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68640768->68644863] (req 68640768->68644863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.836523:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7470 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68640768->68644863] (req 68640768->68644863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.842999:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984067 is committed 00002000:00010000:0.0:1789760797.843109:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68640768 -> 68644864 00002000:00010000:0.0:1789760797.843114:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134088 -> 134096 00010000:00010000:0.0:1789760797.843125:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7470 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68640768->68644863] (req 68640768->68644863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.843137:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7470 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68640768->68644863] (req 68640768->68644863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.843146:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7470 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68640768->68644863] (req 68640768->68644863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.843153:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760797.843158:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7470 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68640768->68644863] (req 68640768->68644863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.843167:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7470 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68640768->68644863] (req 68640768->68644863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.843185:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7470 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68640768->68644863] (req 68640768->68644863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.843192:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7470 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68640768->68644863] (req 68640768->68644863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.843284:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046f9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554913024:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6906us (7138us total) trans 4294984067 rc 0/0 00000100:00100000:0.0:1789760797.843294:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16769 00000100:00100000:0.0:1789760797.843303:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741054794933 time 741081981728 state 0x2 ref 0 00000100:00100000:1.0:1789760797.850956:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760797.851199:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554913152 00000100:00100000:0.0:1789760797.851210:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554913152 00000100:00100000:0.0:1789760797.851265:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741054794933 time 741081981728 state 0x0 ref 1 00000100:00100000:0.0:1789760797.851280:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741082242144 time 741081981728 state 0x1 ref 1 00000100:00100000:0.0:1789760797.851306:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741082242144 time 741096924413 state 0x0 ref 1 00000100:00100000:0.0:1789760797.851317:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554913152:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760797.851365:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68644864, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20bd0 00010000:00010000:0.0:1789760797.851375:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7477 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.851439:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760797.851444:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.851451:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7477 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68644864->68648959] (req 68644864->68648959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.851465:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7477 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68644864->68648959] (req 68644864->68648959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.860732:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984068 is committed 00002000:00010000:0.0:1789760797.860884:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68644864 -> 68648960 00002000:00010000:0.0:1789760797.860889:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134096 -> 134104 00010000:00010000:0.0:1789760797.860904:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7477 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68644864->68648959] (req 68644864->68648959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.860916:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7477 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68644864->68648959] (req 68644864->68648959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.860925:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7477 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68644864->68648959] (req 68644864->68648959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.860932:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760797.860936:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7477 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68644864->68648959] (req 68644864->68648959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.860956:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7477 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68644864->68648959] (req 68644864->68648959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.860975:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7477 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68644864->68648959] (req 68644864->68648959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.860982:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7477 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68644864->68648959] (req 68644864->68648959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.861082:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554913152:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9761us (10130us total) trans 4294984068 rc 0/0 00000100:00100000:0.0:1789760797.861094:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16770 00000100:00100000:0.0:1789760797.861103:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741082242144 time 741096924413 state 0x2 ref 0 00000100:00100000:1.0:1789760797.879520:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760797.879683:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554913280 00000100:00100000:2.0:1789760797.879705:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554913280 00000100:00100000:2.0:1789760797.879757:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741082242144 time 741096924413 state 0x0 ref 1 00000100:00100000:2.0:1789760797.879773:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741097184829 time 741096924413 state 0x1 ref 1 00000100:00100000:2.0:1789760797.879804:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741097184829 time 741125423030 state 0x0 ref 1 00000100:00100000:2.0:1789760797.879816:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554913280:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760797.879862:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68648960, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20bd8 00010000:00010000:2.0:1789760797.879873:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c747e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.879888:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760797.879892:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.879898:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c747e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68648960->68653055] (req 68648960->68653055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.879911:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c747e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68648960->68653055] (req 68648960->68653055) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760797.889618:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984069 is committed 00002000:00010000:2.0:1789760797.890067:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68648960 -> 68653056 00002000:00010000:2.0:1789760797.890073:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134104 -> 134112 00010000:00010000:2.0:1789760797.890085:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c747e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68648960->68653055] (req 68648960->68653055) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.890100:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c747e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68648960->68653055] (req 68648960->68653055) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.890109:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c747e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68648960->68653055] (req 68648960->68653055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.890116:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.890122:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c747e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68648960->68653055] (req 68648960->68653055) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.890131:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c747e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68648960->68653055] (req 68648960->68653055) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.890150:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c747e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68648960->68653055] (req 68648960->68653055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.890157:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c747e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68648960->68653055] (req 68648960->68653055) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.890268:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554913280:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10449us (10753us total) trans 4294984069 rc 0/0 00000100:00100000:2.0:1789760797.890279:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16771 00000100:00100000:2.0:1789760797.890289:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741097184829 time 741125423030 state 0x2 ref 0 00000100:00100000:1.0:1789760797.897222:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760797.897366:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554913408 00000100:00100000:3.0:1789760797.897490:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554913408 00000100:00100000:3.0:1789760797.897542:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741097184829 time 741125423030 state 0x0 ref 1 00000100:00100000:3.0:1789760797.897557:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741125683446 time 741125423030 state 0x1 ref 1 00000100:00100000:3.0:1789760797.897595:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741125683446 time 741143213271 state 0x0 ref 1 00000100:00100000:3.0:1789760797.897607:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554913408:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760797.897653:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68653056, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20be0 00010000:00010000:3.0:1789760797.897663:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7485 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.897678:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760797.897682:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.897689:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7485 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68653056->68657151] (req 68653056->68657151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.897703:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7485 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68653056->68657151] (req 68653056->68657151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.906670:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984070 is committed 00002000:00010000:3.0:1789760797.906789:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68653056 -> 68657152 00002000:00010000:3.0:1789760797.906795:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134112 -> 134120 00010000:00010000:3.0:1789760797.906809:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7485 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68653056->68657151] (req 68653056->68657151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.906820:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7485 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68653056->68657151] (req 68653056->68657151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.906828:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7485 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68653056->68657151] (req 68653056->68657151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.906834:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760797.906839:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7485 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68653056->68657151] (req 68653056->68657151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.906847:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7485 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68653056->68657151] (req 68653056->68657151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.906866:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7485 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68653056->68657151] (req 68653056->68657151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.906874:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7485 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68653056->68657151] (req 68653056->68657151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.907119:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554913408:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9508us (9901us total) trans 4294984070 rc 0/0 00000100:00100000:3.0:1789760797.907130:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16772 00000100:00100000:3.0:1789760797.907139:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741125683446 time 741143213271 state 0x2 ref 0 00000100:00100000:1.0:1789760797.913433:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760797.913515:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554913536 00000100:00100000:0.0:1789760797.913525:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554913536 00000100:00100000:0.0:1789760797.913589:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741125683446 time 741143213271 state 0x0 ref 1 00000100:00100000:0.0:1789760797.913604:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741143473687 time 741143213271 state 0x1 ref 1 00000100:00100000:0.0:1789760797.913721:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741143473687 time 741159337917 state 0x0 ref 1 00000100:00100000:0.0:1789760797.913733:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554913536:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760797.913786:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68657152, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20be8 00010000:00010000:0.0:1789760797.913795:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c748c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.913811:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760797.913815:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.913821:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c748c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68657152->68661247] (req 68657152->68661247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.913835:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c748c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68657152->68661247] (req 68657152->68661247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.920438:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984071 is committed 00002000:00010000:0.0:1789760797.921006:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68657152 -> 68661248 00002000:00010000:0.0:1789760797.921012:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134120 -> 134128 00010000:00010000:0.0:1789760797.921025:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c748c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68657152->68661247] (req 68657152->68661247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.921037:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c748c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68657152->68661247] (req 68657152->68661247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.921045:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c748c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68657152->68661247] (req 68657152->68661247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.921052:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760797.921057:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c748c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68657152->68661247] (req 68657152->68661247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.921066:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c748c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68657152->68661247] (req 68657152->68661247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.921086:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c748c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68657152->68661247] (req 68657152->68661247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.921094:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c748c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68657152->68661247] (req 68657152->68661247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.921182:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554913536:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7447us (7754us total) trans 4294984071 rc 0/0 00000100:00100000:0.0:1789760797.921192:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16773 00000100:00100000:0.0:1789760797.921202:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741143473687 time 741159337917 state 0x2 ref 0 00000100:00100000:1.0:1789760797.926621:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760797.926707:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554913664 00000100:00100000:0.0:1789760797.926716:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554913664 00000100:00100000:0.0:1789760797.926775:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741143473687 time 741159337917 state 0x0 ref 1 00000100:00100000:0.0:1789760797.926789:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741159598333 time 741159337917 state 0x1 ref 1 00000100:00100000:0.0:1789760797.926817:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741159598333 time 741172435995 state 0x0 ref 1 00000100:00100000:0.0:1789760797.926828:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554913664:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760797.926871:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68661248, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20bf0 00010000:00010000:0.0:1789760797.926879:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7493 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.926895:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760797.926899:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.926905:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7493 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68661248->68665343] (req 68661248->68665343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.926919:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7493 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68661248->68665343] (req 68661248->68665343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.931871:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984072 is committed 00002000:00010000:0.0:1789760797.931977:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68661248 -> 68665344 00002000:00010000:0.0:1789760797.931983:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134128 -> 134136 00010000:00010000:0.0:1789760797.931995:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7493 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68661248->68665343] (req 68661248->68665343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.932022:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7493 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68661248->68665343] (req 68661248->68665343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.932032:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7493 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68661248->68665343] (req 68661248->68665343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.932039:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760797.932044:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7493 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68661248->68665343] (req 68661248->68665343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.932054:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7493 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68661248->68665343] (req 68661248->68665343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.932073:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7493 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68661248->68665343] (req 68661248->68665343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.932086:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c7493 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68661248->68665343] (req 68661248->68665343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.932174:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554913664:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5342us (5558us total) trans 4294984072 rc 0/0 00000100:00100000:0.0:1789760797.932184:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16774 00000100:00100000:0.0:1789760797.932193:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741159598333 time 741172435995 state 0x2 ref 0 00000100:00100000:1.0:1789760797.936540:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760797.936639:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554913792 00000100:00100000:2.0:1789760797.936649:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554913792 00000100:00100000:2.0:1789760797.936702:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741159598333 time 741172435995 state 0x0 ref 1 00000100:00100000:2.0:1789760797.936718:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741172696411 time 741172435995 state 0x1 ref 1 00000100:00100000:2.0:1789760797.936752:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741172696411 time 741182370297 state 0x0 ref 1 00000100:00100000:2.0:1789760797.936764:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554913792:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760797.936814:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68665344, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20bf8 00010000:00010000:2.0:1789760797.936824:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c749a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.936840:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760797.936844:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.936850:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c749a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68665344->68669439] (req 68665344->68669439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.936864:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c749a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68665344->68669439] (req 68665344->68669439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.943032:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984073 is committed 00002000:00010000:2.0:1789760797.943290:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68665344 -> 68669440 00002000:00010000:2.0:1789760797.943295:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134136 -> 134144 00010000:00010000:2.0:1789760797.943308:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c749a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68665344->68669439] (req 68665344->68669439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.943331:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c749a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68665344->68669439] (req 68665344->68669439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.943347:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c749a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68665344->68669439] (req 68665344->68669439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.943354:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.943359:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c749a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68665344->68669439] (req 68665344->68669439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.943368:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c749a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68665344->68669439] (req 68665344->68669439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.943416:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c749a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68665344->68669439] (req 68665344->68669439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.943425:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c749a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68665344->68669439] (req 68665344->68669439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.943551:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554913792:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6784us (7016us total) trans 4294984073 rc 0/0 00000100:00100000:2.0:1789760797.943561:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16775 00000100:00100000:2.0:1789760797.943578:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741172696411 time 741182370297 state 0x2 ref 0 00000100:00100000:1.0:1789760797.957136:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760797.957243:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554913920 00000100:00100000:3.0:1789760797.957254:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554913920 00000100:00100000:3.0:1789760797.957310:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741172696411 time 741182370297 state 0x0 ref 1 00000100:00100000:3.0:1789760797.957326:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741182630713 time 741182370297 state 0x1 ref 1 00000100:00100000:3.0:1789760797.957814:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741182630713 time 741203430414 state 0x0 ref 1 00000100:00100000:3.0:1789760797.957829:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554913920:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760797.957882:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68669440, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20c00 00010000:00010000:3.0:1789760797.957891:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74a1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.957908:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760797.957912:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760797.957919:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68669440->68673535] (req 68669440->68673535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.957933:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68669440->68673535] (req 68669440->68673535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.963997:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984074 is committed 00002000:00010000:3.0:1789760797.964167:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68669440 -> 68673536 00002000:00010000:3.0:1789760797.964173:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134144 -> 134152 00010000:00010000:3.0:1789760797.964189:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74a1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68669440->68673535] (req 68669440->68673535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.964203:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74a1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68669440->68673535] (req 68669440->68673535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.964213:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68669440->68673535] (req 68669440->68673535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.964221:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760797.964226:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74a1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68669440->68673535] (req 68669440->68673535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.964235:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74a1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68669440->68673535] (req 68669440->68673535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.964255:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74a1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68669440->68673535] (req 68669440->68673535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760797.964264:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74a1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68669440->68673535] (req 68669440->68673535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760797.964433:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554913920:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6590us (7289us total) trans 4294984074 rc 0/0 00000100:00100000:3.0:1789760797.964445:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16776 00000100:00100000:3.0:1789760797.964454:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741182630713 time 741203430414 state 0x2 ref 0 00000100:00100000:1.0:1789760797.968477:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760797.968553:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554914048 00000100:00100000:0.0:1789760797.968577:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554914048 00000100:00100000:0.0:1789760797.968630:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741182630713 time 741203430414 state 0x0 ref 1 00000100:00100000:0.0:1789760797.968645:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741203690830 time 741203430414 state 0x1 ref 1 00000100:00100000:0.0:1789760797.968674:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741203690830 time 741214292220 state 0x0 ref 1 00000100:00100000:0.0:1789760797.968686:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554914048:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760797.968732:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68673536, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20c08 00010000:00010000:0.0:1789760797.968742:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74a8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.968757:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760797.968762:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.968768:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68673536->68677631] (req 68673536->68677631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.968781:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68673536->68677631] (req 68673536->68677631) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.973773:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984075 is committed 00002000:00010000:0.0:1789760797.973926:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68673536 -> 68677632 00002000:00010000:0.0:1789760797.973932:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134152 -> 134160 00010000:00010000:0.0:1789760797.974005:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74a8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68673536->68677631] (req 68673536->68677631) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.974019:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74a8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68673536->68677631] (req 68673536->68677631) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.974028:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68673536->68677631] (req 68673536->68677631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.974034:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760797.974039:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74a8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68673536->68677631] (req 68673536->68677631) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.974048:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74a8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68673536->68677631] (req 68673536->68677631) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.974071:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74a8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68673536->68677631] (req 68673536->68677631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.974080:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74a8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68673536->68677631] (req 68673536->68677631) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.974208:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554914048:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5519us (5736us total) trans 4294984075 rc 0/0 00000100:00100000:0.0:1789760797.974220:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16777 00000100:00100000:0.0:1789760797.974229:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741203690830 time 741214292220 state 0x2 ref 0 00000100:00100000:1.0:1789760797.979080:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760797.979165:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554914176 00000100:00100000:0.0:1789760797.979174:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554914176 00000100:00100000:0.0:1789760797.979227:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741203690830 time 741214292220 state 0x0 ref 1 00000100:00100000:0.0:1789760797.979243:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741214552636 time 741214292220 state 0x1 ref 1 00000100:00100000:0.0:1789760797.979270:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741214552636 time 741224888407 state 0x0 ref 1 00000100:00100000:0.0:1789760797.979282:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb049d4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554914176:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760797.979341:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68677632, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20c10 00010000:00010000:0.0:1789760797.979350:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c74af lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.979366:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760797.979371:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.979406:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c74af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68677632->68681727] (req 68677632->68681727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.979421:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c74af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68677632->68681727] (req 68677632->68681727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760797.983704:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984076 is committed 00002000:00010000:0.0:1789760797.983859:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68677632 -> 68681728 00002000:00010000:0.0:1789760797.983865:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134160 -> 134168 00010000:00010000:0.0:1789760797.984000:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c74af lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68677632->68681727] (req 68677632->68681727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.984012:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c74af lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68677632->68681727] (req 68677632->68681727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.984021:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c74af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68677632->68681727] (req 68677632->68681727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.984033:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760797.984038:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c74af lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68677632->68681727] (req 68677632->68681727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.984048:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c74af lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68677632->68681727] (req 68677632->68681727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.984067:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c74af lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68677632->68681727] (req 68677632->68681727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.984075:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c74af lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68677632->68681727] (req 68677632->68681727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760797.984206:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb049d4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554914176:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4920us (5130us total) trans 4294984076 rc 0/0 00000100:00100000:0.0:1789760797.984217:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16778 00000100:00100000:0.0:1789760797.984226:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741214552636 time 741224888407 state 0x2 ref 0 00000100:00100000:1.0:1789760797.988925:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760797.989097:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554914304 00000100:00100000:2.0:1789760797.989106:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554914304 00000100:00100000:2.0:1789760797.989153:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741214552636 time 741224888407 state 0x0 ref 1 00000100:00100000:2.0:1789760797.989168:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741225148823 time 741224888407 state 0x1 ref 1 00000100:00100000:2.0:1789760797.989218:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741225148823 time 741234835972 state 0x0 ref 1 00000100:00100000:2.0:1789760797.989229:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554914304:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760797.989270:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68681728, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20c18 00010000:00010000:2.0:1789760797.989278:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74b6 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.989293:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760797.989298:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760797.989303:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68681728->68685823] (req 68681728->68685823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.989316:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68681728->68685823] (req 68681728->68685823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760797.993073:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984077 is committed 00002000:00010000:2.0:1789760797.993334:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68681728 -> 68685824 00002000:00010000:2.0:1789760797.993339:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134168 -> 134176 00010000:00010000:2.0:1789760797.993351:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74b6 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68681728->68685823] (req 68681728->68685823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.993365:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74b6 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68681728->68685823] (req 68681728->68685823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.993545:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68681728->68685823] (req 68681728->68685823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.993553:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760797.993558:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74b6 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68681728->68685823] (req 68681728->68685823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.993576:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74b6 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68681728->68685823] (req 68681728->68685823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.993596:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74b6 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68681728->68685823] (req 68681728->68685823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760797.993605:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74b6 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68681728->68685823] (req 68681728->68685823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760797.993738:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554914304:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4504us (4815us total) trans 4294984077 rc 0/0 00000100:00100000:2.0:1789760797.993748:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16779 00000100:00100000:2.0:1789760797.993757:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741225148823 time 741234835972 state 0x2 ref 0 00000100:00100000:1.0:1789760797.996255:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760797.996445:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554914432 00000100:00100000:3.0:1789760797.996456:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554914432 00000100:00100000:3.0:1789760797.996512:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741225148823 time 741234835972 state 0x0 ref 1 00000100:00100000:3.0:1789760797.996527:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741235096388 time 741234835972 state 0x1 ref 1 00000100:00100000:0.0:1789760797.997126:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741235096388 time 741242740573 state 0x0 ref 1 00000100:00100000:0.0:1789760797.997144:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554914432:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760797.997190:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68685824, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20c20 00010000:00010000:0.0:1789760797.997256:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c74bd lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.997272:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760797.997412:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760797.997420:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c74bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68685824->68689919] (req 68685824->68689919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760797.997827:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c74bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68685824->68689919] (req 68685824->68689919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.002646:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984078 is committed 00002000:00010000:3.0:1789760798.002774:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68685824 -> 68689920 00002000:00010000:3.0:1789760798.002780:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134176 -> 134184 00010000:00010000:3.0:1789760798.002794:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c74bd lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68685824->68689919] (req 68685824->68689919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.002807:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c74bd lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68685824->68689919] (req 68685824->68689919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.002817:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c74bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68685824->68689919] (req 68685824->68689919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.002824:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760798.002829:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c74bd lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68685824->68689919] (req 68685824->68689919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.002839:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c74bd lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68685824->68689919] (req 68685824->68689919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.002875:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c74bd lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68685824->68689919] (req 68685824->68689919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.002884:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aa6892cd/0x1f99687d854c74bd lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68685824->68689919] (req 68685824->68689919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.002986:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554914432:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5842us (6738us total) trans 4294984078 rc 0/0 00000100:00100000:3.0:1789760798.002997:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16780 00000100:00100000:3.0:1789760798.003007:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741235096388 time 741242740573 state 0x2 ref 0 00000100:00100000:1.0:1789760798.007509:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760798.007856:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554914560 00000100:00100000:0.0:1789760798.007876:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554914560 00000100:00100000:0.0:1789760798.007933:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741235096388 time 741242740573 state 0x0 ref 1 00000100:00100000:0.0:1789760798.008138:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741243000989 time 741242740573 state 0x1 ref 1 00000100:00100000:0.0:1789760798.008174:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741243000989 time 741253792969 state 0x0 ref 1 00000100:00100000:0.0:1789760798.008187:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554914560:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760798.008234:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68689920, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20c28 00010000:00010000:0.0:1789760798.008243:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74c4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.008259:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760798.008263:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.008269:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68689920->68694015] (req 68689920->68694015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.008284:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68689920->68694015] (req 68689920->68694015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.019391:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984079 is committed 00002000:00010000:2.0:1789760798.019588:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68689920 -> 68694016 00002000:00010000:2.0:1789760798.019594:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134184 -> 134192 00010000:00010000:2.0:1789760798.019608:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74c4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68689920->68694015] (req 68689920->68694015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.019625:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74c4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68689920->68694015] (req 68689920->68694015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.019634:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68689920->68694015] (req 68689920->68694015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.019641:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760798.019646:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74c4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68689920->68694015] (req 68689920->68694015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.019655:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74c4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68689920->68694015] (req 68689920->68694015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.019675:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74c4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68689920->68694015] (req 68689920->68694015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.019683:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74c4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68689920->68694015] (req 68689920->68694015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.019834:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554914560:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11645us (12331us total) trans 4294984079 rc 0/0 00000100:00100000:2.0:1789760798.019845:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16781 00000100:00100000:2.0:1789760798.019855:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741243000989 time 741253792969 state 0x2 ref 0 00000100:00100000:1.0:1789760798.025210:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760798.025299:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554914688 00000100:00100000:3.0:1789760798.025308:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554914688 00000100:00100000:3.0:1789760798.025368:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741243000989 time 741253792969 state 0x0 ref 1 00000100:00100000:3.0:1789760798.025401:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741254053385 time 741253792969 state 0x1 ref 1 00000100:00100000:3.0:1789760798.025431:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741254053385 time 741271050094 state 0x0 ref 1 00000100:00100000:3.0:1789760798.025444:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554914688:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760798.025490:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68694016, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20c30 00010000:00010000:3.0:1789760798.025499:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74cb lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.025514:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760798.025519:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.025525:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68694016->68698111] (req 68694016->68698111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.025538:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68694016->68698111] (req 68694016->68698111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.031920:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984080 is committed 00002000:00010000:3.0:1789760798.032099:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68694016 -> 68698112 00002000:00010000:3.0:1789760798.032104:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134192 -> 134200 00010000:00010000:3.0:1789760798.032119:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74cb lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68694016->68698111] (req 68694016->68698111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.032133:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74cb lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68694016->68698111] (req 68694016->68698111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.032142:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68694016->68698111] (req 68694016->68698111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.032149:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760798.032154:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74cb lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68694016->68698111] (req 68694016->68698111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.032164:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74cb lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68694016->68698111] (req 68694016->68698111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.032184:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74cb lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68694016->68698111] (req 68694016->68698111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.032198:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74cb lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68694016->68698111] (req 68694016->68698111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.032292:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554914688:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6845us (7086us total) trans 4294984080 rc 0/0 00000100:00100000:3.0:1789760798.032303:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16782 00000100:00100000:3.0:1789760798.032312:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741254053385 time 741271050094 state 0x2 ref 0 00000100:00100000:1.0:1789760798.036204:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760798.036579:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554914816 00000100:00100000:2.0:1789760798.036589:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554914816 00000100:00100000:2.0:1789760798.036645:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741254053385 time 741271050094 state 0x0 ref 1 00000100:00100000:2.0:1789760798.036659:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741271310510 time 741271050094 state 0x1 ref 1 00000100:00100000:2.0:1789760798.036690:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741271310510 time 741282308743 state 0x0 ref 1 00000100:00100000:2.0:1789760798.036703:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554914816:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760798.036750:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68698112, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20c38 00010000:00010000:2.0:1789760798.036759:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74d2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.036774:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760798.036778:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.036783:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68698112->68702207] (req 68698112->68702207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.036797:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68698112->68702207] (req 68698112->68702207) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.041766:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984081 is committed 00002000:00010000:2.0:1789760798.041874:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68698112 -> 68702208 00002000:00010000:2.0:1789760798.041879:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134200 -> 134208 00010000:00010000:2.0:1789760798.041892:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74d2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68698112->68702207] (req 68698112->68702207) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.041904:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74d2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68698112->68702207] (req 68698112->68702207) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.041920:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68698112->68702207] (req 68698112->68702207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.041928:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760798.041933:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74d2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68698112->68702207] (req 68698112->68702207) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.041950:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74d2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68698112->68702207] (req 68698112->68702207) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.041969:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74d2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68698112->68702207] (req 68698112->68702207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.041976:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74d2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68698112->68702207] (req 68698112->68702207) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.042070:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554914816:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5365us (5874us total) trans 4294984081 rc 0/0 00000100:00100000:2.0:1789760798.042080:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16783 00000100:00100000:2.0:1789760798.042090:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741271310510 time 741282308743 state 0x2 ref 0 00000100:00100000:1.0:1789760798.045089:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760798.045850:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554914944 00000100:00100000:0.0:1789760798.045861:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554914944 00000100:00100000:0.0:1789760798.045913:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741271310510 time 741282308743 state 0x0 ref 1 00000100:00100000:0.0:1789760798.045929:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741282569159 time 741282308743 state 0x1 ref 1 00000100:00100000:0.0:1789760798.045968:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741282569159 time 741291586530 state 0x0 ref 1 00000100:00100000:0.0:1789760798.045980:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554914944:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760798.046039:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68702208, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20c40 00010000:00010000:0.0:1789760798.046048:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74d9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.046064:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760798.046068:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.046075:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68702208->68706303] (req 68702208->68706303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.046088:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68702208->68706303] (req 68702208->68706303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.050901:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984082 is committed 00002000:00010000:0.0:1789760798.051028:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68702208 -> 68706304 00002000:00010000:0.0:1789760798.051034:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134208 -> 134216 00010000:00010000:0.0:1789760798.051047:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74d9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68702208->68706303] (req 68702208->68706303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.051059:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74d9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68702208->68706303] (req 68702208->68706303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.051068:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68702208->68706303] (req 68702208->68706303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.051075:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760798.051080:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74d9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68702208->68706303] (req 68702208->68706303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.051089:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74d9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68702208->68706303] (req 68702208->68706303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.051108:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74d9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68702208->68706303] (req 68702208->68706303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.051116:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74d9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68702208->68706303] (req 68702208->68706303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.051237:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554914944:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5255us (6154us total) trans 4294984082 rc 0/0 00000100:00100000:0.0:1789760798.051249:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16784 00000100:00100000:0.0:1789760798.051259:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741282569159 time 741291586530 state 0x2 ref 0 00000100:00100000:1.0:1789760798.055192:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760798.055254:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554915072 00000100:00100000:0.0:1789760798.055264:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554915072 00000100:00100000:0.0:1789760798.055534:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741282569159 time 741291586530 state 0x0 ref 1 00000100:00100000:0.0:1789760798.055552:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741291846946 time 741291586530 state 0x1 ref 1 00000100:00100000:0.0:1789760798.055622:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741291846946 time 741301240519 state 0x0 ref 1 00000100:00100000:0.0:1789760798.055636:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554915072:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760798.055681:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68706304, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20c48 00010000:00010000:0.0:1789760798.055691:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74e0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.055708:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760798.055713:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.055724:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68706304->68710399] (req 68706304->68710399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.055738:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68706304->68710399] (req 68706304->68710399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.062253:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984083 is committed 00002000:00010000:0.0:1789760798.062656:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68706304 -> 68710400 00002000:00010000:0.0:1789760798.062661:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134216 -> 134224 00010000:00010000:0.0:1789760798.062676:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74e0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68706304->68710399] (req 68706304->68710399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.062688:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74e0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68706304->68710399] (req 68706304->68710399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.062700:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68706304->68710399] (req 68706304->68710399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.062707:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760798.062711:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74e0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68706304->68710399] (req 68706304->68710399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.062719:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74e0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68706304->68710399] (req 68706304->68710399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.062737:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74e0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68706304->68710399] (req 68706304->68710399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.062744:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74e0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68706304->68710399] (req 68706304->68710399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.062836:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554915072:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7198us (7648us total) trans 4294984083 rc 0/0 00000100:00100000:0.0:1789760798.062846:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16785 00000100:00100000:0.0:1789760798.062855:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741291846946 time 741301240519 state 0x2 ref 0 00000100:00100000:1.0:1789760798.066458:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760798.066662:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554915200 00000100:00100000:3.0:1789760798.066672:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554915200 00000100:00100000:3.0:1789760798.066728:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741291846946 time 741301240519 state 0x0 ref 1 00000100:00100000:3.0:1789760798.066743:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741301500935 time 741301240519 state 0x1 ref 1 00000100:00100000:3.0:1789760798.066802:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741301500935 time 741312420153 state 0x0 ref 1 00000100:00100000:3.0:1789760798.066816:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554915200:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760798.066860:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68710400, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20c50 00010000:00010000:3.0:1789760798.066879:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74e7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.066894:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760798.066898:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.066904:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68710400->68714495] (req 68710400->68714495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.066922:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68710400->68714495] (req 68710400->68714495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.071667:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984084 is committed 00002000:00010000:3.0:1789760798.071760:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68710400 -> 68714496 00002000:00010000:3.0:1789760798.071765:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134224 -> 134232 00010000:00010000:3.0:1789760798.071779:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74e7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68710400->68714495] (req 68710400->68714495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.071791:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74e7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68710400->68714495] (req 68710400->68714495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.071801:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68710400->68714495] (req 68710400->68714495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.071809:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760798.071814:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74e7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68710400->68714495] (req 68710400->68714495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.071823:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74e7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68710400->68714495] (req 68710400->68714495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.071843:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74e7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68710400->68714495] (req 68710400->68714495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.071851:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74e7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68710400->68714495] (req 68710400->68714495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.072077:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554915200:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5256us (5619us total) trans 4294984084 rc 0/0 00000100:00100000:3.0:1789760798.072088:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16786 00000100:00100000:3.0:1789760798.072098:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741301500935 time 741312420153 state 0x2 ref 0 00000100:00100000:1.0:1789760798.075823:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760798.076029:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554915328 00000100:00100000:2.0:1789760798.076039:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554915328 00000100:00100000:2.0:1789760798.076092:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741301500935 time 741312420153 state 0x0 ref 1 00000100:00100000:2.0:1789760798.076108:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741312680569 time 741312420153 state 0x1 ref 1 00000100:00100000:2.0:1789760798.076188:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741312680569 time 741321805943 state 0x0 ref 1 00000100:00100000:2.0:1789760798.076200:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554915328:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760798.076245:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68714496, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20c58 00010000:00010000:2.0:1789760798.076254:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74ee lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.076269:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760798.076273:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.076280:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68714496->68718591] (req 68714496->68718591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.076294:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68714496->68718591] (req 68714496->68718591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.080989:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984085 is committed 00002000:00010000:2.0:1789760798.081111:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68714496 -> 68718592 00002000:00010000:2.0:1789760798.081116:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134232 -> 134240 00010000:00010000:2.0:1789760798.081130:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74ee lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68714496->68718591] (req 68714496->68718591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.081143:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74ee lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68714496->68718591] (req 68714496->68718591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.081153:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68714496->68718591] (req 68714496->68718591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.081168:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760798.081173:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74ee lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68714496->68718591] (req 68714496->68718591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.081182:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74ee lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68714496->68718591] (req 68714496->68718591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.081202:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74ee lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68714496->68718591] (req 68714496->68718591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.081211:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c74ee lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68714496->68718591] (req 68714496->68718591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.081720:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554915328:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5119us (5505us total) trans 4294984085 rc 0/0 00000100:00100000:2.0:1789760798.081732:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16787 00000100:00100000:2.0:1789760798.081742:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741312680569 time 741321805943 state 0x2 ref 0 00000100:00100000:1.0:1789760798.087835:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760798.088263:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554915456 00000100:00100000:0.0:1789760798.088273:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554915456 00000100:00100000:0.0:1789760798.088348:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741312680569 time 741321805943 state 0x0 ref 1 00000100:00100000:0.0:1789760798.088363:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741322066359 time 741321805943 state 0x1 ref 1 00000100:00100000:0.0:1789760798.088424:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741322066359 time 741334041672 state 0x0 ref 1 00000100:00100000:0.0:1789760798.088436:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554915456:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760798.088483:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68718592, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20c60 00010000:00010000:0.0:1789760798.088493:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74f5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.088508:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760798.088513:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.088519:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68718592->68722687] (req 68718592->68722687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.088540:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68718592->68722687] (req 68718592->68722687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.096141:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984086 is committed 00002000:00010000:3.0:1789760798.096471:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68718592 -> 68722688 00002000:00010000:3.0:1789760798.096476:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134240 -> 134248 00010000:00010000:3.0:1789760798.096490:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74f5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68718592->68722687] (req 68718592->68722687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.096503:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74f5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68718592->68722687] (req 68718592->68722687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.096512:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68718592->68722687] (req 68718592->68722687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.096519:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760798.096524:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74f5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68718592->68722687] (req 68718592->68722687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.096533:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74f5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68718592->68722687] (req 68718592->68722687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.096553:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74f5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68718592->68722687] (req 68718592->68722687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.096576:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009597cd3a/0x1f99687d854c74f5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68718592->68722687] (req 68718592->68722687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.096678:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554915456:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8232us (8840us total) trans 4294984086 rc 0/0 00000100:00100000:3.0:1789760798.096689:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16788 00000100:00100000:3.0:1789760798.096698:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741322066359 time 741334041672 state 0x2 ref 0 00000100:00100000:1.0:1789760798.105180:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760798.105270:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554915584 00000100:00100000:3.0:1789760798.105280:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554915584 00000100:00100000:3.0:1789760798.105348:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741322066359 time 741334041672 state 0x0 ref 1 00000100:00100000:3.0:1789760798.105363:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741334302088 time 741334041672 state 0x1 ref 1 00000100:00100000:3.0:1789760798.105435:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741334302088 time 741351053137 state 0x0 ref 1 00000100:00100000:3.0:1789760798.105449:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554915584:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760798.105503:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68722688, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20c68 00010000:00010000:3.0:1789760798.105513:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74fc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.105527:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760798.105531:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.105537:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68722688->68726783] (req 68722688->68726783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.105556:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68722688->68726783] (req 68722688->68726783) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.113235:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984087 is committed 00002000:00010000:3.0:1789760798.113366:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68722688 -> 68726784 00002000:00010000:3.0:1789760798.113373:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134248 -> 134256 00010000:00010000:3.0:1789760798.113419:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74fc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68722688->68726783] (req 68722688->68726783) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.113435:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74fc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68722688->68726783] (req 68722688->68726783) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.113447:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68722688->68726783] (req 68722688->68726783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.113466:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760798.113471:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74fc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68722688->68726783] (req 68722688->68726783) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.113482:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74fc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68722688->68726783] (req 68722688->68726783) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.113503:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74fc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68722688->68726783] (req 68722688->68726783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.113768:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c74fc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68722688->68726783] (req 68722688->68726783) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.113868:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554915584:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8417us (9541us total) trans 4294984087 rc 0/0 00000100:00100000:3.0:1789760798.113880:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16789 00000100:00100000:3.0:1789760798.113890:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741334302088 time 741351053137 state 0x2 ref 0 00000100:00100000:0.0:1789760798.119619:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760798.119722:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554915712 00000100:00100000:3.0:1789760798.119731:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554915712 00000100:00100000:3.0:1789760798.119785:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741334302088 time 741351053137 state 0x0 ref 1 00000100:00100000:3.0:1789760798.119800:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741351313553 time 741351053137 state 0x1 ref 1 00000100:00100000:3.0:1789760798.119836:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741351313553 time 741365454275 state 0x0 ref 1 00000100:00100000:3.0:1789760798.119848:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a387480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554915712:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760798.119894:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68726784, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20c70 00010000:00010000:3.0:1789760798.119903:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7503 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.119920:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760798.119941:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.119947:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7503 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68726784->68730879] (req 68726784->68730879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.119961:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7503 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68726784->68730879] (req 68726784->68730879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.125867:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984088 is committed 00002000:00010000:3.0:1789760798.125989:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68726784 -> 68730880 00002000:00010000:3.0:1789760798.125995:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134256 -> 134264 00010000:00010000:3.0:1789760798.126008:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7503 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68726784->68730879] (req 68726784->68730879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.126021:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7503 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68726784->68730879] (req 68726784->68730879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.126029:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7503 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68726784->68730879] (req 68726784->68730879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.126036:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760798.126041:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7503 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68726784->68730879] (req 68726784->68730879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.126050:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7503 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68726784->68730879] (req 68726784->68730879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.126069:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7503 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68726784->68730879] (req 68726784->68730879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.126078:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7503 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68726784->68730879] (req 68726784->68730879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.126169:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a387480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554915712:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6317us (6553us total) trans 4294984088 rc 0/0 00000100:00100000:3.0:1789760798.126181:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16790 00000100:00100000:3.0:1789760798.126191:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741351313553 time 741365454275 state 0x2 ref 0 00000100:00100000:1.0:1789760798.135747:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760798.135820:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554915840 00000100:00100000:3.0:1789760798.135831:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554915840 00000100:00100000:3.0:1789760798.135887:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741351313553 time 741365454275 state 0x0 ref 1 00000100:00100000:3.0:1789760798.135902:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741365714691 time 741365454275 state 0x1 ref 1 00000100:00100000:3.0:1789760798.135946:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741365714691 time 741381555218 state 0x0 ref 1 00000100:00100000:3.0:1789760798.135960:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554915840:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760798.136009:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68730880, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20c78 00010000:00010000:3.0:1789760798.136018:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c750a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.136035:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760798.136040:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.136047:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c750a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68730880->68734975] (req 68730880->68734975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.136061:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c750a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68730880->68734975] (req 68730880->68734975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.140906:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984089 is committed 00002000:00010000:3.0:1789760798.144283:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68730880 -> 68734976 00002000:00010000:3.0:1789760798.144292:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134264 -> 134272 00010000:00010000:3.0:1789760798.144306:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c750a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68730880->68734975] (req 68730880->68734975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.144445:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c750a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68730880->68734975] (req 68730880->68734975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.144459:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c750a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68730880->68734975] (req 68730880->68734975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.144467:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760798.144472:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c750a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68730880->68734975] (req 68730880->68734975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.144482:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c750a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68730880->68734975] (req 68730880->68734975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.144503:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c750a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68730880->68734975] (req 68730880->68734975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.144511:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c750a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68730880->68734975] (req 68730880->68734975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.144722:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098df800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554915840:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8758us (8978us total) trans 4294984089 rc 0/0 00000100:00100000:2.0:1789760798.144733:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16791 00000100:00100000:2.0:1789760798.144743:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741365714691 time 741381555218 state 0x2 ref 0 00000100:00100000:0.0:1789760798.148169:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760798.148286:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554915968 00000100:00100000:3.0:1789760798.148295:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554915968 00000100:00100000:3.0:1789760798.148360:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741365714691 time 741381555218 state 0x0 ref 1 00000100:00100000:3.0:1789760798.148507:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741381815634 time 741381555218 state 0x1 ref 1 00000100:00100000:2.0:1789760798.148624:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741381815634 time 741394240668 state 0x0 ref 1 00000100:00100000:2.0:1789760798.148643:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a384e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554915968:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760798.148690:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68734976, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20c80 00010000:00010000:2.0:1789760798.148700:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7511 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.148718:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760798.148723:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.148729:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7511 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68734976->68739071] (req 68734976->68739071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.148742:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7511 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68734976->68739071] (req 68734976->68739071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.152931:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984090 is committed 00002000:00010000:2.0:1789760798.153081:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68734976 -> 68739072 00002000:00010000:2.0:1789760798.153086:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134272 -> 134280 00010000:00010000:2.0:1789760798.153099:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7511 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68734976->68739071] (req 68734976->68739071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.153112:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7511 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68734976->68739071] (req 68734976->68739071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.153121:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7511 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68734976->68739071] (req 68734976->68739071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.153130:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760798.153135:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7511 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68734976->68739071] (req 68734976->68739071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.153144:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7511 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68734976->68739071] (req 68734976->68739071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.153163:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7511 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68734976->68739071] (req 68734976->68739071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.153171:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7511 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68734976->68739071] (req 68734976->68739071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.153261:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a384e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554915968:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4618us (5097us total) trans 4294984090 rc 0/0 00000100:00100000:2.0:1789760798.153272:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16792 00000100:00100000:2.0:1789760798.153282:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741381815634 time 741394240668 state 0x2 ref 0 00000100:00100000:0.0:1789760798.157085:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760798.157156:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554916096 00000100:00100000:3.0:1789760798.157165:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554916096 00000100:00100000:3.0:1789760798.157213:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741381815634 time 741394240668 state 0x0 ref 1 00000100:00100000:3.0:1789760798.157227:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741394501084 time 741394240668 state 0x1 ref 1 00000100:00100000:3.0:1789760798.157253:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741394501084 time 741402872102 state 0x0 ref 1 00000100:00100000:3.0:1789760798.157264:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554916096:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760798.157311:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68739072, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20c88 00010000:00010000:3.0:1789760798.157329:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7518 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.157346:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760798.157351:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.157357:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7518 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68739072->68743167] (req 68739072->68743167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.157371:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7518 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68739072->68743167] (req 68739072->68743167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.165973:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984091 is committed 00002000:00010000:2.0:1789760798.166532:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68739072 -> 68743168 00002000:00010000:2.0:1789760798.166538:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134280 -> 134288 00010000:00010000:2.0:1789760798.166552:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7518 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68739072->68743167] (req 68739072->68743167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.166573:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7518 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68739072->68743167] (req 68739072->68743167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.166582:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7518 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68739072->68743167] (req 68739072->68743167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.166594:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760798.166599:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7518 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68739072->68743167] (req 68739072->68743167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.166608:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7518 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68739072->68743167] (req 68739072->68743167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.166627:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7518 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68739072->68743167] (req 68739072->68743167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.166635:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a755e2ae/0x1f99687d854c7518 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68739072->68743167] (req 68739072->68743167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.166761:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554916096:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9493us (9680us total) trans 4294984091 rc 0/0 00000100:00100000:2.0:1789760798.166772:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16793 00000100:00100000:2.0:1789760798.166782:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741394501084 time 741402872102 state 0x2 ref 0 00000100:00100000:1.0:1789760798.174113:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760798.174264:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554916224 00000100:00100000:2.0:1789760798.174275:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554916224 00000100:00100000:2.0:1789760798.174406:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741394501084 time 741402872102 state 0x0 ref 1 00000100:00100000:2.0:1789760798.174424:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741403132518 time 741402872102 state 0x1 ref 1 00000100:00100000:2.0:1789760798.174447:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741403132518 time 741420066031 state 0x0 ref 1 00000100:00100000:2.0:1789760798.174462:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554916224:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760798.174506:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68743168, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20c90 00010000:00010000:2.0:1789760798.174515:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c751f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.174530:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760798.174534:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.174540:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c751f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68743168->68747263] (req 68743168->68747263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.174554:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c751f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68743168->68747263] (req 68743168->68747263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.179650:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984092 is committed 00002000:00010000:2.0:1789760798.179767:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68743168 -> 68747264 00002000:00010000:2.0:1789760798.179772:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134288 -> 134296 00010000:00010000:2.0:1789760798.179786:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c751f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68743168->68747263] (req 68743168->68747263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.179799:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c751f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68743168->68747263] (req 68743168->68747263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.179809:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c751f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68743168->68747263] (req 68743168->68747263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.179817:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760798.179823:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c751f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68743168->68747263] (req 68743168->68747263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.179831:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c751f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68743168->68747263] (req 68743168->68747263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.179850:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c751f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68743168->68747263] (req 68743168->68747263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.179859:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c751f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68743168->68747263] (req 68743168->68747263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.180063:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554916224:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5597us (5954us total) trans 4294984092 rc 0/0 00000100:00100000:2.0:1789760798.180074:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16794 00000100:00100000:2.0:1789760798.180083:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741403132518 time 741420066031 state 0x2 ref 0 00000100:00100000:0.0:1789760798.184476:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760798.184554:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554916352 00000100:00100000:2.0:1789760798.184573:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554916352 00000100:00100000:2.0:1789760798.184627:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741403132518 time 741420066031 state 0x0 ref 1 00000100:00100000:2.0:1789760798.184643:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741420326447 time 741420066031 state 0x1 ref 1 00000100:00100000:2.0:1789760798.184675:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741420326447 time 741430293393 state 0x0 ref 1 00000100:00100000:2.0:1789760798.184687:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a386d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554916352:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760798.184734:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68747264, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20c98 00010000:00010000:2.0:1789760798.184743:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7526 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.184759:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760798.184763:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.184768:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7526 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68747264->68751359] (req 68747264->68751359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.184781:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7526 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68747264->68751359] (req 68747264->68751359) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.189908:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984093 is committed 00002000:00010000:2.0:1789760798.190084:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68747264 -> 68751360 00002000:00010000:2.0:1789760798.190090:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134296 -> 134304 00010000:00010000:2.0:1789760798.190102:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7526 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68747264->68751359] (req 68747264->68751359) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.190114:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7526 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68747264->68751359] (req 68747264->68751359) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.190123:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7526 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68747264->68751359] (req 68747264->68751359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.190130:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760798.190135:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7526 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68747264->68751359] (req 68747264->68751359) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.190145:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7526 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68747264->68751359] (req 68747264->68751359) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.190166:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7526 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68747264->68751359] (req 68747264->68751359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.190173:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7526 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68747264->68751359] (req 68747264->68751359) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.190272:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a386d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554916352:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5582us (5801us total) trans 4294984093 rc 0/0 00000100:00100000:2.0:1789760798.190284:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16795 00000100:00100000:2.0:1789760798.190294:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741420326447 time 741430293393 state 0x2 ref 0 00000100:00100000:0.0:1789760798.194738:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760798.194924:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554916480 00000100:00100000:2.0:1789760798.194935:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554916480 00000100:00100000:2.0:1789760798.195098:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741420326447 time 741430293393 state 0x0 ref 1 00000100:00100000:2.0:1789760798.195113:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741430553809 time 741430293393 state 0x1 ref 1 00000100:00100000:2.0:1789760798.195186:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741430553809 time 741440804453 state 0x0 ref 1 00000100:00100000:2.0:1789760798.195200:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a387b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554916480:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760798.195246:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68751360, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20ca0 00010000:00010000:2.0:1789760798.195256:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c752d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.195271:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760798.195276:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.195282:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c752d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68751360->68755455] (req 68751360->68755455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.195296:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c752d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68751360->68755455] (req 68751360->68755455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.199859:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984094 is committed 00002000:00010000:2.0:1789760798.202006:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68751360 -> 68755456 00002000:00010000:2.0:1789760798.202015:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134304 -> 134312 00010000:00010000:2.0:1789760798.202028:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c752d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68751360->68755455] (req 68751360->68755455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.202041:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c752d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68751360->68755455] (req 68751360->68755455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.202049:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c752d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68751360->68755455] (req 68751360->68755455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.202057:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760798.202062:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c752d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68751360->68755455] (req 68751360->68755455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.202070:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c752d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68751360->68755455] (req 68751360->68755455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.202088:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c752d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68751360->68755455] (req 68751360->68755455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.202101:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c752d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68751360->68755455] (req 68751360->68755455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.202256:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a387b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554916480:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7054us (7522us total) trans 4294984094 rc 0/0 00000100:00100000:2.0:1789760798.202266:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16796 00000100:00100000:2.0:1789760798.202275:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741430553809 time 741440804453 state 0x2 ref 0 00000100:00100000:3.0:1789760798.207041:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760798.207132:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554916608 00000100:00100000:0.0:1789760798.207143:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554916608 00000100:00100000:0.0:1789760798.207197:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741430553809 time 741440804453 state 0x0 ref 1 00000100:00100000:0.0:1789760798.207214:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741441064869 time 741440804453 state 0x1 ref 1 00000100:00100000:0.0:1789760798.207285:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741441064869 time 741452901444 state 0x0 ref 1 00000100:00100000:0.0:1789760798.207298:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb046fb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554916608:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760798.207359:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68755456, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20ca8 00010000:00010000:0.0:1789760798.207368:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7534 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.207415:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760798.207420:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.207426:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7534 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68755456->68759551] (req 68755456->68759551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.207440:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7534 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68755456->68759551] (req 68755456->68759551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.216369:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984095 is committed 00002000:00010000:0.0:1789760798.216527:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68755456 -> 68759552 00002000:00010000:0.0:1789760798.216538:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134312 -> 134320 00010000:00010000:0.0:1789760798.216552:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7534 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68755456->68759551] (req 68755456->68759551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.216574:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7534 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68755456->68759551] (req 68755456->68759551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.216584:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7534 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68755456->68759551] (req 68755456->68759551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.216591:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760798.216596:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7534 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68755456->68759551] (req 68755456->68759551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.216605:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7534 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68755456->68759551] (req 68755456->68759551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.216624:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7534 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68755456->68759551] (req 68755456->68759551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.216633:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000944573a4/0x1f99687d854c7534 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68755456->68759551] (req 68755456->68759551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.216734:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb046fb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554916608:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9434us (9699us total) trans 4294984095 rc 0/0 00000100:00100000:0.0:1789760798.216745:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16797 00000100:00100000:0.0:1789760798.216754:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741441064869 time 741452901444 state 0x2 ref 0 00000100:00100000:0.0:1789760798.231500:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760798.232174:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554916736 00000100:00100000:3.0:1789760798.232185:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554916736 00000100:00100000:3.0:1789760798.232241:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741441064869 time 741452901444 state 0x0 ref 1 00000100:00100000:3.0:1789760798.232257:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741453161860 time 741452901444 state 0x1 ref 1 00000100:00100000:3.0:1789760798.232352:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741453161860 time 741477968913 state 0x0 ref 1 00000100:00100000:3.0:1789760798.232367:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a384a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554916736:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760798.232453:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68759552, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20cb0 00010000:00010000:3.0:1789760798.232462:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c753b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.232485:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760798.232490:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.232496:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c753b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68759552->68763647] (req 68759552->68763647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.232511:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c753b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68759552->68763647] (req 68759552->68763647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.240009:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984096 is committed 00002000:00010000:3.0:1789760798.242112:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68759552 -> 68763648 00002000:00010000:3.0:1789760798.242183:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134320 -> 134328 00010000:00010000:3.0:1789760798.242266:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c753b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68759552->68763647] (req 68759552->68763647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.242280:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c753b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68759552->68763647] (req 68759552->68763647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.242289:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c753b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68759552->68763647] (req 68759552->68763647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.242301:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760798.242305:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c753b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68759552->68763647] (req 68759552->68763647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.242313:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c753b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68759552->68763647] (req 68759552->68763647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.242332:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c753b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68759552->68763647] (req 68759552->68763647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.242340:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c753b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68759552->68763647] (req 68759552->68763647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.243358:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a384a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554916736:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10985us (11858us total) trans 4294984096 rc 0/0 00000100:00100000:2.0:1789760798.243372:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16798 00000100:00100000:2.0:1789760798.243404:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741453161860 time 741477968913 state 0x2 ref 0 00000100:00100000:1.0:1789760798.248049:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760798.248291:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554916864 00000100:00100000:2.0:1789760798.248301:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554916864 00000100:00100000:2.0:1789760798.248354:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741453161860 time 741477968913 state 0x0 ref 1 00000100:00100000:2.0:1789760798.248368:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741478229329 time 741477968913 state 0x1 ref 1 00000100:00100000:2.0:1789760798.248425:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741478229329 time 741494042700 state 0x0 ref 1 00000100:00100000:2.0:1789760798.248438:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb098dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554916864:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760798.248485:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68763648, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20cb8 00010000:00010000:2.0:1789760798.248494:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7542 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.248511:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760798.248516:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.248523:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7542 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68763648->68767743] (req 68763648->68767743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.248537:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7542 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68763648->68767743] (req 68763648->68767743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.253452:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984097 is committed 00002000:00010000:2.0:1789760798.253632:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68763648 -> 68767744 00002000:00010000:2.0:1789760798.253637:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134328 -> 134336 00010000:00010000:2.0:1789760798.253651:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7542 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68763648->68767743] (req 68763648->68767743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.253762:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7542 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68763648->68767743] (req 68763648->68767743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.253774:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7542 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68763648->68767743] (req 68763648->68767743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.253781:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760798.253786:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7542 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68763648->68767743] (req 68763648->68767743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.253795:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7542 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68763648->68767743] (req 68763648->68767743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.253821:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7542 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68763648->68767743] (req 68763648->68767743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.253829:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7542 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68763648->68767743] (req 68763648->68767743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.253991:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb098dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554916864:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5551us (5947us total) trans 4294984097 rc 0/0 00000100:00100000:2.0:1789760798.254002:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16799 00000100:00100000:2.0:1789760798.254012:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741478229329 time 741494042700 state 0x2 ref 0 00000100:00100000:2.0:1789760798.259858:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760798.260011:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554916992 00000100:00100000:3.0:1789760798.260078:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554916992 00000100:00100000:3.0:1789760798.260135:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741478229329 time 741494042700 state 0x0 ref 1 00000100:00100000:3.0:1789760798.260151:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741494303116 time 741494042700 state 0x1 ref 1 00000100:00100000:3.0:1789760798.260213:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741494303116 time 741505827040 state 0x0 ref 1 00000100:00100000:3.0:1789760798.260226:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554916992:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760798.260273:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68767744, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20cc0 00010000:00010000:3.0:1789760798.260282:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7549 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.260304:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760798.260310:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.260420:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7549 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68767744->68771839] (req 68767744->68771839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.260438:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7549 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68767744->68771839] (req 68767744->68771839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.266135:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984098 is committed 00002000:00010000:3.0:1789760798.266268:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68767744 -> 68771840 00002000:00010000:3.0:1789760798.266274:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134336 -> 134344 00010000:00010000:3.0:1789760798.266287:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7549 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68767744->68771839] (req 68767744->68771839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.266299:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7549 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68767744->68771839] (req 68767744->68771839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.266309:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7549 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68767744->68771839] (req 68767744->68771839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.266613:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760798.266620:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7549 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68767744->68771839] (req 68767744->68771839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.266631:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7549 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68767744->68771839] (req 68767744->68771839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.266658:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7549 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68767744->68771839] (req 68767744->68771839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.266667:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7549 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68767744->68771839] (req 68767744->68771839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.266897:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554916992:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6668us (7043us total) trans 4294984098 rc 0/0 00000100:00100000:2.0:1789760798.266909:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16800 00000100:00100000:2.0:1789760798.266919:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741494303116 time 741505827040 state 0x2 ref 0 00000100:00100000:0.0:1789760798.274308:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760798.274550:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554917120 00000100:00100000:2.0:1789760798.274573:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554917120 00000100:00100000:2.0:1789760798.274630:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741494303116 time 741505827040 state 0x0 ref 1 00000100:00100000:2.0:1789760798.274646:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741506087456 time 741505827040 state 0x1 ref 1 00000100:00100000:2.0:1789760798.274717:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741506087456 time 741520334836 state 0x0 ref 1 00000100:00100000:2.0:1789760798.274731:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a386a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554917120:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760798.274788:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68771840, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20cc8 00010000:00010000:2.0:1789760798.274798:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7550 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.274813:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760798.274818:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.274824:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7550 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68771840->68775935] (req 68771840->68775935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.274837:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7550 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68771840->68775935] (req 68771840->68775935) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760798.279103:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984099 is committed 00002000:00010000:0.0:1789760798.279215:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68771840 -> 68775936 00002000:00010000:0.0:1789760798.279220:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134344 -> 134352 00010000:00010000:0.0:1789760798.279233:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7550 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68771840->68775935] (req 68771840->68775935) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.279244:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7550 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68771840->68775935] (req 68771840->68775935) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.279252:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7550 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68771840->68775935] (req 68771840->68775935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.279259:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760798.279264:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7550 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68771840->68775935] (req 68771840->68775935) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.279272:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7550 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68771840->68775935] (req 68771840->68775935) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.279290:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7550 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68771840->68775935] (req 68771840->68775935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.279298:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004954649a/0x1f99687d854c7550 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68771840->68775935] (req 68771840->68775935) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.280338:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a386a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554917120:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5601us (6032us total) trans 4294984099 rc 0/0 00000100:00100000:0.0:1789760798.280354:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16801 00000100:00100000:0.0:1789760798.280364:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741506087456 time 741520334836 state 0x2 ref 0 00000100:00100000:1.0:1789760798.288712:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760798.288951:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554917248 00000100:00100000:2.0:1789760798.288960:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554917248 00000100:00100000:2.0:1789760798.289014:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741506087456 time 741520334836 state 0x0 ref 1 00000100:00100000:2.0:1789760798.289028:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741520595252 time 741520334836 state 0x1 ref 1 00000100:00100000:2.0:1789760798.289108:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741520595252 time 741534725242 state 0x0 ref 1 00000100:00100000:2.0:1789760798.289121:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ccb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554917248:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760798.289166:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68775936, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20cd0 00010000:00010000:2.0:1789760798.289175:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7557 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.289197:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760798.289202:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.289209:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7557 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68775936->68780031] (req 68775936->68780031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.289222:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7557 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68775936->68780031] (req 68775936->68780031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.290247:0:6508:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789760798.293226:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984100 is committed 00002000:00010000:2.0:1789760798.293360:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68775936 -> 68780032 00002000:00010000:2.0:1789760798.293365:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134352 -> 134360 00010000:00010000:2.0:1789760798.293409:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7557 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68775936->68780031] (req 68775936->68780031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.293422:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7557 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68775936->68780031] (req 68775936->68780031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.293431:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7557 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68775936->68780031] (req 68775936->68780031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.293439:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760798.293443:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7557 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68775936->68780031] (req 68775936->68780031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.293452:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7557 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68775936->68780031] (req 68775936->68780031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.293678:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7557 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68775936->68780031] (req 68775936->68780031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.293686:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7557 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68775936->68780031] (req 68775936->68780031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.293864:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ccb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554917248:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4739us (5156us total) trans 4294984100 rc 0/0 00000100:00100000:2.0:1789760798.293876:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16802 00000100:00100000:2.0:1789760798.293886:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741520595252 time 741534725242 state 0x2 ref 0 00000100:00100000:1.0:1789760798.299195:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760798.299953:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554917376 00000100:00100000:0.0:1789760798.299970:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554917376 00000100:00100000:0.0:1789760798.300045:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741520595252 time 741534725242 state 0x0 ref 1 00000100:00100000:0.0:1789760798.300063:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741534985658 time 741534725242 state 0x1 ref 1 00000100:00100000:0.0:1789760798.300099:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741534985658 time 741545717266 state 0x0 ref 1 00000100:00100000:0.0:1789760798.300113:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03cc8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554917376:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760798.300179:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68780032, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20cd8 00010000:00010000:0.0:1789760798.300192:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c755e lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.300210:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760798.300216:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.300223:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c755e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68780032->68784127] (req 68780032->68784127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.300238:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c755e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68780032->68784127] (req 68780032->68784127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.307310:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984101 is committed 00002000:00010000:2.0:1789760798.308284:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68780032 -> 68784128 00002000:00010000:2.0:1789760798.308290:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134360 -> 134368 00010000:00010000:2.0:1789760798.308303:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c755e lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68780032->68784127] (req 68780032->68784127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.308451:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c755e lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68780032->68784127] (req 68780032->68784127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.308524:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c755e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68780032->68784127] (req 68780032->68784127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.308532:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0800 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760798.308537:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c755e lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68780032->68784127] (req 68780032->68784127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.308551:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c755e lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68780032->68784127] (req 68780032->68784127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.308582:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c755e lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68780032->68784127] (req 68780032->68784127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.308590:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000db61e578/0x1f99687d854c755e lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68780032->68784127] (req 68780032->68784127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.308730:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03cc8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554917376:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8616us (9540us total) trans 4294984101 rc 0/0 00000100:00100000:2.0:1789760798.308741:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16803 00000100:00100000:2.0:1789760798.308760:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741534985658 time 741545717266 state 0x2 ref 0 00000100:00100000:0.0:1789760798.313558:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760798.313656:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554917504 00000100:00100000:2.0:1789760798.313665:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554917504 00000100:00100000:2.0:1789760798.313716:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741534985658 time 741545717266 state 0x0 ref 1 00000100:00100000:2.0:1789760798.313732:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741545977682 time 741545717266 state 0x1 ref 1 00000100:00100000:2.0:1789760798.313855:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741545977682 time 741559472099 state 0x0 ref 1 00000100:00100000:2.0:1789760798.313868:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a386300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554917504:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760798.313926:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68784128, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20ce0 00010000:00010000:2.0:1789760798.313944:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7565 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.313961:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760798.313966:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.313972:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7565 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68784128->68788223] (req 68784128->68788223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.313985:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7565 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68784128->68788223] (req 68784128->68788223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.320912:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984102 is committed 00002000:00010000:2.0:1789760798.324879:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68784128 -> 68788224 00002000:00010000:2.0:1789760798.324889:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134368 -> 134376 00010000:00010000:2.0:1789760798.324905:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7565 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68784128->68788223] (req 68784128->68788223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.324919:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7565 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68784128->68788223] (req 68784128->68788223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.324928:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7565 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68784128->68788223] (req 68784128->68788223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.325000:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760798.325006:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7565 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68784128->68788223] (req 68784128->68788223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.325017:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7565 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68784128->68788223] (req 68784128->68788223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.325096:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7565 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68784128->68788223] (req 68784128->68788223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.325104:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c7565 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68784128->68788223] (req 68784128->68788223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.325202:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a386300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554917504:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11332us (11649us total) trans 4294984102 rc 0/0 00000100:00100000:2.0:1789760798.325213:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16804 00000100:00100000:2.0:1789760798.325221:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741545977682 time 741559472099 state 0x2 ref 0 00000100:00100000:1.0:1789760798.331624:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760798.331725:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554917632 00000100:00100000:0.0:1789760798.331736:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554917632 00000100:00100000:0.0:1789760798.331791:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741545977682 time 741559472099 state 0x0 ref 1 00000100:00100000:0.0:1789760798.331808:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741559732515 time 741559472099 state 0x1 ref 1 00000100:00100000:0.0:1789760798.331844:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741559732515 time 741577462189 state 0x0 ref 1 00000100:00100000:0.0:1789760798.331857:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ccb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554917632:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760798.331909:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68788224, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20ce8 00010000:00010000:0.0:1789760798.331918:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c756c lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.331934:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760798.331938:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.331944:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c756c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68788224->68792319] (req 68788224->68792319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.331957:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c756c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68788224->68792319] (req 68788224->68792319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.341907:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984103 is committed 00002000:00010000:1.0:1789760798.342370:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68788224 -> 68792320 00002000:00010000:1.0:1789760798.342375:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134376 -> 134384 00010000:00010000:1.0:1789760798.342412:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c756c lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68788224->68792319] (req 68788224->68792319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.342426:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c756c lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68788224->68792319] (req 68788224->68792319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.342436:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c756c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68788224->68792319] (req 68788224->68792319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.342444:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1e00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760798.342450:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c756c lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68788224->68792319] (req 68788224->68792319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.342459:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c756c lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68788224->68792319] (req 68788224->68792319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.342480:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c756c lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68788224->68792319] (req 68788224->68792319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.342488:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c756c lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68788224->68792319] (req 68788224->68792319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.342597:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ccb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554917632:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10737us (10978us total) trans 4294984103 rc 0/0 00000100:00100000:1.0:1789760798.342612:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16805 00000100:00100000:1.0:1789760798.342621:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741559732515 time 741577462189 state 0x2 ref 0 00000100:00100000:1.0:1789760798.347582:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760798.347664:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554917760 00000100:00100000:0.0:1789760798.347673:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554917760 00000100:00100000:0.0:1789760798.347728:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741559732515 time 741577462189 state 0x0 ref 1 00000100:00100000:0.0:1789760798.347742:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741577722605 time 741577462189 state 0x1 ref 1 00000100:00100000:0.0:1789760798.347778:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741577722605 time 741593396831 state 0x0 ref 1 00000100:00100000:0.0:1789760798.347790:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03cc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554917760:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760798.347843:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68792320, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20cf0 00010000:00010000:0.0:1789760798.347853:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7573 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.347868:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760798.347872:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.347879:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7573 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68792320->68796415] (req 68792320->68796415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.347893:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7573 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68792320->68796415] (req 68792320->68796415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760798.354236:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984104 is committed 00002000:00010000:0.0:1789760798.354696:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68792320 -> 68796416 00002000:00010000:0.0:1789760798.354702:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134384 -> 134392 00010000:00010000:0.0:1789760798.354725:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7573 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68792320->68796415] (req 68792320->68796415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.354739:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7573 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68792320->68796415] (req 68792320->68796415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.354749:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7573 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68792320->68796415] (req 68792320->68796415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.354762:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760798.354767:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7573 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68792320->68796415] (req 68792320->68796415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.354777:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7573 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68792320->68796415] (req 68792320->68796415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.354795:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7573 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68792320->68796415] (req 68792320->68796415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.354802:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c7573 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68792320->68796415] (req 68792320->68796415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.354930:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03cc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554917760:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7137us (7353us total) trans 4294984104 rc 0/0 00000100:00100000:0.0:1789760798.354950:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16806 00000100:00100000:0.0:1789760798.354959:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741577722605 time 741593396831 state 0x2 ref 0 00000100:00100000:0.0:1789760798.365768:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760798.366171:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554917888 00000100:00100000:2.0:1789760798.366180:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554917888 00000100:00100000:2.0:1789760798.366250:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741577722605 time 741593396831 state 0x0 ref 1 00000100:00100000:2.0:1789760798.366265:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741593657247 time 741593396831 state 0x1 ref 1 00000100:00100000:2.0:1789760798.366297:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741593657247 time 741611915659 state 0x0 ref 1 00000100:00100000:2.0:1789760798.366308:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554917888:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760798.366363:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68796416, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20cf8 00010000:00010000:2.0:1789760798.366372:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c757a lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.366503:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760798.366507:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.366514:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c757a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68796416->68800511] (req 68796416->68800511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.366529:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c757a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68796416->68800511] (req 68796416->68800511) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.377130:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984105 is committed 00002000:00010000:3.0:1789760798.379165:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68796416 -> 68800512 00002000:00010000:3.0:1789760798.379174:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134392 -> 134400 00010000:00010000:3.0:1789760798.379188:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c757a lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68796416->68800511] (req 68796416->68800511) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.379202:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c757a lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68796416->68800511] (req 68796416->68800511) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.379211:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c757a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68796416->68800511] (req 68796416->68800511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.379219:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760798.379223:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c757a lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68796416->68800511] (req 68796416->68800511) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.379232:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c757a lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68796416->68800511] (req 68796416->68800511) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.379251:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c757a lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68796416->68800511] (req 68796416->68800511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.379259:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cca4d533/0x1f99687d854c757a lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68796416->68800511] (req 68796416->68800511) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.380782:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554917888:12345-192.168.203.22@tcp:4:dd.0 Request processed in 14462us (15023us total) trans 4294984105 rc 0/0 00000100:00100000:3.0:1789760798.380796:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16807 00000100:00100000:3.0:1789760798.380807:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741593657247 time 741611915659 state 0x2 ref 0 00000100:00100000:1.0:1789760798.390574:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760798.390673:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554918016 00000100:00100000:2.0:1789760798.390684:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554918016 00000100:00100000:2.0:1789760798.390739:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741593657247 time 741611915659 state 0x0 ref 1 00000100:00100000:2.0:1789760798.390754:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741612176075 time 741611915659 state 0x1 ref 1 00000100:00100000:2.0:1789760798.390788:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741612176075 time 741636405873 state 0x0 ref 1 00000100:00100000:2.0:1789760798.390801:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03cc8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554918016:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760798.390847:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68800512, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20d00 00010000:00010000:2.0:1789760798.390860:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7581 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.390876:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760798.390880:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.390886:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7581 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68800512->68804607] (req 68800512->68804607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.390906:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7581 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68800512->68804607] (req 68800512->68804607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.397152:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984106 is committed 00002000:00010000:3.0:1789760798.397696:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68800512 -> 68804608 00002000:00010000:3.0:1789760798.397701:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134400 -> 134408 00010000:00010000:3.0:1789760798.397723:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7581 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68800512->68804607] (req 68800512->68804607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.397736:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7581 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68800512->68804607] (req 68800512->68804607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.397745:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7581 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68800512->68804607] (req 68800512->68804607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.397753:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760798.397758:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7581 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68800512->68804607] (req 68800512->68804607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.397768:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7581 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68800512->68804607] (req 68800512->68804607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.397792:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7581 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68800512->68804607] (req 68800512->68804607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.397800:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7581 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68800512->68804607] (req 68800512->68804607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.397887:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03cc8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554918016:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7085us (7330us total) trans 4294984106 rc 0/0 00000100:00100000:3.0:1789760798.397897:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16808 00000100:00100000:3.0:1789760798.397905:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741612176075 time 741636405873 state 0x2 ref 0 00000100:00100000:1.0:1789760798.405580:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760798.405808:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554918144 00000100:00100000:0.0:1789760798.405819:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554918144 00000100:00100000:0.0:1789760798.405872:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741612176075 time 741636405873 state 0x0 ref 1 00000100:00100000:0.0:1789760798.405888:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741636666289 time 741636405873 state 0x1 ref 1 00000100:00100000:0.0:1789760798.406015:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741636666289 time 741651632502 state 0x0 ref 1 00000100:00100000:0.0:1789760798.406029:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ccaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554918144:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760798.406076:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68804608, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20d08 00010000:00010000:0.0:1789760798.406085:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c7588 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.406101:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760798.406106:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.406112:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c7588 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68804608->68808703] (req 68804608->68808703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.406126:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c7588 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68804608->68808703] (req 68804608->68808703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.429553:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984107 is committed 00002000:00010000:0.0:1789760798.430007:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68804608 -> 68808704 00002000:00010000:0.0:1789760798.430012:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134408 -> 134416 00010000:00010000:0.0:1789760798.430026:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c7588 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68804608->68808703] (req 68804608->68808703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.430038:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c7588 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68804608->68808703] (req 68804608->68808703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.430048:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c7588 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68804608->68808703] (req 68804608->68808703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.430060:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760798.430066:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c7588 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68804608->68808703] (req 68804608->68808703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.430075:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c7588 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68804608->68808703] (req 68804608->68808703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.430096:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c7588 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68804608->68808703] (req 68804608->68808703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.430104:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c7588 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68804608->68808703] (req 68804608->68808703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.430196:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ccaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554918144:12345-192.168.203.22@tcp:4:dd.0 Request processed in 24165us (24633us total) trans 4294984107 rc 0/0 00000100:00100000:0.0:1789760798.430206:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16809 00000100:00100000:0.0:1789760798.430216:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741636666289 time 741651632502 state 0x2 ref 0 00000100:00100000:0.0:1789760798.439107:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760798.439634:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554918272 00000100:00100000:2.0:1789760798.439644:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554918272 00000100:00100000:2.0:1789760798.439697:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741636666289 time 741651632502 state 0x0 ref 1 00000100:00100000:2.0:1789760798.439712:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741651892918 time 741651632502 state 0x1 ref 1 00000100:00100000:2.0:1789760798.439750:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741651892918 time 741685368466 state 0x0 ref 1 00000100:00100000:2.0:1789760798.439764:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a386680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554918272:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760798.439807:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68808704, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20d10 00010000:00010000:2.0:1789760798.439816:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c758f lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.439831:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760798.439835:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.439841:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c758f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68808704->68812799] (req 68808704->68812799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.439855:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c758f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68808704->68812799] (req 68808704->68812799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.445953:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984108 is committed 00002000:00010000:2.0:1789760798.449068:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68808704 -> 68812800 00002000:00010000:2.0:1789760798.449078:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134416 -> 134424 00010000:00010000:2.0:1789760798.449091:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c758f lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68808704->68812799] (req 68808704->68812799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.449104:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c758f lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68808704->68812799] (req 68808704->68812799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.449113:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c758f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68808704->68812799] (req 68808704->68812799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.449120:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760798.449125:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c758f lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68808704->68812799] (req 68808704->68812799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.449134:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c758f lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68808704->68812799] (req 68808704->68812799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.449153:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c758f lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68808704->68812799] (req 68808704->68812799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.449162:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c758f lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68808704->68812799] (req 68808704->68812799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.449299:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a386680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554918272:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9534us (10197us total) trans 4294984108 rc 0/0 00000100:00100000:2.0:1789760798.449311:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16810 00000100:00100000:2.0:1789760798.449320:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741651892918 time 741685368466 state 0x2 ref 0 00000100:00100000:1.0:1789760798.453433:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760798.453696:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554918400 00000100:00100000:3.0:1789760798.453707:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554918400 00000100:00100000:3.0:1789760798.453774:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741651892918 time 741685368466 state 0x0 ref 1 00000100:00100000:3.0:1789760798.453789:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741685628882 time 741685368466 state 0x1 ref 1 00000100:00100000:3.0:1789760798.453811:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741685628882 time 741699429563 state 0x0 ref 1 00000100:00100000:3.0:1789760798.453822:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03cc8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554918400:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760798.453868:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68812800, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20d18 00010000:00010000:3.0:1789760798.453877:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7596 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.453892:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760798.453897:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.453903:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7596 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68812800->68816895] (req 68812800->68816895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.453916:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7596 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68812800->68816895] (req 68812800->68816895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.458773:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984109 is committed 00002000:00010000:3.0:1789760798.460705:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68812800 -> 68816896 00002000:00010000:3.0:1789760798.460714:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134424 -> 134432 00010000:00010000:3.0:1789760798.460727:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7596 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68812800->68816895] (req 68812800->68816895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.460741:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7596 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68812800->68816895] (req 68812800->68816895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.460750:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7596 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68812800->68816895] (req 68812800->68816895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.460756:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760798.460761:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7596 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68812800->68816895] (req 68812800->68816895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.460770:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7596 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68812800->68816895] (req 68812800->68816895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.460789:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7596 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68812800->68816895] (req 68812800->68816895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.460802:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7596 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68812800->68816895] (req 68812800->68816895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.461007:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03cc8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554918400:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7181us (7578us total) trans 4294984109 rc 0/0 00000100:00100000:3.0:1789760798.461272:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16811 00000100:00100000:3.0:1789760798.461282:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741685628882 time 741699429563 state 0x2 ref 0 00000100:00100000:2.0:1789760798.465470:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760798.465599:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554918528 00000100:00100000:3.0:1789760798.465609:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554918528 00000100:00100000:3.0:1789760798.465663:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741685628882 time 741699429563 state 0x0 ref 1 00000100:00100000:3.0:1789760798.465677:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741699689979 time 741699429563 state 0x1 ref 1 00000100:00100000:3.0:1789760798.465707:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741699689979 time 741711324907 state 0x0 ref 1 00000100:00100000:3.0:1789760798.465719:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554918528:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760798.465768:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68816896, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20d20 00010000:00010000:3.0:1789760798.465776:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c759d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.465793:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760798.465798:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.465804:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c759d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68816896->68820991] (req 68816896->68820991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.465822:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c759d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68816896->68820991] (req 68816896->68820991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.469943:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984110 is committed 00002000:00010000:3.0:1789760798.470313:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68816896 -> 68820992 00002000:00010000:3.0:1789760798.470319:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134432 -> 134440 00010000:00010000:3.0:1789760798.470333:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c759d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68816896->68820991] (req 68816896->68820991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.470346:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c759d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68816896->68820991] (req 68816896->68820991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.470355:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c759d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68816896->68820991] (req 68816896->68820991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.470363:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760798.470368:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c759d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68816896->68820991] (req 68816896->68820991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.470458:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c759d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68816896->68820991] (req 68816896->68820991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.470480:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c759d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68816896->68820991] (req 68816896->68820991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.470488:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c759d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68816896->68820991] (req 68816896->68820991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.470615:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554918528:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4894us (5151us total) trans 4294984110 rc 0/0 00000100:00100000:3.0:1789760798.470627:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16812 00000100:00100000:3.0:1789760798.470636:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741699689979 time 741711324907 state 0x2 ref 0 00000100:00100000:1.0:1789760798.486501:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760798.487030:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554918656 00000100:00100000:0.0:1789760798.487041:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554918656 00000100:00100000:0.0:1789760798.487099:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741699689979 time 741711324907 state 0x0 ref 1 00000100:00100000:0.0:1789760798.487116:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741711585323 time 741711324907 state 0x1 ref 1 00000100:00100000:0.0:1789760798.487150:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741711585323 time 741732768954 state 0x0 ref 1 00000100:00100000:0.0:1789760798.487164:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ccad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554918656:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760798.487208:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68820992, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20d28 00010000:00010000:0.0:1789760798.487217:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75a4 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.487232:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760798.487237:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.487244:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68820992->68825087] (req 68820992->68825087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.487257:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68820992->68825087] (req 68820992->68825087) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.495228:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984111 is committed 00002000:00010000:0.0:1789760798.497024:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68820992 -> 68825088 00002000:00010000:0.0:1789760798.497034:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134440 -> 134448 00010000:00010000:0.0:1789760798.497049:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75a4 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68820992->68825087] (req 68820992->68825087) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.497062:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75a4 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68820992->68825087] (req 68820992->68825087) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.497072:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68820992->68825087] (req 68820992->68825087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.497079:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760798.497085:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75a4 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68820992->68825087] (req 68820992->68825087) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.497094:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75a4 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68820992->68825087] (req 68820992->68825087) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.497112:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75a4 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68820992->68825087] (req 68820992->68825087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.497121:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75a4 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68820992->68825087] (req 68820992->68825087) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.497225:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ccad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554918656:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10059us (10729us total) trans 4294984111 rc 0/0 00000100:00100000:0.0:1789760798.497236:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16813 00000100:00100000:0.0:1789760798.497246:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741711585323 time 741732768954 state 0x2 ref 0 00000100:00100000:1.0:1789760798.504706:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760798.505031:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554918784 00000100:00100000:2.0:1789760798.505042:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554918784 00000100:00100000:2.0:1789760798.505096:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741711585323 time 741732768954 state 0x0 ref 1 00000100:00100000:2.0:1789760798.505112:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741733029370 time 741732768954 state 0x1 ref 1 00000100:00100000:2.0:1789760798.505142:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741733029370 time 741750760864 state 0x0 ref 1 00000100:00100000:2.0:1789760798.505155:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03cca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554918784:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760798.505198:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68825088, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20d30 00010000:00010000:2.0:1789760798.505207:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c75ab lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.505224:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760798.505228:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.505235:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c75ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68825088->68829183] (req 68825088->68829183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.505249:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c75ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68825088->68829183] (req 68825088->68829183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.509363:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984112 is committed 00002000:00010000:2.0:1789760798.509652:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68825088 -> 68829184 00002000:00010000:2.0:1789760798.509658:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134448 -> 134456 00010000:00010000:2.0:1789760798.509672:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c75ab lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68825088->68829183] (req 68825088->68829183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.509691:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c75ab lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68825088->68829183] (req 68825088->68829183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.509701:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c75ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68825088->68829183] (req 68825088->68829183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.509709:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f8600 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760798.509713:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c75ab lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68825088->68829183] (req 68825088->68829183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.509723:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c75ab lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68825088->68829183] (req 68825088->68829183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.509742:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c75ab lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68825088->68829183] (req 68825088->68829183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.509751:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c824aa69/0x1f99687d854c75ab lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68825088->68829183] (req 68825088->68829183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.509847:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03cca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554918784:12345-192.168.203.22@tcp:4:dd.0 Request processed in 4689us (5146us total) trans 4294984112 rc 0/0 00000100:00100000:2.0:1789760798.509858:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16814 00000100:00100000:2.0:1789760798.509867:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741733029370 time 741750760864 state 0x2 ref 0 00000100:00100000:2.0:1789760798.517623:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760798.517722:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554918912 00000100:00100000:3.0:1789760798.517733:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554918912 00000100:00100000:3.0:1789760798.517788:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741733029370 time 741750760864 state 0x0 ref 1 00000100:00100000:3.0:1789760798.517802:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741751021280 time 741750760864 state 0x1 ref 1 00000100:00100000:3.0:1789760798.517863:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741751021280 time 741763481119 state 0x0 ref 1 00000100:00100000:3.0:1789760798.517876:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554918912:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760798.517919:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68829184, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20d38 00010000:00010000:3.0:1789760798.517928:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75b2 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.517945:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760798.517950:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.517956:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68829184->68833279] (req 68829184->68833279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.517969:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68829184->68833279] (req 68829184->68833279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.524020:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984113 is committed 00002000:00010000:3.0:1789760798.524375:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68829184 -> 68833280 00002000:00010000:3.0:1789760798.524411:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134456 -> 134464 00010000:00010000:3.0:1789760798.524424:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75b2 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68829184->68833279] (req 68829184->68833279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.524437:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75b2 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68829184->68833279] (req 68829184->68833279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.524447:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68829184->68833279] (req 68829184->68833279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.524455:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760798.524459:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75b2 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68829184->68833279] (req 68829184->68833279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.524468:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75b2 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68829184->68833279] (req 68829184->68833279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.524487:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75b2 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68829184->68833279] (req 68829184->68833279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.524495:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75b2 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68829184->68833279] (req 68829184->68833279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.524592:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554918912:12345-192.168.203.22@tcp:4:dd.0 Request processed in 6713us (6974us total) trans 4294984113 rc 0/0 00000100:00100000:3.0:1789760798.524603:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16815 00000100:00100000:3.0:1789760798.524613:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741751021280 time 741763481119 state 0x2 ref 0 00000100:00100000:1.0:1789760798.530689:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760798.530787:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554919040 00000100:00100000:3.0:1789760798.530798:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554919040 00000100:00100000:3.0:1789760798.530854:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741751021280 time 741763481119 state 0x0 ref 1 00000100:00100000:3.0:1789760798.530867:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741763741535 time 741763481119 state 0x1 ref 1 00000100:00100000:3.0:1789760798.530900:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741763741535 time 741776518414 state 0x0 ref 1 00000100:00100000:3.0:1789760798.530911:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03cc9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554919040:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760798.531067:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68833280, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20d40 00010000:00010000:3.0:1789760798.531077:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75b9 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.531092:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760798.531096:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.531102:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68833280->68837375] (req 68833280->68837375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.531115:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68833280->68837375] (req 68833280->68837375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760798.537980:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984114 is committed 00002000:00010000:3.0:1789760798.540450:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68833280 -> 68837376 00002000:00010000:3.0:1789760798.540465:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134464 -> 134472 00010000:00010000:3.0:1789760798.540481:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75b9 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68833280->68837375] (req 68833280->68837375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.540494:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75b9 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68833280->68837375] (req 68833280->68837375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.540503:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68833280->68837375] (req 68833280->68837375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.540511:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760798.540516:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75b9 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68833280->68837375] (req 68833280->68837375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.540525:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75b9 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68833280->68837375] (req 68833280->68837375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.540545:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75b9 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68833280->68837375] (req 68833280->68837375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.540553:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75b9 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68833280->68837375] (req 68833280->68837375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.540663:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03cc9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554919040:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9749us (9982us total) trans 4294984114 rc 0/0 00000100:00100000:3.0:1789760798.540673:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16816 00000100:00100000:3.0:1789760798.540691:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741763741535 time 741776518414 state 0x2 ref 0 00000100:00100000:1.0:1789760798.550927:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760798.551045:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554919168 00000100:00100000:0.0:1789760798.551056:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554919168 00000100:00100000:0.0:1789760798.551188:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741763741535 time 741776518414 state 0x0 ref 1 00000100:00100000:0.0:1789760798.551204:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741776778830 time 741776518414 state 0x1 ref 1 00000100:00100000:0.0:1789760798.551274:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741776778830 time 741796891466 state 0x0 ref 1 00000100:00100000:0.0:1789760798.551288:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ccb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554919168:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760798.551336:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68837376, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20d48 00010000:00010000:0.0:1789760798.551346:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75c0 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.551360:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760798.551365:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.551372:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68837376->68841471] (req 68837376->68841471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.551430:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68837376->68841471] (req 68837376->68841471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760798.557667:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984115 is committed 00002000:00010000:0.0:1789760798.561228:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68837376 -> 68841472 00002000:00010000:0.0:1789760798.561239:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134472 -> 134480 00010000:00010000:0.0:1789760798.561252:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75c0 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68837376->68841471] (req 68837376->68841471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.561265:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75c0 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68837376->68841471] (req 68837376->68841471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.561274:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68837376->68841471] (req 68837376->68841471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.561282:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760798.561287:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75c0 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68837376->68841471] (req 68837376->68841471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.561296:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75c0 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68837376->68841471] (req 68837376->68841471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.561316:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75c0 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68837376->68841471] (req 68837376->68841471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.561324:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75c0 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68837376->68841471] (req 68837376->68841471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.562078:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ccb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554919168:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10783us (11153us total) trans 4294984115 rc 0/0 00000100:00100000:0.0:1789760798.562091:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16817 00000100:00100000:0.0:1789760798.562100:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741776778830 time 741796891466 state 0x2 ref 0 00000100:00100000:2.0:1789760798.571526:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760798.571792:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554919296 00000100:00100000:0.0:1789760798.571802:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554919296 00000100:00100000:0.0:1789760798.571857:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741776778830 time 741796891466 state 0x0 ref 1 00000100:00100000:0.0:1789760798.571871:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741797151882 time 741796891466 state 0x1 ref 1 00000100:00100000:0.0:1789760798.571898:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741797151882 time 741817516489 state 0x0 ref 1 00000100:00100000:0.0:1789760798.571911:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554919296:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760798.573218:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68841472, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20d50 00010000:00010000:0.0:1789760798.573232:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75c7 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.573250:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760798.573254:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.573260:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68841472->68845567] (req 68841472->68845567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.573274:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68841472->68845567] (req 68841472->68845567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760798.580286:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984116 is committed 00002000:00010000:1.0:1789760798.581413:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68841472 -> 68845568 00002000:00010000:1.0:1789760798.581420:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134480 -> 134488 00010000:00010000:1.0:1789760798.581434:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75c7 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68841472->68845567] (req 68841472->68845567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.581448:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75c7 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68841472->68845567] (req 68841472->68845567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.581461:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68841472->68845567] (req 68841472->68845567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.581469:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760798.581473:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75c7 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68841472->68845567] (req 68841472->68845567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.581483:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75c7 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68841472->68845567] (req 68841472->68845567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.581500:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75c7 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68841472->68845567] (req 68841472->68845567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.581507:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75c7 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68841472->68845567] (req 68841472->68845567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.581600:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554919296:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9686us (10078us total) trans 4294984116 rc 0/0 00000100:00100000:1.0:1789760798.581610:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16818 00000100:00100000:1.0:1789760798.581619:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741797151882 time 741817516489 state 0x2 ref 0 00000100:00100000:1.0:1789760798.585968:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760798.590783:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554919424 00000100:00100000:3.0:1789760798.590796:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554919424 00000100:00100000:3.0:1789760798.590855:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741797151882 time 741817516489 state 0x0 ref 1 00000100:00100000:3.0:1789760798.590870:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741817776905 time 741817516489 state 0x1 ref 1 00000100:00100000:3.0:1789760798.590935:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741817776905 time 741836553200 state 0x0 ref 1 00000100:00100000:3.0:1789760798.590948:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ccbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554919424:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760798.590990:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68845568, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20d58 00010000:00010000:3.0:1789760798.590999:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75ce lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.591023:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760798.591028:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.591034:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68845568->68849663] (req 68845568->68849663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.591048:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68845568->68849663] (req 68845568->68849663) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789760798.596079:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984117 is committed 00002000:00010000:3.0:1789760798.596196:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68845568 -> 68849664 00002000:00010000:3.0:1789760798.596201:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134488 -> 134496 00010000:00010000:3.0:1789760798.596214:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75ce lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68845568->68849663] (req 68845568->68849663) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.596231:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75ce lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68845568->68849663] (req 68845568->68849663) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.596240:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68845568->68849663] (req 68845568->68849663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.596246:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760798.596251:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75ce lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68845568->68849663] (req 68845568->68849663) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.596260:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75ce lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68845568->68849663] (req 68845568->68849663) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.596281:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75ce lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68845568->68849663] (req 68845568->68849663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.596289:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75ce lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68845568->68849663] (req 68845568->68849663) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.596423:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ccbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554919424:12345-192.168.203.22@tcp:4:dd.0 Request processed in 5431us (10419us total) trans 4294984117 rc 0/0 00000100:00100000:3.0:1789760798.596434:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16819 00000100:00100000:3.0:1789760798.596443:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741817776905 time 741836553200 state 0x2 ref 0 00000100:00100000:0.0:1789760798.608776:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760798.608871:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554919552 00000100:00100000:3.0:1789760798.608884:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554919552 00000100:00100000:3.0:1789760798.608961:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741817776905 time 741836553200 state 0x0 ref 1 00000100:00100000:3.0:1789760798.608980:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741836813616 time 741836553200 state 0x1 ref 1 00000100:00100000:3.0:1789760798.609061:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741836813616 time 741854678777 state 0x0 ref 1 00000100:00100000:3.0:1789760798.609074:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554919552:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760798.609126:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68849664, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20d60 00010000:00010000:3.0:1789760798.609136:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75d5 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.609152:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760798.609156:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.609162:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68849664->68853759] (req 68849664->68853759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.609176:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68849664->68853759] (req 68849664->68853759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760798.625373:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984118 is committed 00002000:00010000:0.0:1789760798.625836:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68849664 -> 68853760 00002000:00010000:0.0:1789760798.625842:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134496 -> 134504 00010000:00010000:0.0:1789760798.625858:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75d5 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68849664->68853759] (req 68849664->68853759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.625872:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75d5 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68849664->68853759] (req 68849664->68853759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.625882:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68849664->68853759] (req 68849664->68853759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.625889:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760798.625894:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75d5 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68849664->68853759] (req 68849664->68853759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.625903:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75d5 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68849664->68853759] (req 68849664->68853759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.625923:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75d5 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68849664->68853759] (req 68849664->68853759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.625931:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e05a4ee2/0x1f99687d854c75d5 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68849664->68853759] (req 68849664->68853759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.626051:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554919552:12345-192.168.203.22@tcp:4:dd.0 Request processed in 16974us (17282us total) trans 4294984118 rc 0/0 00000100:00100000:0.0:1789760798.626062:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16820 00000100:00100000:0.0:1789760798.626072:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741836813616 time 741854678777 state 0x2 ref 0 00000100:00100000:2.0:1789760798.638100:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760798.638186:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554919680 00000100:00100000:0.0:1789760798.638195:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554919680 00000100:00100000:0.0:1789760798.638243:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741836813616 time 741854678777 state 0x0 ref 1 00000100:00100000:0.0:1789760798.638257:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741854939193 time 741854678777 state 0x1 ref 1 00000100:00100000:0.0:1789760798.638292:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741854939193 time 741883910589 state 0x0 ref 1 00000100:00100000:0.0:1789760798.638303:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554919680:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760798.638349:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68853760, mtime: 0x6aad951d, atime: 0x0, ctime: 0x6aad951d, blocks: 0x20d68 00010000:00010000:0.0:1789760798.638358:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75dc lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.638374:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760798.638413:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.638420:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68853760->68857855] (req 68853760->68857855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.638438:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68853760->68857855] (req 68853760->68857855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760798.648946:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984119 is committed 00002000:00010000:0.0:1789760798.649077:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68853760 -> 68857856 00002000:00010000:0.0:1789760798.649082:0:8442:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb mtime from disk: 1789760797 -> 1789760798 00002000:00010000:0.0:1789760798.649084:0:8442:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb ctime from disk: 1789760797 -> 1789760798 00002000:00010000:0.0:1789760798.649086:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134504 -> 134512 00010000:00010000:0.0:1789760798.649099:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75dc lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68853760->68857855] (req 68853760->68857855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.649113:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75dc lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68853760->68857855] (req 68853760->68857855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.649122:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68853760->68857855] (req 68853760->68857855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.649130:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760798.649135:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75dc lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68853760->68857855] (req 68853760->68857855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.649144:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75dc lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68853760->68857855] (req 68853760->68857855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.649166:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75dc lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68853760->68857855] (req 68853760->68857855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.649179:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ae3b1666/0x1f99687d854c75dc lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68853760->68857855] (req 68853760->68857855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.649291:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2411:x1876699554919680:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10984us (11195us total) trans 4294984119 rc 0/0 00000100:00100000:0.0:1789760798.649302:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16821 00000100:00100000:0.0:1789760798.649311:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741854939193 time 741883910589 state 0x2 ref 0 00000100:00100000:1.0:1789760798.652101:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760798.652356:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554919808 00000100:00100000:2.0:1789760798.652366:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554919808 00000100:00100000:2.0:1789760798.652460:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741854939193 time 741883910589 state 0x0 ref 1 00000100:00100000:2.0:1789760798.652476:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741884171005 time 741883910589 state 0x1 ref 1 00000100:00100000:2.0:1789760798.652498:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741884171005 time 741898116560 state 0x0 ref 1 00000100:00100000:2.0:1789760798.652510:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03cc9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554919808:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760798.652555:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68857856, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x20d70 00010000:00010000:2.0:1789760798.652575:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c75e3 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.652589:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760798.652594:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.652600:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c75e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68857856->68861951] (req 68857856->68861951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.652613:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c75e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68857856->68861951] (req 68857856->68861951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760798.661452:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984120 is committed 00002000:00010000:2.0:1789760798.661689:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68857856 -> 68861952 00002000:00010000:2.0:1789760798.661694:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134512 -> 134520 00010000:00010000:2.0:1789760798.661708:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c75e3 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68857856->68861951] (req 68857856->68861951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.661721:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c75e3 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68857856->68861951] (req 68857856->68861951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.661729:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c75e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68857856->68861951] (req 68857856->68861951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.661736:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760798.661742:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c75e3 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68857856->68861951] (req 68857856->68861951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.661751:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c75e3 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68857856->68861951] (req 68857856->68861951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.661771:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c75e3 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68857856->68861951] (req 68857856->68861951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.661848:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005cc20887/0x1f99687d854c75e3 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68857856->68861951] (req 68857856->68861951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.662145:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03cc9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554919808:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9631us (10048us total) trans 4294984120 rc 0/0 00000100:00100000:2.0:1789760798.662156:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16822 00000100:00100000:2.0:1789760798.662165:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741884171005 time 741898116560 state 0x2 ref 0 00000100:00100000:1.0:1789760798.671460:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760798.671548:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554919936 00000100:00100000:3.0:1789760798.671569:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554919936 00000100:00100000:3.0:1789760798.671628:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741884171005 time 741898116560 state 0x0 ref 1 00000100:00100000:3.0:1789760798.671645:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741898376976 time 741898116560 state 0x1 ref 1 00000100:00100000:3.0:1789760798.671667:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741898376976 time 741917285911 state 0x0 ref 1 00000100:00100000:3.0:1789760798.671678:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03cca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554919936:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760798.671724:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68861952, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x20d78 00010000:00010000:3.0:1789760798.671732:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c75ea lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.671750:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760798.671755:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.671761:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c75ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68861952->68866047] (req 68861952->68866047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.671775:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c75ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68861952->68866047] (req 68861952->68866047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.683311:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984121 is committed 00002000:00010000:3.0:1789760798.683501:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68861952 -> 68866048 00002000:00010000:3.0:1789760798.683506:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134520 -> 134528 00010000:00010000:3.0:1789760798.683519:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c75ea lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68861952->68866047] (req 68861952->68866047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.683532:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c75ea lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68861952->68866047] (req 68861952->68866047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.683541:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c75ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68861952->68866047] (req 68861952->68866047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.683553:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:3.0:1789760798.683567:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c75ea lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68861952->68866047] (req 68861952->68866047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.683576:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c75ea lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68861952->68866047] (req 68861952->68866047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.683596:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c75ea lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68861952->68866047] (req 68861952->68866047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.683603:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c75ea lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68861952->68866047] (req 68861952->68866047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789760798.683693:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03cca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554919936:12345-192.168.203.22@tcp:4:dd.0 Request processed in 12011us (12241us total) trans 4294984121 rc 0/0 00000100:00100000:3.0:1789760798.683704:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16823 00000100:00100000:3.0:1789760798.683713:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741898376976 time 741917285911 state 0x2 ref 0 00000100:00100000:0.0:1789760798.691531:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760798.691656:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554920064 00000100:00100000:3.0:1789760798.691665:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554920064 00000100:00100000:3.0:1789760798.691718:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741898376976 time 741917285911 state 0x0 ref 1 00000100:00100000:3.0:1789760798.691732:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741917546327 time 741917285911 state 0x1 ref 1 00000100:00100000:3.0:1789760798.691762:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741917546327 time 741937380579 state 0x0 ref 1 00000100:00100000:3.0:1789760798.691787:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3786dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554920064:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760798.691838:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68866048, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x20d80 00010000:00010000:3.0:1789760798.691847:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c75f1 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.691862:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:3.0:1789760798.691866:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.691873:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c75f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68866048->68870143] (req 68866048->68870143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.691887:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c75f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68866048->68870143] (req 68866048->68870143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760798.699760:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984122 is committed 00002000:00010000:1.0:1789760798.702205:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68866048 -> 68870144 00002000:00010000:1.0:1789760798.702215:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134528 -> 134536 00010000:00010000:1.0:1789760798.702228:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c75f1 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68866048->68870143] (req 68866048->68870143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.702241:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c75f1 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68866048->68870143] (req 68866048->68870143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.702250:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c75f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68866048->68870143] (req 68866048->68870143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.702258:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a6400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760798.702263:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c75f1 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68866048->68870143] (req 68866048->68870143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.702272:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c75f1 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68866048->68870143] (req 68866048->68870143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.702293:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c75f1 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68866048->68870143] (req 68866048->68870143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.702301:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005ed923d2/0x1f99687d854c75f1 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68866048->68870143] (req 68866048->68870143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.702538:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3786dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554920064:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10757us (11011us total) trans 4294984122 rc 0/0 00000100:00100000:1.0:1789760798.702549:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16824 00000100:00100000:1.0:1789760798.702573:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741917546327 time 741937380579 state 0x2 ref 0 00000100:00100000:1.0:1789760798.711669:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760798.711768:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554920192 00000100:00100000:0.0:1789760798.711789:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554920192 00000100:00100000:0.0:1789760798.711850:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741917546327 time 741937380579 state 0x0 ref 1 00000100:00100000:0.0:1789760798.711867:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741937640995 time 741937380579 state 0x1 ref 1 00000100:00100000:0.0:1789760798.711898:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741937640995 time 741957517184 state 0x0 ref 1 00000100:00100000:0.0:1789760798.711910:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ccb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554920192:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760798.711964:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68870144, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x20d88 00010000:00010000:0.0:1789760798.711972:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c75f8 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.712198:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:0.0:1789760798.712205:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.712212:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c75f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68870144->68874239] (req 68870144->68874239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.712228:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c75f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68870144->68874239] (req 68870144->68874239) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789760798.723159:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984123 is committed 00002000:00010000:0.0:1789760798.723300:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68870144 -> 68874240 00002000:00010000:0.0:1789760798.723305:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134536 -> 134544 00010000:00010000:0.0:1789760798.723400:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c75f8 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68870144->68874239] (req 68870144->68874239) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.723415:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c75f8 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68870144->68874239] (req 68870144->68874239) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.723424:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c75f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68870144->68874239] (req 68870144->68874239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.723431:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe1c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760798.723436:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c75f8 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68870144->68874239] (req 68870144->68874239) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.723446:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c75f8 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68870144->68874239] (req 68870144->68874239) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.723466:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c75f8 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68870144->68874239] (req 68870144->68874239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.723474:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000002bac6d6/0x1f99687d854c75f8 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68870144->68874239] (req 68870144->68874239) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.723589:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ccb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554920192:12345-192.168.203.22@tcp:4:dd.0 Request processed in 11677us (11928us total) trans 4294984123 rc 0/0 00000100:00100000:0.0:1789760798.723600:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16825 00000100:00100000:0.0:1789760798.723610:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741937640995 time 741957517184 state 0x2 ref 0 00000100:00100000:0.0:1789760798.728582:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760798.728703:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554920320 00000100:00100000:2.0:1789760798.728713:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554920320 00000100:00100000:2.0:1789760798.728767:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741937640995 time 741957517184 state 0x0 ref 1 00000100:00100000:2.0:1789760798.728851:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741957777600 time 741957517184 state 0x1 ref 1 00000100:00100000:2.0:1789760798.728882:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741957777600 time 741974500596 state 0x0 ref 1 00000100:00100000:2.0:1789760798.728895:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554920320:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760798.728958:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68874240, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x20d90 00010000:00010000:2.0:1789760798.728967:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c75ff lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.728984:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=741s v=(0 0) 00010000:00010000:2.0:1789760798.729177:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.729184:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c75ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68874240->68878335] (req 68874240->68878335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.729199:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c75ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68874240->68878335] (req 68874240->68878335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.739082:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984124 is committed 00002000:00010000:2.0:1789760798.739179:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68874240 -> 68878336 00002000:00010000:2.0:1789760798.739184:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134544 -> 134552 00010000:00010000:2.0:1789760798.739196:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c75ff lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68874240->68878335] (req 68874240->68878335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.739215:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c75ff lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68874240->68878335] (req 68874240->68878335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.739223:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c75ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68874240->68878335] (req 68874240->68878335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.739230:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9200 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760798.739235:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c75ff lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68874240->68878335] (req 68874240->68878335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.739243:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c75ff lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68874240->68878335] (req 68874240->68878335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.739260:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c75ff lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68874240->68878335] (req 68874240->68878335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.739268:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015d74282/0x1f99687d854c75ff lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68874240->68878335] (req 68874240->68878335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.739606:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554920320:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10706us (11028us total) trans 4294984124 rc 0/0 00000100:00100000:2.0:1789760798.739615:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16826 00000100:00100000:2.0:1789760798.739624:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741957777600 time 741974500596 state 0x2 ref 0 00000100:00100000:2.0:1789760798.755754:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:3.0:1789760798.755879:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554920448 00000100:00100000:3.0:1789760798.755889:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554920448 00000100:00100000:3.0:1789760798.756095:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741957777600 time 741974500596 state 0x0 ref 1 00000100:00100000:3.0:1789760798.756111:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741974761012 time 741974500596 state 0x1 ref 1 00000100:00100000:3.0:1789760798.756178:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741974761012 time 742001796255 state 0x0 ref 1 00000100:00100000:3.0:1789760798.756191:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554920448:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:3.0:1789760798.756236:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68878336, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x20d98 00010000:00010000:3.0:1789760798.756245:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7606 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.756261:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=742s v=(0 0) 00010000:00010000:3.0:1789760798.756265:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789760798.756271:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7606 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68878336->68882431] (req 68878336->68882431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789760798.756286:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7606 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68878336->68882431] (req 68878336->68882431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.762486:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984125 is committed 00002000:00010000:2.0:1789760798.771660:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68878336 -> 68882432 00002000:00010000:2.0:1789760798.771670:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134552 -> 134560 00010000:00010000:2.0:1789760798.771684:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7606 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68878336->68882431] (req 68878336->68882431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.771697:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7606 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68878336->68882431] (req 68878336->68882431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.771705:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7606 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68878336->68882431] (req 68878336->68882431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.771712:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc3a1a7400 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760798.771717:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7606 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68878336->68882431] (req 68878336->68882431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.771725:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7606 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68878336->68882431] (req 68878336->68882431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.771745:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7606 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68878336->68882431] (req 68878336->68882431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.771753:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000246df114/0x1f99687d854c7606 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68878336->68882431] (req 68878336->68882431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.772646:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554920448:12345-192.168.203.22@tcp:4:dd.0 Request processed in 16446us (16891us total) trans 4294984125 rc 0/0 00000100:00100000:2.0:1789760798.772659:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16827 00000100:00100000:2.0:1789760798.772669:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741974761012 time 742001796255 state 0x2 ref 0 00000100:00100000:1.0:1789760798.777231:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760798.777336:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554920576 00000100:00100000:0.0:1789760798.777345:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554920576 00000100:00100000:0.0:1789760798.777428:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 741974761012 time 742001796255 state 0x0 ref 1 00000100:00100000:0.0:1789760798.777445:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 742002056671 time 742001796255 state 0x1 ref 1 00000100:00100000:0.0:1789760798.777471:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 742002056671 time 742023089880 state 0x0 ref 1 00000100:00100000:0.0:1789760798.777483:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03cc8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554920576:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760798.777531:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68882432, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x20da0 00010000:00010000:0.0:1789760798.777539:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c760d lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.777572:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=742s v=(0 0) 00010000:00010000:0.0:1789760798.777578:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.777584:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c760d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68882432->68886527] (req 68882432->68886527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.777603:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c760d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68882432->68886527] (req 68882432->68886527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760798.784740:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984126 is committed 00002000:00010000:0.0:1789760798.785059:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68882432 -> 68886528 00002000:00010000:0.0:1789760798.785064:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134560 -> 134568 00010000:00010000:0.0:1789760798.785078:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c760d lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68882432->68886527] (req 68882432->68886527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.785091:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c760d lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68882432->68886527] (req 68882432->68886527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.785100:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c760d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68882432->68886527] (req 68882432->68886527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.785107:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0a00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760798.785112:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c760d lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68882432->68886527] (req 68882432->68886527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.785121:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c760d lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68882432->68886527] (req 68882432->68886527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.785141:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c760d lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68882432->68886527] (req 68882432->68886527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.785148:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6224a1d/0x1f99687d854c760d lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68882432->68886527] (req 68882432->68886527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.785246:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03cc8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554920576:12345-192.168.203.22@tcp:4:dd.0 Request processed in 7759us (8019us total) trans 4294984126 rc 0/0 00000100:00100000:0.0:1789760798.785257:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16828 00000100:00100000:0.0:1789760798.785266:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 742002056671 time 742023089880 state 0x2 ref 0 00000100:00100000:0.0:1789760798.792285:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:2.0:1789760798.794708:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554920704 00000100:00100000:2.0:1789760798.794723:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554920704 00000100:00100000:2.0:1789760798.794795:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 742002056671 time 742023089880 state 0x0 ref 1 00000100:00100000:2.0:1789760798.794812:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 742023350296 time 742023089880 state 0x1 ref 1 00000100:00100000:2.0:1789760798.794848:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 742023350296 time 742040466684 state 0x0 ref 1 00000100:00100000:2.0:1789760798.794861:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a385f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554920704:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:2.0:1789760798.794906:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68886528, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x20da8 00010000:00010000:2.0:1789760798.794914:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7614 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.794929:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=742s v=(0 0) 00010000:00010000:2.0:1789760798.794943:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789760798.794950:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7614 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68886528->68890623] (req 68886528->68890623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.794963:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7614 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68886528->68890623] (req 68886528->68890623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760798.802085:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984127 is committed 00002000:00010000:2.0:1789760798.803344:0:8442:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68886528 -> 68890624 00002000:00010000:2.0:1789760798.803351:0:8442:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134568 -> 134576 00010000:00010000:2.0:1789760798.803369:0:8442:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7614 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68886528->68890623] (req 68886528->68890623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.803405:0:8442:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7614 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68886528->68890623] (req 68886528->68890623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.803415:0:8442:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7614 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68886528->68890623] (req 68886528->68890623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.803423:0:8442:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cc421f9c00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:2.0:1789760798.803428:0:8442:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7614 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68886528->68890623] (req 68886528->68890623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.803437:0:8442:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7614 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68886528->68890623] (req 68886528->68890623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.803457:0:8442:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7614 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68886528->68890623] (req 68886528->68890623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789760798.803465:0:8442:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e98910f1/0x1f99687d854c7614 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68886528->68890623] (req 68886528->68890623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789760798.803790:0:8442:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a385f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554920704:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8917us (11502us total) trans 4294984127 rc 0/0 00000100:00100000:2.0:1789760798.803802:0:8442:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16829 00000100:00100000:2.0:1789760798.803812:0:8442:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 742023350296 time 742040466684 state 0x2 ref 0 00000100:00100000:2.0:1789760798.815570:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760798.815721:0:8443:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554920832 00000100:00100000:1.0:1789760798.815731:0:8443:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554920832 00000100:00100000:1.0:1789760798.816866:0:8443:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 742023350296 time 742040466684 state 0x0 ref 1 00000100:00100000:1.0:1789760798.816886:0:8443:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 742040727100 time 742040466684 state 0x1 ref 1 00000100:00100000:1.0:1789760798.817674:0:8443:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 742040727100 time 742063286472 state 0x0 ref 1 00000100:00100000:1.0:1789760798.817697:0:8443:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554920832:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760798.817752:0:8443:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68890624, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x20db0 00010000:00010000:1.0:1789760798.817762:0:8443:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c761b lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.817777:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760798.817782:0:8443:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.817788:0:8443:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c761b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68890624->68894719] (req 68890624->68894719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.817802:0:8443:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c761b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68890624->68894719] (req 68890624->68894719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789760798.827113:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984128 is committed 00002000:00010000:1.0:1789760798.827454:0:8443:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68890624 -> 68894720 00002000:00010000:1.0:1789760798.827460:0:8443:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134576 -> 134584 00010000:00010000:1.0:1789760798.827474:0:8443:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c761b lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68890624->68894719] (req 68890624->68894719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.827488:0:8443:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c761b lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68890624->68894719] (req 68890624->68894719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.827495:0:8443:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c761b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68890624->68894719] (req 68890624->68894719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.827502:0:8443:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760798.827507:0:8443:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c761b lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68890624->68894719] (req 68890624->68894719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.827516:0:8443:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c761b lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68890624->68894719] (req 68890624->68894719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.827535:0:8443:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c761b lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68890624->68894719] (req 68890624->68894719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.827542:0:8443:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c761b lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68890624->68894719] (req 68890624->68894719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8443 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.828026:0:8443:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc1d94d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554920832:12345-192.168.203.22@tcp:4:dd.0 Request processed in 10333us (12474us total) trans 4294984128 rc 0/0 00000100:00100000:1.0:1789760798.828037:0:8443:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16830 00000100:00100000:1.0:1789760798.828047:0:8443:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 742040727100 time 742063286472 state 0x2 ref 0 00000100:00100000:1.0:1789760798.833611:0:3641:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760798.834204:0:15101:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554920960 00000100:00100000:0.0:1789760798.834216:0:15101:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554920960 00000100:00100000:0.0:1789760798.834271:0:15101:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 742040727100 time 742063286472 state 0x0 ref 1 00000100:00100000:0.0:1789760798.834286:0:15101:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 742063546888 time 742063286472 state 0x1 ref 1 00000100:00100000:0.0:1789760798.834321:0:15101:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 742063546888 time 742079939559 state 0x0 ref 1 00000100:00100000:0.0:1789760798.834333:0:15101:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cb03ccb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554920960:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760798.834423:0:15101:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68894720, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x20db8 00010000:00010000:0.0:1789760798.834432:0:15101:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c7622 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.834449:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=742s v=(0 0) 00010000:00010000:0.0:1789760798.834453:0:15101:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.834459:0:15101:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c7622 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68894720->68898815] (req 68894720->68898815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.834479:0:15101:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c7622 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68894720->68898815] (req 68894720->68898815) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760798.842775:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984129 is committed 00002000:00010000:0.0:1789760798.842902:0:15101:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68894720 -> 68898816 00002000:00010000:0.0:1789760798.842907:0:15101:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134584 -> 134592 00010000:00010000:0.0:1789760798.842920:0:15101:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c7622 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68894720->68898815] (req 68894720->68898815) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.842988:0:15101:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c7622 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68894720->68898815] (req 68894720->68898815) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.842999:0:15101:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c7622 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68894720->68898815] (req 68894720->68898815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.843007:0:15101:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb04fe0000 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:0.0:1789760798.843012:0:15101:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c7622 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68894720->68898815] (req 68894720->68898815) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.843022:0:15101:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c7622 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68894720->68898815] (req 68894720->68898815) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.843042:0:15101:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c7622 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68894720->68898815] (req 68894720->68898815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.843049:0:15101:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dcf29214/0x1f99687d854c7622 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68894720->68898815] (req 68894720->68898815) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 15101 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789760798.843137:0:15101:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cb03ccb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2413:x1876699554920960:12345-192.168.203.22@tcp:4:dd.0 Request processed in 8802us (9535us total) trans 4294984129 rc 0/0 00000100:00100000:0.0:1789760798.843146:0:15101:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16831 00000100:00100000:0.0:1789760798.843155:0:15101:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 742063546888 time 742079939559 state 0x2 ref 0 00000100:00100000:0.0:1789760798.851553:0:3639:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:1.0:1789760798.851666:0:8441:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554921088 00000100:00100000:1.0:1789760798.851676:0:8441:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554921088 00000100:00100000:1.0:1789760798.851740:0:8441:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 742063546888 time 742079939559 state 0x0 ref 1 00000100:00100000:1.0:1789760798.851756:0:8441:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 742080199975 time 742079939559 state 0x1 ref 1 00000100:00100000:1.0:1789760798.851791:0:8441:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 742080199975 time 742097410021 state 0x0 ref 1 00000100:00100000:1.0:1789760798.851803:0:8441:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc3a387480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554921088:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:1.0:1789760798.851850:0:8441:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68898816, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x20dc0 00010000:00010000:1.0:1789760798.851860:0:8441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7629 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.851875:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=742s v=(0 0) 00010000:00010000:1.0:1789760798.851880:0:8441:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789760798.851886:0:8441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7629 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68898816->68902911] (req 68898816->68902911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.851900:0:8441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7629 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68898816->68902911] (req 68898816->68902911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789760798.860242:0:8430:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294984130 is committed 00002000:00010000:1.0:1789760798.860896:0:8441:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb size from disk: 68898816 -> 68902912 00002000:00010000:1.0:1789760798.860902:0:8441:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000401:0x2:0x0] updating lvb blocks from disk: 134592 -> 134600 00010000:00010000:1.0:1789760798.860915:0:8441:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7629 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68898816->68902911] (req 68898816->68902911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.861042:0:8441:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7629 lrc: 2/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68898816->68902911] (req 68898816->68902911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.861053:0:8441:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7629 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68898816->68902911] (req 68898816->68902911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.861060:0:8441:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff91cb0655dc00 already unused, calling callback (ffffffffc0ba8350) 00010000:00010000:1.0:1789760798.861064:0:8441:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7629 lrc: 3/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68898816->68902911] (req 68898816->68902911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.861072:0:8441:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7629 lrc: 4/0,0 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68898816->68902911] (req 68898816->68902911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.861090:0:8441:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7629 lrc: 2/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68898816->68902911] (req 68898816->68902911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789760798.861097:0:8441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000befcc122/0x1f99687d854c7629 lrc: 0/0,0 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68898816->68902911] (req 68898816->68902911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789760798.861227:0:8441:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff91cc3a387480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2414:x1876699554921088:12345-192.168.203.22@tcp:4:dd.0 Request processed in 9421us (9678us total) trans 4294984130 rc 0/0 00000100:00100000:1.0:1789760798.861237:0:8441:0:(nrs_tbf.c:3377:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.22@tcp, seq: 16832 00000100:00100000:1.0:1789760798.861247:0:8441:0:(nrs_tbf.c:1542:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 742080199975 time 742097410021 state 0x2 ref 0 00000100:00100000:2.0:1789760798.866019:0:3640:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.22@tcp (source: 12345-192.168.203.22@tcp) 00000100:00100000:0.0:1789760798.866145:0:8442:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876699554921216 00000100:00100000:0.0:1789760798.866155:0:8442:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876699554921216 00000100:00100000:0.0:1789760798.866205:0:8442:0:(nrs_tbf.c:1562:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 742080199975 time 742097410021 state 0x0 ref 1 00000100:00100000:0.0:1789760798.866220:0:8442:0:(nrs_tbf.c:3322:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 742097670437 time 742097410021 state 0x1 ref 1 00000100:00100000:0.0:1789760798.866252:0:8442:0:(nrs_tbf.c:3136:nrs_req_dequeue_bps()) ost_io.0 NRS: TBF dequeues in BPS mode: class@ffff91cb03741600 key nid={192.168.203.22@tcp}&opcode={ost_write} rule@0xffff91cb0314b600 gen 0 rate 15 depth 3 token 3 deadline 742097670437 time 742111870326 state 0x0 ref 1 00000100:00100000:0.0:1789760798.866264:0:8442:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff91cc1d94c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:780dc4e4-6b5c-4c38-8004-c70d2f08ee5f+5:2412:x1876699554921216:12345-192.168.203.22@tcp:4:dd.0 00002000:00010000:0.0:1789760798.866311:0:8442:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000401:0x2:0x0] initial LVB size: 68902912, mtime: 0x6aad951e, atime: 0x0, ctime: 0x6aad951e, blocks: 0x20dc8 00010000:00010000:0.0:1789760798.866330:0:8442:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c7630 lrc: 3/0,1 mode: --/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.866344:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=742s v=(0 0) 00010000:00010000:0.0:1789760798.866348:0:8442:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000401:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789760798.866354:0:8442:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c7630 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68902912->68907007] (req 68902912->68907007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789760798.866366:0:8442:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000273ad2a3/0x1f99687d854c7630 lrc: 3/0,1 mode: PW/PW res: [0x280000401:0x2:0x0].0x0 rrc: 2 type: EXT [68902912->68907007] (req 68902912->68907007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8442 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 Debug log: 57232 lines, 57232 kept, 0 dropped, 0 bad.